Max level shown:
1 [ Dec 19 00:56:06 Enabled. ]
2 [ Dec 19 00:56:06 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Dec 19 00:56:06 Method "start" exited with status 0. ]
4 [ Dec 19 00:56:06 Rereading configuration. ]
5 [ Dec 19 00:56:06 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-12-19T00:56:06.092ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-12-19T00:56:06.092ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-12-19T00:56:06.143ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-12-19T00:56:06.143ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-12-19T00:56:06.258ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-12-19T00:56:06.267ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-12-19T00:56:06.275ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-12-19T00:56:06.878ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:a02a::, width: 64 }
152024-12-19T00:56:07.019ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-12-19T00:56:07.019ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-12-19T00:56:07.019ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-12-19T00:56:07.019ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192024-12-19T00:56:07.020ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
202024-12-19T00:56:07.020ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:717
212024-12-19T00:56:07.022ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "sock", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:624
222024-12-19T00:56:07.022ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:766
232024-12-19T00:56:07.022ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-12-19T00:56:07.022ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:825
252024-12-19T00:56:07.022ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-12-19T00:56:07.023ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-12-19T00:56:07.023ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
282024-12-19T00:56:07.023ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-12-19T00:56:07.023ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-12-19T00:56:07.035ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:367
312024-12-19T00:56:07.035ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool)
322024-12-19T00:56:07.156ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool), kind: Internal }
332024-12-19T00:56:07.766ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:603
342024-12-19T00:56:07.766ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-12-19T00:56:07.766ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-12-19T00:56:07.766ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-12-19T00:56:07.766ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
382024-12-19T00:56:07.766ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-12-19T00:56:07.766ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-12-19T00:56:07.775ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:367
412024-12-19T00:56:07.775ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool)
422024-12-19T00:56:07.883ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool), kind: Internal }
432024-12-19T00:56:08.477ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442024-12-19T00:56:08.477ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-12-19T00:56:08.477ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
462024-12-19T00:56:08.477ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
472024-12-19T00:56:08.477ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
482024-12-19T00:56:08.477ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
492024-12-19T00:56:08.477ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
502024-12-19T00:56:08.477ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
512024-12-19T00:56:08.477ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
522024-12-19T00:56:08.478ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
532024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
542024-12-19T00:56:08.478ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
552024-12-19T00:56:08.478ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562024-12-19T00:56:08.478ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
572024-12-19T00:56:08.478ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
582024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
592024-12-19T00:56:08.478ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
602024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
612024-12-19T00:56:08.478ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622024-12-19T00:56:08.478ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
632024-12-19T00:56:08.478ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
642024-12-19T00:56:08.478ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
652024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
662024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
672024-12-19T00:56:08.478ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682024-12-19T00:56:08.478ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
692024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
702024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
712024-12-19T00:56:08.478ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
722024-12-19T00:56:08.478ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
732024-12-19T00:56:08.478ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742024-12-19T00:56:08.478ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
752024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
762024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
772024-12-19T00:56:08.478ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
782024-12-19T00:56:08.478ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
792024-12-19T00:56:08.478ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802024-12-19T00:56:08.478ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
812024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
822024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
832024-12-19T00:56:08.478ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
842024-12-19T00:56:08.478ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
852024-12-19T00:56:08.478ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
862024-12-19T00:56:08.478ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
872024-12-19T00:56:08.478ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
882024-12-19T00:56:08.478ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
892024-12-19T00:56:08.478ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
902024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
912024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
922024-12-19T00:56:08.478ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
932024-12-19T00:56:08.478ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
942024-12-19T00:56:08.478ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
952024-12-19T00:56:08.478ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
962024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
972024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
982024-12-19T00:56:08.478ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
992024-12-19T00:56:08.478ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1002024-12-19T00:56:08.478ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1012024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1022024-12-19T00:56:08.478ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1032024-12-19T00:56:08.478ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1042024-12-19T00:56:08.478ZINFOSledAgent (bootstore): No ledger in /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1052024-12-19T00:56:08.478ZINFOSledAgent (bootstore): No ledger in /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1062024-12-19T00:56:08.478ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-sock-i86pc
1072024-12-19T00:56:08.478ZINFOSledAgent (bootstore): No ledger in /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1082024-12-19T00:56:08.478ZINFOSledAgent (bootstore): No ledger in /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1092024-12-19T00:56:08.478ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-sock-i86pc
1102024-12-19T00:56:08.478ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112024-12-19T00:56:08.478ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122024-12-19T00:56:08.489ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:737
1132024-12-19T00:56:08.489ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1257, tv_nsec: 105510190 } time_to_next_cleanup = 599.999910858s
1142024-12-19T00:56:08.489ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3962
1152024-12-19T00:56:08.489ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1162024-12-19T00:56:08.489ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-12-19T00:56:08.492ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-12-19T00:56:08.493ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
1192024-12-19T00:56:08.494ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:51 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
1202024-12-19T00:56:08.495ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some( None, ), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1187
1212024-12-19T00:56:08.495ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1222024-12-19T00:56:08.495ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1232024-12-19T00:56:08.495ZINFOSledAgent (RSS): No ledger in /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1242024-12-19T00:56:08.495ZINFOSledAgent (RSS): No ledger in /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1252024-12-19T00:56:08.495ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:1250
1262024-12-19T00:56:08.495ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1272024-12-19T00:56:08.495ZINFOSledAgent (RSS): No ledger in /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/rss-sled-plan.json file = common/src/ledger.rs:188
1282024-12-19T00:56:08.495ZINFOSledAgent (RSS): No ledger in /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/rss-sled-plan.json file = common/src/ledger.rs:188
1292024-12-19T00:56:08.495ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1292
1302024-12-19T00:56:08.495ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:156
1312024-12-19T00:56:08.495ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:129
1322024-12-19T00:56:08.495ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1332024-12-19T00:56:08.495ZINFOSledAgent (RSS): Writing ledger to /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1342024-12-19T00:56:08.496ZINFOSledAgent (RSS): Writing ledger to /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1352024-12-19T00:56:08.496ZINFOSledAgent (RSS): Sled plan written to storage: Plan { rack_id: 6f1cca69-e82a-429e-8f71-b2c5888a42c1, sleds: { [fdb0:18c0:4d0d:a02a::1]:12346: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: de9bec7d-f8ce-44a0-879c-5e088768f91c (sled), rack_id: 6f1cca69-e82a-429e-8f71-b2c5888a42c1, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64, }, }, }, }, }, config: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some( None, ), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, }, } file = sled-agent/src/rack_setup/plan/sled.rs:176
1362024-12-19T00:56:08.496ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1326
1372024-12-19T00:56:08.496ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
1382024-12-19T00:56:08.496ZINFOSledAgent (bootstore): Writing ledger to /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1392024-12-19T00:56:08.496ZINFOSledAgent (bootstore): Writing ledger to /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1402024-12-19T00:56:08.496ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
1412024-12-19T00:56:08.496ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:232 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: de9bec7d-f8ce-44a0-879c-5e088768f91c (sled), rack_id: 6f1cca69-e82a-429e-8f71-b2c5888a42c1, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1422024-12-19T00:56:08.497ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4259
1432024-12-19T00:56:08.497ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4659
1442024-12-19T00:56:08.501ZINFOSledAgent (SledAgentSprocketsClient): Loaded keys and certs BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/client.rs:50
1452024-12-19T00:56:08.507ZINFOSledAgent (SledAgentSprocketsClient): Certificate chain verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1462024-12-19T00:56:08.507ZINFOSledAgent (SledAgentSprocketsClient): Signature verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1472024-12-19T00:56:08.512ZINFOSledAgent (SledAgentSprocketsServer): Certificate chain verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1482024-12-19T00:56:08.512ZINFOSledAgent (SledAgentSprocketsServer): Signature verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1492024-12-19T00:56:08.512ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:74 remote_addr = [fdb0:18c0:4d0d:a02a::1]:42403
1502024-12-19T00:56:08.512ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: de9bec7d-f8ce-44a0-879c-5e088768f91c (sled), rack_id: 6f1cca69-e82a-429e-8f71-b2c5888a42c1, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:362
1512024-12-19T00:56:08.512ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:377
1522024-12-19T00:56:08.512ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1532024-12-19T00:56:08.512ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1542024-12-19T00:56:08.512ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:603
1552024-12-19T00:56:08.512ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:809
1562024-12-19T00:56:08.512ZINFOSledAgent (StorageManager): No ledger in /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1572024-12-19T00:56:08.512ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1582024-12-19T00:56:08.512ZINFOSledAgent (StorageManager): No ledger in /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1592024-12-19T00:56:08.512ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:798
1602024-12-19T00:56:08.512ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:826
1612024-12-19T00:56:08.512ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:592
1622024-12-19T00:56:08.512ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1632024-12-19T00:56:08.512ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:697 result = DisksManagementResult { status: [] }
1642024-12-19T00:56:08.530ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1652024-12-19T00:56:08.551ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:325
1662024-12-19T00:56:08.569ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:363
1672024-12-19T00:56:08.594ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:415 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
1682024-12-19T00:56:08.594ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1692024-12-19T00:56:08.594ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:427 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
1702024-12-19T00:56:08.594ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1712024-12-19T00:56:08.597ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 143.140353ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "37185154-d3f0-4e7a-aa52-10280b353c37", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "37185154-d3f0-4e7a-aa52-10280b353c37" } file = clients/ddm-admin-client/src/lib.rs:177
1722024-12-19T00:56:08.607ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_8fd565e7-8dbe-4e0e-b9f6-62127926c764/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1732024-12-19T00:56:08.636ZINFOSledAgent: successfully created encrypted zvol: name="oxi_8fd565e7-8dbe-4e0e-b9f6-62127926c764/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1742024-12-19T00:56:08.636ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_8fd565e7-8dbe-4e0e-b9f6-62127926c764/swap" file = sled-agent/src/swap_device.rs:96
1752024-12-19T00:56:08.641ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:442 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
1762024-12-19T00:56:08.641ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:128
1772024-12-19T00:56:08.641ZINFOSledAgent (BackingFs): Ensuring dataset oxi_8fd565e7-8dbe-4e0e-b9f6-62127926c764/backing/fmd file = sled-agent/src/backing_fs.rs:138
1782024-12-19T00:56:08.742ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1792024-12-19T00:56:08.742ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 91.845828ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "7a2e800d-57a8-4c22-ab26-5da8d53dc0a1", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "7a2e800d-57a8-4c22-ab26-5da8d53dc0a1" } file = clients/ddm-admin-client/src/lib.rs:177
1802024-12-19T00:56:08.748ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:173
1812024-12-19T00:56:08.836ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1822024-12-19T00:56:08.837ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 73.175817ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "14c5540f-0fb1-40ce-902e-1a02e5b8aa75", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "14c5540f-0fb1-40ce-902e-1a02e5b8aa75" } file = clients/ddm-admin-client/src/lib.rs:177
1832024-12-19T00:56:08.911ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1842024-12-19T00:56:08.911ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 160.984344ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "7260fd10-ae68-4ca4-a5e1-da25e187159e", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "7260fd10-ae68-4ca4-a5e1-da25e187159e" } file = clients/ddm-admin-client/src/lib.rs:177
1852024-12-19T00:56:08.968ZINFOSledAgent (BackingFs): Mounting oxi_8fd565e7-8dbe-4e0e-b9f6-62127926c764/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:181
1862024-12-19T00:56:08.980ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:189
1872024-12-19T00:56:08.981ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:189
1882024-12-19T00:56:08.981ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:189
1892024-12-19T00:56:08.981ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:195
1902024-12-19T00:56:09.073ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1912024-12-19T00:56:09.074ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 272.573445ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "aa8d62e0-364b-415d-b857-405ccc31be5d", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "aa8d62e0-364b-415d-b857-405ccc31be5d" } file = clients/ddm-admin-client/src/lib.rs:177
1922024-12-19T00:56:09.347ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1932024-12-19T00:56:09.347ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 171.2332ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9df822ff-63e6-4e57-9489-a8b58ce6b340", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9df822ff-63e6-4e57-9489-a8b58ce6b340" } file = clients/ddm-admin-client/src/lib.rs:177
1942024-12-19T00:56:09.519ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1952024-12-19T00:56:09.520ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 164.153878ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "b6701005-9a22-49bd-889d-58f1264a64a8", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "b6701005-9a22-49bd-889d-58f1264a64a8" } file = clients/ddm-admin-client/src/lib.rs:177
1962024-12-19T00:56:09.685ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1972024-12-19T00:56:09.685ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 457.614659ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "32d83c07-7563-4463-9354-5ecd5a590b4f", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "32d83c07-7563-4463-9354-5ecd5a590b4f" } file = clients/ddm-admin-client/src/lib.rs:177
1982024-12-19T00:56:09.741ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:93 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
1992024-12-19T00:56:09.746ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:37755 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2002024-12-19T00:56:09.746ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2012024-12-19T00:56:09.746ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:37755 file = oximeter/producer/src/lib.rs:279 interval = 30s producer_id = de9bec7d-f8ce-44a0-879c-5e088768f91c sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2022024-12-19T00:56:09.747ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 372.277238ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2032024-12-19T00:56:09.748ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2042024-12-19T00:56:09.748ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2052024-12-19T00:56:09.748ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:921 underlay_address = fd00:1122:3344:101::1
2062024-12-19T00:56:09.748ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
2072024-12-19T00:56:10.121ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 262.596985ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2082024-12-19T00:56:10.144ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2092024-12-19T00:56:10.385ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 905.721823ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2102024-12-19T00:56:11.292ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.869326894s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2112024-12-19T00:56:12.360ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2683
2122024-12-19T00:56:12.360ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2928
2132024-12-19T00:56:12.360ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2553
2142024-12-19T00:56:12.360ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2600
2152024-12-19T00:56:12.360ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:3009
2162024-12-19T00:56:12.360ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:3054
2172024-12-19T00:56:12.361ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2853
2182024-12-19T00:56:12.363ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2594
2192024-12-19T00:56:12.363ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='51ab14f8-f146-4958-aacf-1a2334aec785'/> <propval type="astring" name="rack_id" value='6f1cca69-e82a-429e-8f71-b2c5888a42c1'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:a02a::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fe7e:3f06'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:a02a::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"sock","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="rack_id" value='6f1cca69-e82a-429e-8f71-b2c5888a42c1'/> <propval type="astring" name="sled_id" value='de9bec7d-f8ce-44a0-879c-5e088768f91c'/> <propval type="astring" name="sled_model" value='i86pc'/> <propval type="astring" name="sled_serial" value='sock'/> <propval type="astring" name="sled_revision" value='0'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='sock'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='de9bec7d-f8ce-44a0-879c-5e088768f91c'/> <propval type="astring" name="rack_uuid" value='6f1cca69-e82a-429e-8f71-b2c5888a42c1'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='de9bec7d-f8ce-44a0-879c-5e088768f91c'/> <propval type="astring" name="rack_uuid" value='6f1cca69-e82a-429e-8f71-b2c5888a42c1'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2202024-12-19T00:56:12.363ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2212024-12-19T00:56:12.695ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 53.766542ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222024-12-19T00:56:12.902ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 72.268692ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2232024-12-19T00:56:13.085ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 134.127729ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242024-12-19T00:56:13.164ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 5.24196168s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2252024-12-19T00:56:13.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 427.458328ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2262024-12-19T00:56:13.727ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 898.629246ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2272024-12-19T00:56:14.664ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 750.855455ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2282024-12-19T00:56:15.609ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.463312294s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2292024-12-19T00:56:17.200ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.418728629s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2302024-12-19T00:56:18.408ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 7.773168159s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2312024-12-19T00:56:18.930ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/nexus.rs:527 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2322024-12-19T00:56:18.960ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2332024-12-19T00:56:18.961ZINFOSledAgent (dropshot (Repo Depot)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12348 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2342024-12-19T00:56:18.962ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12345
2352024-12-19T00:56:18.962ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:432
2362024-12-19T00:56:18.962ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2372024-12-19T00:56:18.962ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2382024-12-19T00:56:18.962ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2392024-12-19T00:56:18.962ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2402024-12-19T00:56:18.962ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some(None), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None }], bgp: [], bfd: [] })))
2412024-12-19T00:56:18.962ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3962
2422024-12-19T00:56:18.962ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:81 remote_addr = [fdb0:18c0:4d0d:a02a::1]:42403
2432024-12-19T00:56:19.188ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:117 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2442024-12-19T00:56:19.188ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:252 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2452024-12-19T00:56:19.188ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2462024-12-19T00:56:19.188ZINFOSledAgent (RSS): No ledger in /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/rss-service-plan-v5.json file = common/src/ledger.rs:188
2472024-12-19T00:56:19.188ZINFOSledAgent (RSS): No ledger in /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/rss-service-plan-v5.json file = common/src/ledger.rs:188
2482024-12-19T00:56:19.188ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2492024-12-19T00:56:19.188ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2502024-12-19T00:56:19.188ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2512024-12-19T00:56:19.188ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2522024-12-19T00:56:19.194ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4283 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2532024-12-19T00:56:19.194ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4299
2542024-12-19T00:56:19.194ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2552024-12-19T00:56:19.487ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34187
2562024-12-19T00:56:19.487ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2572024-12-19T00:56:19.487ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2582024-12-19T00:56:19.488ZINFOSledAgent (ServiceManager): No ledger in /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:188
2592024-12-19T00:56:19.488ZINFOSledAgent (ServiceManager): No ledger in /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:188
2602024-12-19T00:56:19.488ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 404 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:34187 req_id = 67b828f5-981a-470c-93b9-4dacc0a03639 response_code = 200 uri = /inventory
2612024-12-19T00:56:19.518ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37903
2622024-12-19T00:56:19.518ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:37903 req_id = a6249094-b00d-4484-890f-71578da6c7d5 response_code = 200 uri = /sled-role
2632024-12-19T00:56:19.520ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2642024-12-19T00:56:19.520ZINFOSledAgent (RSS): Writing ledger to /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/.rss-service-plan-v5.json.tmp file = common/src/ledger.rs:199
2652024-12-19T00:56:19.520ZINFOSledAgent (RSS): Writing ledger to /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/.rss-service-plan-v5.json.tmp file = common/src/ledger.rs:199
2662024-12-19T00:56:19.521ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:982
2672024-12-19T00:56:19.559ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(2), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: ce62597e-2729-48a1-921b-4cac7f3531af (physical_disk), pool_id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 102e84b6-d999-4e2d-9197-2dbf4e24cc6f (physical_disk), pool_id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: ab630384-d0a9-4a31-b245-216dc4c0cae7 (physical_disk), pool_id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: a0a0d721-30df-4b06-a7fe-c9bc97fd50b7 (physical_disk), pool_id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: e90be939-df02-4310-9594-8d62cc391c8d (physical_disk), pool_id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: fe87d667-3785-43a2-9aa6-e0fc0f861fca (physical_disk), pool_id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 49219027-45c3-4bcf-a055-4fb7ccaf0cba (physical_disk), pool_id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 981b68b6-f98a-4251-965f-7550971ec76f (physical_disk), pool_id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 8b0a8190-15a3-4ef0-b57d-ed513e1b258a (physical_disk), pool_id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool) }] } file = sled-agent/src/rack_setup/service.rs:369 sled_address = [fd00:1122:3344:101::1]:12345
2682024-12-19T00:56:19.559ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63066
2692024-12-19T00:56:19.560ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:896 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2702024-12-19T00:56:19.560ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(2), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: ce62597e-2729-48a1-921b-4cac7f3531af (physical_disk), pool_id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 102e84b6-d999-4e2d-9197-2dbf4e24cc6f (physical_disk), pool_id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: ab630384-d0a9-4a31-b245-216dc4c0cae7 (physical_disk), pool_id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: a0a0d721-30df-4b06-a7fe-c9bc97fd50b7 (physical_disk), pool_id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: e90be939-df02-4310-9594-8d62cc391c8d (physical_disk), pool_id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: fe87d667-3785-43a2-9aa6-e0fc0f861fca (physical_disk), pool_id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 49219027-45c3-4bcf-a055-4fb7ccaf0cba (physical_disk), pool_id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 981b68b6-f98a-4251-965f-7550971ec76f (physical_disk), pool_id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 8b0a8190-15a3-4ef0-b57d-ed513e1b258a (physical_disk), pool_id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:603
2712024-12-19T00:56:19.560ZINFOSledAgent (StorageManager): No ledger in /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2722024-12-19T00:56:19.560ZINFOSledAgent (StorageManager): No ledger in /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2732024-12-19T00:56:19.560ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:1171 request = omicron_physical_disks_ensure
2742024-12-19T00:56:19.560ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2752024-12-19T00:56:19.560ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2762024-12-19T00:56:19.560ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2772024-12-19T00:56:19.560ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2782024-12-19T00:56:19.703ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:367
2792024-12-19T00:56:19.703ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = a6644750-441b-4297-9acc-8dd23a71b4cb (zpool)
2802024-12-19T00:56:19.912ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8023753382245910886 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2812024-12-19T00:56:19.912ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 7374314792341526415 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2822024-12-19T00:56:20.153ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2832024-12-19T00:56:20.365ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
2842024-12-19T00:56:20.422ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }
2852024-12-19T00:56:20.429ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:240
2862024-12-19T00:56:20.430ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:242
2872024-12-19T00:56:20.430ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:246
2882024-12-19T00:56:20.430ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:248
2892024-12-19T00:56:20.430ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2902024-12-19T00:56:20.430ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2912024-12-19T00:56:20.430ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
2922024-12-19T00:56:20.474ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2932024-12-19T00:56:20.483ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone file = sled-storage/src/dataset.rs:303
2942024-12-19T00:56:20.701ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_a6644750-441b-4297-9acc-8dd23a71b4cb file = sled-storage/src/dataset.rs:389
2952024-12-19T00:56:20.709ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2962024-12-19T00:56:20.709ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2972024-12-19T00:56:20.709ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2982024-12-19T00:56:20.709ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2992024-12-19T00:56:20.709ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
3002024-12-19T00:56:20.718ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:367
3012024-12-19T00:56:20.718ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool)
3022024-12-19T00:56:20.832ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }
3032024-12-19T00:56:20.840ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:240
3042024-12-19T00:56:20.840ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:242
3052024-12-19T00:56:20.840ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:246
3062024-12-19T00:56:20.840ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:248
3072024-12-19T00:56:20.840ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
3082024-12-19T00:56:20.840ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
3092024-12-19T00:56:20.840ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3102024-12-19T00:56:20.883ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3112024-12-19T00:56:20.890ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone file = sled-storage/src/dataset.rs:303
3122024-12-19T00:56:21.099ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa file = sled-storage/src/dataset.rs:389
3132024-12-19T00:56:21.106ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
3142024-12-19T00:56:21.106ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
3152024-12-19T00:56:21.106ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
3162024-12-19T00:56:21.106ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3172024-12-19T00:56:21.106ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3182024-12-19T00:56:21.116ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:367
3192024-12-19T00:56:21.116ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 93eb8a71-6519-482e-a366-b15e30b4605e (zpool)
3202024-12-19T00:56:21.231ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }
3212024-12-19T00:56:21.240ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:240
3222024-12-19T00:56:21.240ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:242
3232024-12-19T00:56:21.240ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:246
3242024-12-19T00:56:21.240ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:248
3252024-12-19T00:56:21.240ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3262024-12-19T00:56:21.240ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3272024-12-19T00:56:21.240ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3282024-12-19T00:56:21.243ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
3292024-12-19T00:56:21.287ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3302024-12-19T00:56:21.295ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_93eb8a71-6519-482e-a366-b15e30b4605e/crypt/zone file = sled-storage/src/dataset.rs:303
3312024-12-19T00:56:21.510ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_93eb8a71-6519-482e-a366-b15e30b4605e file = sled-storage/src/dataset.rs:389
3322024-12-19T00:56:21.518ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3332024-12-19T00:56:21.518ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3342024-12-19T00:56:21.518ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3352024-12-19T00:56:21.518ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3362024-12-19T00:56:21.518ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3372024-12-19T00:56:21.528ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:367
3382024-12-19T00:56:21.528ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 98a100ce-cd82-4209-8530-7b48ca829744 (zpool)
3392024-12-19T00:56:21.637ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }
3402024-12-19T00:56:21.644ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:240
3412024-12-19T00:56:21.644ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:242
3422024-12-19T00:56:21.644ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:246
3432024-12-19T00:56:21.644ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:248
3442024-12-19T00:56:21.644ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3452024-12-19T00:56:21.644ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3462024-12-19T00:56:21.644ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3472024-12-19T00:56:21.690ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3482024-12-19T00:56:21.698ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_98a100ce-cd82-4209-8530-7b48ca829744/crypt/zone file = sled-storage/src/dataset.rs:303
3492024-12-19T00:56:21.914ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_98a100ce-cd82-4209-8530-7b48ca829744 file = sled-storage/src/dataset.rs:389
3502024-12-19T00:56:21.921ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3512024-12-19T00:56:21.921ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3522024-12-19T00:56:21.921ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3532024-12-19T00:56:21.921ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3542024-12-19T00:56:21.922ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3552024-12-19T00:56:21.931ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:367
3562024-12-19T00:56:21.931ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool)
3572024-12-19T00:56:22.048ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }
3582024-12-19T00:56:22.056ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:240
3592024-12-19T00:56:22.056ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:242
3602024-12-19T00:56:22.056ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:246
3612024-12-19T00:56:22.056ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:248
3622024-12-19T00:56:22.056ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3632024-12-19T00:56:22.056ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3642024-12-19T00:56:22.056ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3652024-12-19T00:56:22.102ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3662024-12-19T00:56:22.110ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/zone file = sled-storage/src/dataset.rs:303
3672024-12-19T00:56:22.247ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
3682024-12-19T00:56:22.332ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6 file = sled-storage/src/dataset.rs:389
3692024-12-19T00:56:22.341ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3702024-12-19T00:56:22.341ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3712024-12-19T00:56:22.341ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3722024-12-19T00:56:22.341ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3732024-12-19T00:56:22.341ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3742024-12-19T00:56:22.350ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:367
3752024-12-19T00:56:22.350ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool)
3762024-12-19T00:56:22.464ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }
3772024-12-19T00:56:22.473ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:240
3782024-12-19T00:56:22.473ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:242
3792024-12-19T00:56:22.473ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:246
3802024-12-19T00:56:22.473ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:248
3812024-12-19T00:56:22.474ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3822024-12-19T00:56:22.474ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3832024-12-19T00:56:22.474ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3842024-12-19T00:56:22.520ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3852024-12-19T00:56:22.527ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb/crypt/zone file = sled-storage/src/dataset.rs:303
3862024-12-19T00:56:22.747ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb file = sled-storage/src/dataset.rs:389
3872024-12-19T00:56:22.755ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3882024-12-19T00:56:22.755ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3892024-12-19T00:56:22.755ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3902024-12-19T00:56:22.755ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3912024-12-19T00:56:22.755ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3922024-12-19T00:56:22.765ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:367
3932024-12-19T00:56:22.765ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = f4009784-7bde-44ff-854e-49d614e9e682 (zpool)
3942024-12-19T00:56:22.878ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }
3952024-12-19T00:56:22.886ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:240
3962024-12-19T00:56:22.886ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:242
3972024-12-19T00:56:22.886ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:246
3982024-12-19T00:56:22.886ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:248
3992024-12-19T00:56:22.886ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
4002024-12-19T00:56:22.886ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
4012024-12-19T00:56:22.886ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
4022024-12-19T00:56:22.929ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4032024-12-19T00:56:22.937ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_f4009784-7bde-44ff-854e-49d614e9e682/crypt/zone file = sled-storage/src/dataset.rs:303
4042024-12-19T00:56:23.154ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_f4009784-7bde-44ff-854e-49d614e9e682 file = sled-storage/src/dataset.rs:389
4052024-12-19T00:56:23.162ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
4062024-12-19T00:56:23.162ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
4072024-12-19T00:56:23.162ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
4082024-12-19T00:56:23.162ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4092024-12-19T00:56:23.162ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
4102024-12-19T00:56:23.171ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:367
4112024-12-19T00:56:23.171ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool)
4122024-12-19T00:56:23.282ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }
4132024-12-19T00:56:23.290ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:240
4142024-12-19T00:56:23.290ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:242
4152024-12-19T00:56:23.290ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:246
4162024-12-19T00:56:23.290ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:248
4172024-12-19T00:56:23.290ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4182024-12-19T00:56:23.290ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4192024-12-19T00:56:23.290ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
4202024-12-19T00:56:23.294ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
4212024-12-19T00:56:23.335ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4222024-12-19T00:56:23.343ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_1025d0d6-c165-4f43-bef0-cb2eeb04b8c2/crypt/zone file = sled-storage/src/dataset.rs:303
4232024-12-19T00:56:23.566ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 file = sled-storage/src/dataset.rs:389
4242024-12-19T00:56:23.575ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4252024-12-19T00:56:23.575ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4262024-12-19T00:56:23.575ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4272024-12-19T00:56:23.575ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4282024-12-19T00:56:23.575ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4292024-12-19T00:56:23.584ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:367
4302024-12-19T00:56:23.584ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool)
4312024-12-19T00:56:23.698ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }
4322024-12-19T00:56:23.705ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:240
4332024-12-19T00:56:23.705ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:242
4342024-12-19T00:56:23.705ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:246
4352024-12-19T00:56:23.705ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:248
4362024-12-19T00:56:23.705ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4372024-12-19T00:56:23.705ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4382024-12-19T00:56:23.705ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
4392024-12-19T00:56:23.748ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4402024-12-19T00:56:23.755ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_499e53d3-d67c-44b6-9c03-b66767baed7e/crypt/zone file = sled-storage/src/dataset.rs:303
4412024-12-19T00:56:23.967ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_499e53d3-d67c-44b6-9c03-b66767baed7e file = sled-storage/src/dataset.rs:389
4422024-12-19T00:56:23.975ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4432024-12-19T00:56:23.975ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4442024-12-19T00:56:23.976ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4452024-12-19T00:56:23.976ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(2), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8fd565e7-8dbe-4e0e-b9f6-62127926c764 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c0070a0a-b66c-49d5-a725-e7820b89ee73 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4462024-12-19T00:56:23.976ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4472024-12-19T00:56:23.976ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4482024-12-19T00:56:23.976ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4492024-12-19T00:56:23.976ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:900 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
4502024-12-19T00:56:23.976ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
4512024-12-19T00:56:23.976ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:926 generation = Generation(2) sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
4522024-12-19T00:56:23.976ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:931 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
4532024-12-19T00:56:23.976ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:936 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
4542024-12-19T00:56:23.976ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:122 gen = Generation(2) sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
4552024-12-19T00:56:23.976ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:942 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
4562024-12-19T00:56:23.976ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:799 gen = Generation(2)
4572024-12-19T00:56:23.976ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:947 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
4582024-12-19T00:56:23.976ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4416896 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:63066 req_id = 2ad40e69-f23d-4589-a017-137500173c29 response_code = 200 uri = /omicron-physical-disks
4592024-12-19T00:56:24.008ZINFOSledAgent (RSS): attempting to set up sled's datasets: DatasetsConfig { generation: Generation(1), datasets: {0c1f03e5-3bed-467f-84a1-9dc884f38543 (dataset): DatasetConfig { id: 0c1f03e5-3bed-467f-84a1-9dc884f38543 (dataset), name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 10ba9674-ef78-433b-a8f2-9320ebad6a9c (dataset): DatasetConfig { id: 10ba9674-ef78-433b-a8f2-9320ebad6a9c (dataset), name: DatasetName { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 126e87bf-2484-4627-928a-3c57cb4aa513 (dataset): DatasetConfig { id: 126e87bf-2484-4627-928a-3c57cb4aa513 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 13904540-64e0-4947-95ca-3a42d21da539 (dataset): DatasetConfig { id: 13904540-64e0-4947-95ca-3a42d21da539 (dataset), name: DatasetName { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 15905486-b138-4480-a26a-badbfe200d05 (dataset): DatasetConfig { id: 15905486-b138-4480-a26a-badbfe200d05 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1ac138c9-d12a-4f90-81c7-ea704097d6ff (dataset): DatasetConfig { id: 1ac138c9-d12a-4f90-81c7-ea704097d6ff (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1cf73bff-0fc4-451b-a8e9-6846cbb418ba (dataset): DatasetConfig { id: 1cf73bff-0fc4-451b-a8e9-6846cbb418ba (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 250268e1-d63c-4cea-9cc1-25eb67c87ccd (dataset): DatasetConfig { id: 250268e1-d63c-4cea-9cc1-25eb67c87ccd (dataset), name: DatasetName { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 290b13b8-fe37-4038-8448-62397605641f (dataset): DatasetConfig { id: 290b13b8-fe37-4038-8448-62397605641f (dataset), name: DatasetName { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 29e0b6e7-fb6a-487d-b082-5ba62d2514b8 (dataset): DatasetConfig { id: 29e0b6e7-fb6a-487d-b082-5ba62d2514b8 (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2ac8679b-8d40-43ac-a777-d63cbe442df7 (dataset): DatasetConfig { id: 2ac8679b-8d40-43ac-a777-d63cbe442df7 (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 31a15041-5516-4075-8b69-f57f2d226378 (dataset): DatasetConfig { id: 31a15041-5516-4075-8b69-f57f2d226378 (dataset), name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 34ee0d58-3bfb-4c8b-ac7a-479e76bfe06e (dataset): DatasetConfig { id: 34ee0d58-3bfb-4c8b-ac7a-479e76bfe06e (dataset), name: DatasetName { pool_name: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3c70e9c2-e71f-435d-99f1-1afaa25a0b6f (dataset): DatasetConfig { id: 3c70e9c2-e71f-435d-99f1-1afaa25a0b6f (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 448a7c2d-6742-44e4-b0c6-32bbdc404ab0 (dataset): DatasetConfig { id: 448a7c2d-6742-44e4-b0c6-32bbdc404ab0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 479a38a2-1a5c-4b98-9aeb-c28c11a8402d (dataset): DatasetConfig { id: 479a38a2-1a5c-4b98-9aeb-c28c11a8402d (dataset), name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 4dca8fbe-46f4-4405-a647-c07125a499af (dataset): DatasetConfig { id: 4dca8fbe-46f4-4405-a647-c07125a499af (dataset), name: DatasetName { pool_name: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 50244e26-7cc9-4962-acf0-199203205981 (dataset): DatasetConfig { id: 50244e26-7cc9-4962-acf0-199203205981 (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 53cc6680-7ec3-4131-bcc7-1eb7fe50fe5c (dataset): DatasetConfig { id: 53cc6680-7ec3-4131-bcc7-1eb7fe50fe5c (dataset), name: DatasetName { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 54722e9e-7288-4c1b-8c0a-92df7364958e (dataset): DatasetConfig { id: 54722e9e-7288-4c1b-8c0a-92df7364958e (dataset), name: DatasetName { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 647cba61-fb30-4aaf-a32f-3450ca64213c (dataset): DatasetConfig { id: 647cba61-fb30-4aaf-a32f-3450ca64213c (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 65eda1c6-17d3-448e-b4fd-ee2f7120ba69 (dataset): DatasetConfig { id: 65eda1c6-17d3-448e-b4fd-ee2f7120ba69 (dataset), name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 67fee7ae-7451-4ef6-83ac-8b9514dc43dd (dataset): DatasetConfig { id: 67fee7ae-7451-4ef6-83ac-8b9514dc43dd (dataset), name: DatasetName { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 69158c22-0fc6-424f-99ec-dbfb62a81040 (dataset): DatasetConfig { id: 69158c22-0fc6-424f-99ec-dbfb62a81040 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 699c85fd-c7dc-4e0c-a41a-3bd9a038bf8a (dataset): DatasetConfig { id: 699c85fd-c7dc-4e0c-a41a-3bd9a038bf8a (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6d4b0401-dcf5-40aa-9db9-e65f5a700e08 (dataset): DatasetConfig { id: 6d4b0401-dcf5-40aa-9db9-e65f5a700e08 (dataset), name: DatasetName { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 77ae8b75-e627-4b3e-b687-d69dcf3b9027 (dataset): DatasetConfig { id: 77ae8b75-e627-4b3e-b687-d69dcf3b9027 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7c9a0b3a-69bb-4368-8775-7d01f11f6158 (dataset): DatasetConfig { id: 7c9a0b3a-69bb-4368-8775-7d01f11f6158 (dataset), name: DatasetName { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7ee56b2e-1ca4-4fbd-b756-28eaf1856818 (dataset): DatasetConfig { id: 7ee56b2e-1ca4-4fbd-b756-28eaf1856818 (dataset), name: DatasetName { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8457fa42-6bd4-4560-bcfb-e822603e0d8b (dataset): DatasetConfig { id: 8457fa42-6bd4-4560-bcfb-e822603e0d8b (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8880b771-b7a9-4bab-8a41-1cb8c0f34cf2 (dataset): DatasetConfig { id: 8880b771-b7a9-4bab-8a41-1cb8c0f34cf2 (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8d048623-5cf4-4cee-acfc-725220e30304 (dataset): DatasetConfig { id: 8d048623-5cf4-4cee-acfc-725220e30304 (dataset), name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 94008417-af9e-4945-8a06-6e97e7cf9f20 (dataset): DatasetConfig { id: 94008417-af9e-4945-8a06-6e97e7cf9f20 (dataset), name: DatasetName { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9c09cbd3-1c7e-456c-ac7c-fb64137b3ca3 (dataset): DatasetConfig { id: 9c09cbd3-1c7e-456c-ac7c-fb64137b3ca3 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9c48dc9c-38de-46f7-ab4f-bca6682e60f0 (dataset): DatasetConfig { id: 9c48dc9c-38de-46f7-ab4f-bca6682e60f0 (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a3a604a2-3d1d-430a-8702-8e50b8ea3211 (dataset): DatasetConfig { id: a3a604a2-3d1d-430a-8702-8e50b8ea3211 (dataset), name: DatasetName { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a6a65b63-84f8-4003-935a-c9b81d8be35f (dataset): DatasetConfig { id: a6a65b63-84f8-4003-935a-c9b81d8be35f (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ac9cb593-830a-4f00-8192-f1c0414ef57a (dataset): DatasetConfig { id: ac9cb593-830a-4f00-8192-f1c0414ef57a (dataset), name: DatasetName { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, addb2499-a343-4e97-b3a3-dfe3df1a6559 (dataset): DatasetConfig { id: addb2499-a343-4e97-b3a3-dfe3df1a6559 (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, addfef0a-5871-437e-b72b-1c066812a798 (dataset): DatasetConfig { id: addfef0a-5871-437e-b72b-1c066812a798 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b317d544-828e-4b3b-b5dd-4e06e95668e3 (dataset): DatasetConfig { id: b317d544-828e-4b3b-b5dd-4e06e95668e3 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b5b932fa-768b-4db4-9bb6-6de7c947cb2c (dataset): DatasetConfig { id: b5b932fa-768b-4db4-9bb6-6de7c947cb2c (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b6dc690f-d642-43c2-ba8d-0b3a8f4b05f7 (dataset): DatasetConfig { id: b6dc690f-d642-43c2-ba8d-0b3a8f4b05f7 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, b7c19725-ee79-4150-b636-fc2805711098 (dataset): DatasetConfig { id: b7c19725-ee79-4150-b636-fc2805711098 (dataset), name: DatasetName { pool_name: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, be728f00-b23e-4c62-8475-500b6bbb7fa3 (dataset): DatasetConfig { id: be728f00-b23e-4c62-8475-500b6bbb7fa3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, c553820b-70cb-4e22-ba91-91a5d98b09d3 (dataset): DatasetConfig { id: c553820b-70cb-4e22-ba91-91a5d98b09d3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c70aa226-8bdc-49c0-a594-404a5c21df0d (dataset): DatasetConfig { id: c70aa226-8bdc-49c0-a594-404a5c21df0d (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c7af6a52-2df5-4fbe-8be4-137a89281dd5 (dataset): DatasetConfig { id: c7af6a52-2df5-4fbe-8be4-137a89281dd5 (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, cc9cd523-95b8-48e2-8416-faf7397478f3 (dataset): DatasetConfig { id: cc9cd523-95b8-48e2-8416-faf7397478f3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d02437e8-48ef-42b4-a269-821901e9659c (dataset): DatasetConfig { id: d02437e8-48ef-42b4-a269-821901e9659c (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d4e37dbd-aaa0-49de-822e-ea14642a8bc0 (dataset): DatasetConfig { id: d4e37dbd-aaa0-49de-822e-ea14642a8bc0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d6002de5-1ac5-4b5d-93fe-250b4ab50c61 (dataset): DatasetConfig { id: d6002de5-1ac5-4b5d-93fe-250b4ab50c61 (dataset), name: DatasetName { pool_name: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d6446fe4-8072-4c05-a795-61de32999ef4 (dataset): DatasetConfig { id: d6446fe4-8072-4c05-a795-61de32999ef4 (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d7b53c28-3485-42f4-9463-90f94e3a26fc (dataset): DatasetConfig { id: d7b53c28-3485-42f4-9463-90f94e3a26fc (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d9738f13-4c92-4f79-a4f3-06eedbe9fb1e (dataset): DatasetConfig { id: d9738f13-4c92-4f79-a4f3-06eedbe9fb1e (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, dd7b59d6-71c8-4b43-8273-6cb1fd95b874 (dataset): DatasetConfig { id: dd7b59d6-71c8-4b43-8273-6cb1fd95b874 (dataset), name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e38b579e-065b-4a03-a7e7-3cc97138bfc0 (dataset): DatasetConfig { id: e38b579e-065b-4a03-a7e7-3cc97138bfc0 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ede43e87-0e01-4b3f-ac84-1a416c2e298e (dataset): DatasetConfig { id: ede43e87-0e01-4b3f-ac84-1a416c2e298e (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ee374d7a-28db-4707-a598-a144c4419a19 (dataset): DatasetConfig { id: ee374d7a-28db-4707-a598-a144c4419a19 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ef115bcb-b602-4851-90b5-6938bafd25ee (dataset): DatasetConfig { id: ef115bcb-b602-4851-90b5-6938bafd25ee (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f107a4c2-d44a-470d-b5e7-a290597493da (dataset): DatasetConfig { id: f107a4c2-d44a-470d-b5e7-a290597493da (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, fac87f15-4b9a-412f-8833-86e4ed6dfc2c (dataset): DatasetConfig { id: fac87f15-4b9a-412f-8833-86e4ed6dfc2c (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fb09d888-dc28-4507-b88d-6f928937860f (dataset): DatasetConfig { id: fb09d888-dc28-4507-b88d-6f928937860f (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fe0df524-4ff9-496b-a4ee-c645e6c17486 (dataset): DatasetConfig { id: fe0df524-4ff9-496b-a4ee-c645e6c17486 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ff632582-5d03-44b4-9ff1-cb6fc98a168c (dataset): DatasetConfig { id: ff632582-5d03-44b4-9ff1-cb6fc98a168c (dataset), name: DatasetName { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, ffe052c3-9b90-420c-b46f-437737833444 (dataset): DatasetConfig { id: ffe052c3-9b90-420c-b46f-437737833444 (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }} } file = sled-agent/src/rack_setup/service.rs:444 sled_address = [fd00:1122:3344:101::1]:12345
4602024-12-19T00:56:24.008ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47584
4612024-12-19T00:56:24.008ZINFOSledAgent: datasets ensure file = sled-agent/src/sled_agent.rs:862 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
4622024-12-19T00:56:24.009ZINFOSledAgent (StorageManager): Received DatasetsEnsure { config: DatasetsConfig { generation: Generation(1), datasets: {0c1f03e5-3bed-467f-84a1-9dc884f38543 (dataset): DatasetConfig { id: 0c1f03e5-3bed-467f-84a1-9dc884f38543 (dataset), name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 10ba9674-ef78-433b-a8f2-9320ebad6a9c (dataset): DatasetConfig { id: 10ba9674-ef78-433b-a8f2-9320ebad6a9c (dataset), name: DatasetName { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 126e87bf-2484-4627-928a-3c57cb4aa513 (dataset): DatasetConfig { id: 126e87bf-2484-4627-928a-3c57cb4aa513 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 13904540-64e0-4947-95ca-3a42d21da539 (dataset): DatasetConfig { id: 13904540-64e0-4947-95ca-3a42d21da539 (dataset), name: DatasetName { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 15905486-b138-4480-a26a-badbfe200d05 (dataset): DatasetConfig { id: 15905486-b138-4480-a26a-badbfe200d05 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1ac138c9-d12a-4f90-81c7-ea704097d6ff (dataset): DatasetConfig { id: 1ac138c9-d12a-4f90-81c7-ea704097d6ff (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1cf73bff-0fc4-451b-a8e9-6846cbb418ba (dataset): DatasetConfig { id: 1cf73bff-0fc4-451b-a8e9-6846cbb418ba (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 250268e1-d63c-4cea-9cc1-25eb67c87ccd (dataset): DatasetConfig { id: 250268e1-d63c-4cea-9cc1-25eb67c87ccd (dataset), name: DatasetName { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 290b13b8-fe37-4038-8448-62397605641f (dataset): DatasetConfig { id: 290b13b8-fe37-4038-8448-62397605641f (dataset), name: DatasetName { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 29e0b6e7-fb6a-487d-b082-5ba62d2514b8 (dataset): DatasetConfig { id: 29e0b6e7-fb6a-487d-b082-5ba62d2514b8 (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2ac8679b-8d40-43ac-a777-d63cbe442df7 (dataset): DatasetConfig { id: 2ac8679b-8d40-43ac-a777-d63cbe442df7 (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 31a15041-5516-4075-8b69-f57f2d226378 (dataset): DatasetConfig { id: 31a15041-5516-4075-8b69-f57f2d226378 (dataset), name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 34ee0d58-3bfb-4c8b-ac7a-479e76bfe06e (dataset): DatasetConfig { id: 34ee0d58-3bfb-4c8b-ac7a-479e76bfe06e (dataset), name: DatasetName { pool_name: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3c70e9c2-e71f-435d-99f1-1afaa25a0b6f (dataset): DatasetConfig { id: 3c70e9c2-e71f-435d-99f1-1afaa25a0b6f (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 448a7c2d-6742-44e4-b0c6-32bbdc404ab0 (dataset): DatasetConfig { id: 448a7c2d-6742-44e4-b0c6-32bbdc404ab0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 479a38a2-1a5c-4b98-9aeb-c28c11a8402d (dataset): DatasetConfig { id: 479a38a2-1a5c-4b98-9aeb-c28c11a8402d (dataset), name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 4dca8fbe-46f4-4405-a647-c07125a499af (dataset): DatasetConfig { id: 4dca8fbe-46f4-4405-a647-c07125a499af (dataset), name: DatasetName { pool_name: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 50244e26-7cc9-4962-acf0-199203205981 (dataset): DatasetConfig { id: 50244e26-7cc9-4962-acf0-199203205981 (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 53cc6680-7ec3-4131-bcc7-1eb7fe50fe5c (dataset): DatasetConfig { id: 53cc6680-7ec3-4131-bcc7-1eb7fe50fe5c (dataset), name: DatasetName { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 54722e9e-7288-4c1b-8c0a-92df7364958e (dataset): DatasetConfig { id: 54722e9e-7288-4c1b-8c0a-92df7364958e (dataset), name: DatasetName { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 647cba61-fb30-4aaf-a32f-3450ca64213c (dataset): DatasetConfig { id: 647cba61-fb30-4aaf-a32f-3450ca64213c (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 65eda1c6-17d3-448e-b4fd-ee2f7120ba69 (dataset): DatasetConfig { id: 65eda1c6-17d3-448e-b4fd-ee2f7120ba69 (dataset), name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 67fee7ae-7451-4ef6-83ac-8b9514dc43dd (dataset): DatasetConfig { id: 67fee7ae-7451-4ef6-83ac-8b9514dc43dd (dataset), name: DatasetName { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 69158c22-0fc6-424f-99ec-dbfb62a81040 (dataset): DatasetConfig { id: 69158c22-0fc6-424f-99ec-dbfb62a81040 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 699c85fd-c7dc-4e0c-a41a-3bd9a038bf8a (dataset): DatasetConfig { id: 699c85fd-c7dc-4e0c-a41a-3bd9a038bf8a (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6d4b0401-dcf5-40aa-9db9-e65f5a700e08 (dataset): DatasetConfig { id: 6d4b0401-dcf5-40aa-9db9-e65f5a700e08 (dataset), name: DatasetName { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 77ae8b75-e627-4b3e-b687-d69dcf3b9027 (dataset): DatasetConfig { id: 77ae8b75-e627-4b3e-b687-d69dcf3b9027 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7c9a0b3a-69bb-4368-8775-7d01f11f6158 (dataset): DatasetConfig { id: 7c9a0b3a-69bb-4368-8775-7d01f11f6158 (dataset), name: DatasetName { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7ee56b2e-1ca4-4fbd-b756-28eaf1856818 (dataset): DatasetConfig { id: 7ee56b2e-1ca4-4fbd-b756-28eaf1856818 (dataset), name: DatasetName { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8457fa42-6bd4-4560-bcfb-e822603e0d8b (dataset): DatasetConfig { id: 8457fa42-6bd4-4560-bcfb-e822603e0d8b (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8880b771-b7a9-4bab-8a41-1cb8c0f34cf2 (dataset): DatasetConfig { id: 8880b771-b7a9-4bab-8a41-1cb8c0f34cf2 (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8d048623-5cf4-4cee-acfc-725220e30304 (dataset): DatasetConfig { id: 8d048623-5cf4-4cee-acfc-725220e30304 (dataset), name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 94008417-af9e-4945-8a06-6e97e7cf9f20 (dataset): DatasetConfig { id: 94008417-af9e-4945-8a06-6e97e7cf9f20 (dataset), name: DatasetName { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9c09cbd3-1c7e-456c-ac7c-fb64137b3ca3 (dataset): DatasetConfig { id: 9c09cbd3-1c7e-456c-ac7c-fb64137b3ca3 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9c48dc9c-38de-46f7-ab4f-bca6682e60f0 (dataset): DatasetConfig { id: 9c48dc9c-38de-46f7-ab4f-bca6682e60f0 (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a3a604a2-3d1d-430a-8702-8e50b8ea3211 (dataset): DatasetConfig { id: a3a604a2-3d1d-430a-8702-8e50b8ea3211 (dataset), name: DatasetName { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a6a65b63-84f8-4003-935a-c9b81d8be35f (dataset): DatasetConfig { id: a6a65b63-84f8-4003-935a-c9b81d8be35f (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ac9cb593-830a-4f00-8192-f1c0414ef57a (dataset): DatasetConfig { id: ac9cb593-830a-4f00-8192-f1c0414ef57a (dataset), name: DatasetName { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, addb2499-a343-4e97-b3a3-dfe3df1a6559 (dataset): DatasetConfig { id: addb2499-a343-4e97-b3a3-dfe3df1a6559 (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, addfef0a-5871-437e-b72b-1c066812a798 (dataset): DatasetConfig { id: addfef0a-5871-437e-b72b-1c066812a798 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b317d544-828e-4b3b-b5dd-4e06e95668e3 (dataset): DatasetConfig { id: b317d544-828e-4b3b-b5dd-4e06e95668e3 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b5b932fa-768b-4db4-9bb6-6de7c947cb2c (dataset): DatasetConfig { id: b5b932fa-768b-4db4-9bb6-6de7c947cb2c (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b6dc690f-d642-43c2-ba8d-0b3a8f4b05f7 (dataset): DatasetConfig { id: b6dc690f-d642-43c2-ba8d-0b3a8f4b05f7 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, b7c19725-ee79-4150-b636-fc2805711098 (dataset): DatasetConfig { id: b7c19725-ee79-4150-b636-fc2805711098 (dataset), name: DatasetName { pool_name: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, be728f00-b23e-4c62-8475-500b6bbb7fa3 (dataset): DatasetConfig { id: be728f00-b23e-4c62-8475-500b6bbb7fa3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, c553820b-70cb-4e22-ba91-91a5d98b09d3 (dataset): DatasetConfig { id: c553820b-70cb-4e22-ba91-91a5d98b09d3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c70aa226-8bdc-49c0-a594-404a5c21df0d (dataset): DatasetConfig { id: c70aa226-8bdc-49c0-a594-404a5c21df0d (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c7af6a52-2df5-4fbe-8be4-137a89281dd5 (dataset): DatasetConfig { id: c7af6a52-2df5-4fbe-8be4-137a89281dd5 (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, cc9cd523-95b8-48e2-8416-faf7397478f3 (dataset): DatasetConfig { id: cc9cd523-95b8-48e2-8416-faf7397478f3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d02437e8-48ef-42b4-a269-821901e9659c (dataset): DatasetConfig { id: d02437e8-48ef-42b4-a269-821901e9659c (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d4e37dbd-aaa0-49de-822e-ea14642a8bc0 (dataset): DatasetConfig { id: d4e37dbd-aaa0-49de-822e-ea14642a8bc0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d6002de5-1ac5-4b5d-93fe-250b4ab50c61 (dataset): DatasetConfig { id: d6002de5-1ac5-4b5d-93fe-250b4ab50c61 (dataset), name: DatasetName { pool_name: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d6446fe4-8072-4c05-a795-61de32999ef4 (dataset): DatasetConfig { id: d6446fe4-8072-4c05-a795-61de32999ef4 (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d7b53c28-3485-42f4-9463-90f94e3a26fc (dataset): DatasetConfig { id: d7b53c28-3485-42f4-9463-90f94e3a26fc (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d9738f13-4c92-4f79-a4f3-06eedbe9fb1e (dataset): DatasetConfig { id: d9738f13-4c92-4f79-a4f3-06eedbe9fb1e (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, dd7b59d6-71c8-4b43-8273-6cb1fd95b874 (dataset): DatasetConfig { id: dd7b59d6-71c8-4b43-8273-6cb1fd95b874 (dataset), name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e38b579e-065b-4a03-a7e7-3cc97138bfc0 (dataset): DatasetConfig { id: e38b579e-065b-4a03-a7e7-3cc97138bfc0 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ede43e87-0e01-4b3f-ac84-1a416c2e298e (dataset): DatasetConfig { id: ede43e87-0e01-4b3f-ac84-1a416c2e298e (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ee374d7a-28db-4707-a598-a144c4419a19 (dataset): DatasetConfig { id: ee374d7a-28db-4707-a598-a144c4419a19 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ef115bcb-b602-4851-90b5-6938bafd25ee (dataset): DatasetConfig { id: ef115bcb-b602-4851-90b5-6938bafd25ee (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f107a4c2-d44a-470d-b5e7-a290597493da (dataset): DatasetConfig { id: f107a4c2-d44a-470d-b5e7-a290597493da (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, fac87f15-4b9a-412f-8833-86e4ed6dfc2c (dataset): DatasetConfig { id: fac87f15-4b9a-412f-8833-86e4ed6dfc2c (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fb09d888-dc28-4507-b88d-6f928937860f (dataset): DatasetConfig { id: fb09d888-dc28-4507-b88d-6f928937860f (dataset), name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fe0df524-4ff9-496b-a4ee-c645e6c17486 (dataset): DatasetConfig { id: fe0df524-4ff9-496b-a4ee-c645e6c17486 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ff632582-5d03-44b4-9ff1-cb6fc98a168c (dataset): DatasetConfig { id: ff632582-5d03-44b4-9ff1-cb6fc98a168c (dataset), name: DatasetName { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, ffe052c3-9b90-420c-b46f-437737833444 (dataset): DatasetConfig { id: ffe052c3-9b90-420c-b46f-437737833444 (dataset), name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }} }, tx: ... } file = sled-storage/src/manager.rs:603
4632024-12-19T00:56:24.009ZINFOSledAgent (StorageManager): No ledger in /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4642024-12-19T00:56:24.009ZINFOSledAgent (StorageManager): No ledger in /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4652024-12-19T00:56:24.009ZINFOSledAgent (StorageManager): No previously-stored 'requested datasets', creating new ledger file = sled-storage/src/manager.rs:905 request = datasets_ensure
4662024-12-19T00:56:24.009ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4672024-12-19T00:56:24.009ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/98a100ce-cd82-4209-8530-7b48ca829744/crypt/zone/oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16"), full_name: "oxp_98a100ce-cd82-4209-8530-7b48ca829744/crypt/zone/oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16" } file = sled-storage/src/manager.rs:1331
4682024-12-19T00:56:24.136ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4692024-12-19T00:56:24.136ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_499e53d3-d67c-44b6-9c03-b66767baed7e/crucible" } file = sled-storage/src/manager.rs:1331
4702024-12-19T00:56:24.259ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4712024-12-19T00:56:24.259ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_93eb8a71-6519-482e-a366-b15e30b4605e/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
4722024-12-19T00:56:24.296ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
4732024-12-19T00:56:24.394ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4742024-12-19T00:56:24.394ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/f4009784-7bde-44ff-854e-49d614e9e682/crypt/zone/oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3"), full_name: "oxp_f4009784-7bde-44ff-854e-49d614e9e682/crypt/zone/oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3" } file = sled-storage/src/manager.rs:1331
4752024-12-19T00:56:24.527ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4762024-12-19T00:56:24.527ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/93eb8a71-6519-482e-a366-b15e30b4605e/crypt/zone"), full_name: "oxp_93eb8a71-6519-482e-a366-b15e30b4605e/crypt/zone" } file = sled-storage/src/manager.rs:1331
4772024-12-19T00:56:24.616ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4782024-12-19T00:56:24.616ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
4792024-12-19T00:56:24.710ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4802024-12-19T00:56:24.711ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b" } file = sled-storage/src/manager.rs:1331
4812024-12-19T00:56:24.828ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4822024-12-19T00:56:24.828ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/499e53d3-d67c-44b6-9c03-b66767baed7e/crypt/zone/oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0"), full_name: "oxp_499e53d3-d67c-44b6-9c03-b66767baed7e/crypt/zone/oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0" } file = sled-storage/src/manager.rs:1331
4832024-12-19T00:56:24.952ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4842024-12-19T00:56:24.952ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb/crypt/zone/oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c"), full_name: "oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb/crypt/zone/oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c" } file = sled-storage/src/manager.rs:1331
4852024-12-19T00:56:25.055ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4862024-12-19T00:56:25.055ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/zone"), full_name: "oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/zone" } file = sled-storage/src/manager.rs:1331
4872024-12-19T00:56:25.132ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4882024-12-19T00:56:25.132ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone/oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99"), full_name: "oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone/oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99" } file = sled-storage/src/manager.rs:1331
4892024-12-19T00:56:25.253ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4902024-12-19T00:56:25.253ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_98a100ce-cd82-4209-8530-7b48ca829744/crucible" } file = sled-storage/src/manager.rs:1331
4912024-12-19T00:56:25.298ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
4922024-12-19T00:56:25.364ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4932024-12-19T00:56:25.364ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_1025d0d6-c165-4f43-bef0-cb2eeb04b8c2/crucible" } file = sled-storage/src/manager.rs:1331
4942024-12-19T00:56:25.477ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4952024-12-19T00:56:25.477ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc" } file = sled-storage/src/manager.rs:1331
4962024-12-19T00:56:25.601ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4972024-12-19T00:56:25.601ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/93eb8a71-6519-482e-a366-b15e30b4605e/crypt/zone/oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172"), full_name: "oxp_93eb8a71-6519-482e-a366-b15e30b4605e/crypt/zone/oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172" } file = sled-storage/src/manager.rs:1331
4982024-12-19T00:56:25.726ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4992024-12-19T00:56:25.726ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/98a100ce-cd82-4209-8530-7b48ca829744/crypt/debug"), full_name: "oxp_98a100ce-cd82-4209-8530-7b48ca829744/crypt/debug" } file = sled-storage/src/manager.rs:1331
5002024-12-19T00:56:25.812ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5012024-12-19T00:56:25.812ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/1025d0d6-c165-4f43-bef0-cb2eeb04b8c2/crypt/zone"), full_name: "oxp_1025d0d6-c165-4f43-bef0-cb2eeb04b8c2/crypt/zone" } file = sled-storage/src/manager.rs:1331
5022024-12-19T00:56:25.897ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5032024-12-19T00:56:25.897ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/zone/oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf"), full_name: "oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/zone/oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf" } file = sled-storage/src/manager.rs:1331
5042024-12-19T00:56:26.020ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5052024-12-19T00:56:26.020ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/f4009784-7bde-44ff-854e-49d614e9e682/crypt/debug"), full_name: "oxp_f4009784-7bde-44ff-854e-49d614e9e682/crypt/debug" } file = sled-storage/src/manager.rs:1331
5062024-12-19T00:56:26.106ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5072024-12-19T00:56:26.106ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/f4009784-7bde-44ff-854e-49d614e9e682/crypt/zone/oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5"), full_name: "oxp_f4009784-7bde-44ff-854e-49d614e9e682/crypt/zone/oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5" } file = sled-storage/src/manager.rs:1331
5082024-12-19T00:56:26.183ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 11.65087653s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
5092024-12-19T00:56:26.236ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5102024-12-19T00:56:26.236ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone/oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b"), full_name: "oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone/oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b" } file = sled-storage/src/manager.rs:1331
5112024-12-19T00:56:26.300ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
5122024-12-19T00:56:26.361ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5132024-12-19T00:56:26.361ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_98a100ce-cd82-4209-8530-7b48ca829744/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5142024-12-19T00:56:26.478ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5152024-12-19T00:56:26.478ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb/crypt/zone/oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa"), full_name: "oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb/crypt/zone/oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa" } file = sled-storage/src/manager.rs:1331
5162024-12-19T00:56:26.603ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5172024-12-19T00:56:26.603ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5182024-12-19T00:56:26.723ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5192024-12-19T00:56:26.723ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/clickhouse" } file = sled-storage/src/manager.rs:1331
5202024-12-19T00:56:26.839ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5212024-12-19T00:56:26.839ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/f4009784-7bde-44ff-854e-49d614e9e682/crypt/zone"), full_name: "oxp_f4009784-7bde-44ff-854e-49d614e9e682/crypt/zone" } file = sled-storage/src/manager.rs:1331
5222024-12-19T00:56:26.924ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5232024-12-19T00:56:26.924ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_93eb8a71-6519-482e-a366-b15e30b4605e/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
5242024-12-19T00:56:27.041ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5252024-12-19T00:56:27.041ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_f4009784-7bde-44ff-854e-49d614e9e682/crucible" } file = sled-storage/src/manager.rs:1331
5262024-12-19T00:56:27.189ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5272024-12-19T00:56:27.189ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/499e53d3-d67c-44b6-9c03-b66767baed7e/crypt/zone"), full_name: "oxp_499e53d3-d67c-44b6-9c03-b66767baed7e/crypt/zone" } file = sled-storage/src/manager.rs:1331
5282024-12-19T00:56:27.278ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5292024-12-19T00:56:27.278ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5302024-12-19T00:56:27.302ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
5312024-12-19T00:56:27.395ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5322024-12-19T00:56:27.395ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone/oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9"), full_name: "oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone/oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9" } file = sled-storage/src/manager.rs:1331
5332024-12-19T00:56:27.520ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5342024-12-19T00:56:27.520ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/98a100ce-cd82-4209-8530-7b48ca829744/crypt/zone"), full_name: "oxp_98a100ce-cd82-4209-8530-7b48ca829744/crypt/zone" } file = sled-storage/src/manager.rs:1331
5352024-12-19T00:56:27.604ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5362024-12-19T00:56:27.604ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb/crucible" } file = sled-storage/src/manager.rs:1331
5372024-12-19T00:56:27.721ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5382024-12-19T00:56:27.721ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995" } file = sled-storage/src/manager.rs:1331
5392024-12-19T00:56:27.843ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5402024-12-19T00:56:27.843ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
5412024-12-19T00:56:27.959ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5422024-12-19T00:56:27.959ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb/crypt/zone"), full_name: "oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb/crypt/zone" } file = sled-storage/src/manager.rs:1331
5432024-12-19T00:56:28.044ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5442024-12-19T00:56:28.044ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5452024-12-19T00:56:28.160ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5462024-12-19T00:56:28.160ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/499e53d3-d67c-44b6-9c03-b66767baed7e/crypt/debug"), full_name: "oxp_499e53d3-d67c-44b6-9c03-b66767baed7e/crypt/debug" } file = sled-storage/src/manager.rs:1331
5472024-12-19T00:56:28.249ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5482024-12-19T00:56:28.249ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone"), full_name: "oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone" } file = sled-storage/src/manager.rs:1331
5492024-12-19T00:56:28.304ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
5502024-12-19T00:56:28.339ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5512024-12-19T00:56:28.339ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90" } file = sled-storage/src/manager.rs:1331
5522024-12-19T00:56:28.461ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5532024-12-19T00:56:28.461ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone" } file = sled-storage/src/manager.rs:1331
5542024-12-19T00:56:28.546ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5552024-12-19T00:56:28.546ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1" } file = sled-storage/src/manager.rs:1331
5562024-12-19T00:56:28.668ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5572024-12-19T00:56:28.668ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/debug"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/debug" } file = sled-storage/src/manager.rs:1331
5582024-12-19T00:56:28.713ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4307
5592024-12-19T00:56:28.713ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4317 gateway = fd00:1122:3344:101::1
5602024-12-19T00:56:28.720ZINFOSledAgent (ServiceManager): Default route already exists file = sled-agent/src/services.rs:4331 gateway = fd00:1122:3344:101::1
5612024-12-19T00:56:28.720ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4396
5622024-12-19T00:56:28.759ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5632024-12-19T00:56:28.759ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/1025d0d6-c165-4f43-bef0-cb2eeb04b8c2/crypt/debug"), full_name: "oxp_1025d0d6-c165-4f43-bef0-cb2eeb04b8c2/crypt/debug" } file = sled-storage/src/manager.rs:1331
5642024-12-19T00:56:28.851ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5652024-12-19T00:56:28.851ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/93eb8a71-6519-482e-a366-b15e30b4605e/crypt/debug"), full_name: "oxp_93eb8a71-6519-482e-a366-b15e30b4605e/crypt/debug" } file = sled-storage/src/manager.rs:1331
5662024-12-19T00:56:28.874ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4462
5672024-12-19T00:56:28.874ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4489
5682024-12-19T00:56:28.915ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4502
5692024-12-19T00:56:28.915ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4348
5702024-12-19T00:56:28.931ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 7357276260268799775 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
5712024-12-19T00:56:28.931ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11866606071799879623 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
5722024-12-19T00:56:28.942ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5732024-12-19T00:56:28.942ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/499e53d3-d67c-44b6-9c03-b66767baed7e/crypt/zone/oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc"), full_name: "oxp_499e53d3-d67c-44b6-9c03-b66767baed7e/crypt/zone/oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc" } file = sled-storage/src/manager.rs:1331
5742024-12-19T00:56:28.967ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4390
5752024-12-19T00:56:28.967ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4469 rack_subnet = fd00:1122:3344::
5762024-12-19T00:56:28.987ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4480
5772024-12-19T00:56:28.988ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4522
5782024-12-19T00:56:29.059ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4554
5792024-12-19T00:56:29.059ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4560
5802024-12-19T00:56:29.076ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5812024-12-19T00:56:29.076ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/external_dns" } file = sled-storage/src/manager.rs:1331
5822024-12-19T00:56:29.149ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4601
5832024-12-19T00:56:29.149ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:370
5842024-12-19T00:56:29.189ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:420
5852024-12-19T00:56:29.189ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/nexus.rs:527 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
5862024-12-19T00:56:29.194ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5872024-12-19T00:56:29.194ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/zone/oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1"), full_name: "oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/zone/oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1" } file = sled-storage/src/manager.rs:1331
5882024-12-19T00:56:29.227ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:717
5892024-12-19T00:56:29.227ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:762
5902024-12-19T00:56:29.228ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:765 uptime = 1
5912024-12-19T00:56:29.228ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: Some(\n None,\n ),\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n tx_eq: None,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:444
5922024-12-19T00:56:29.274ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
5932024-12-19T00:56:29.311ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
5942024-12-19T00:56:29.324ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5952024-12-19T00:56:29.324ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_93eb8a71-6519-482e-a366-b15e30b4605e/crucible" } file = sled-storage/src/manager.rs:1331
5962024-12-19T00:56:29.435ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5972024-12-19T00:56:29.435ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/external_dns" } file = sled-storage/src/manager.rs:1331
5982024-12-19T00:56:29.549ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5992024-12-19T00:56:29.549ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/93eb8a71-6519-482e-a366-b15e30b4605e/crypt/zone/oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09"), full_name: "oxp_93eb8a71-6519-482e-a366-b15e30b4605e/crypt/zone/oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09" } file = sled-storage/src/manager.rs:1331
6002024-12-19T00:56:29.662ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6012024-12-19T00:56:29.662ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/1025d0d6-c165-4f43-bef0-cb2eeb04b8c2/crypt/zone/oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad"), full_name: "oxp_1025d0d6-c165-4f43-bef0-cb2eeb04b8c2/crypt/zone/oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad" } file = sled-storage/src/manager.rs:1331
6022024-12-19T00:56:29.781ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6032024-12-19T00:56:29.781ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6/crucible" } file = sled-storage/src/manager.rs:1331
6042024-12-19T00:56:29.886ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6052024-12-19T00:56:29.886ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/zone/oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282"), full_name: "oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/zone/oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282" } file = sled-storage/src/manager.rs:1331
6062024-12-19T00:56:29.914ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8023753382245910886 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
6072024-12-19T00:56:29.914ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 7374314792341526415 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
6082024-12-19T00:56:30.003ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6092024-12-19T00:56:30.003ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crucible" } file = sled-storage/src/manager.rs:1331
6102024-12-19T00:56:30.107ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6112024-12-19T00:56:30.107ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/98a100ce-cd82-4209-8530-7b48ca829744/crypt/zone/oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67"), full_name: "oxp_98a100ce-cd82-4209-8530-7b48ca829744/crypt/zone/oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67" } file = sled-storage/src/manager.rs:1331
6122024-12-19T00:56:30.223ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6132024-12-19T00:56:30.223ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43" } file = sled-storage/src/manager.rs:1331
6142024-12-19T00:56:30.313ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
6152024-12-19T00:56:30.339ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6162024-12-19T00:56:30.339ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/93eb8a71-6519-482e-a366-b15e30b4605e/crypt/zone/oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439"), full_name: "oxp_93eb8a71-6519-482e-a366-b15e30b4605e/crypt/zone/oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439" } file = sled-storage/src/manager.rs:1331
6172024-12-19T00:56:30.458ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6182024-12-19T00:56:30.458ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/93eb8a71-6519-482e-a366-b15e30b4605e/crypt/zone/oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763"), full_name: "oxp_93eb8a71-6519-482e-a366-b15e30b4605e/crypt/zone/oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763" } file = sled-storage/src/manager.rs:1331
6192024-12-19T00:56:30.576ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6202024-12-19T00:56:30.576ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crucible" } file = sled-storage/src/manager.rs:1331
6212024-12-19T00:56:30.680ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6222024-12-19T00:56:30.680ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/debug"), full_name: "oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/debug" } file = sled-storage/src/manager.rs:1331
6232024-12-19T00:56:30.760ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6242024-12-19T00:56:30.760ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone/oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317"), full_name: "oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone/oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317" } file = sled-storage/src/manager.rs:1331
6252024-12-19T00:56:30.877ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6262024-12-19T00:56:30.877ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone/oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9"), full_name: "oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/zone/oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9" } file = sled-storage/src/manager.rs:1331
6272024-12-19T00:56:30.993ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6282024-12-19T00:56:30.993ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb"), full_name: "oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/zone/oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb" } file = sled-storage/src/manager.rs:1331
6292024-12-19T00:56:31.110ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6302024-12-19T00:56:31.110ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb/crypt/debug"), full_name: "oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb/crypt/debug" } file = sled-storage/src/manager.rs:1331
6312024-12-19T00:56:31.189ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6322024-12-19T00:56:31.189ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/debug"), full_name: "oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/debug" } file = sled-storage/src/manager.rs:1331
6332024-12-19T00:56:31.193ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/nexus.rs:527 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
6342024-12-19T00:56:31.268ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6352024-12-19T00:56:31.269ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6362024-12-19T00:56:31.269ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:603
6372024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
6382024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
6392024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
6402024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
6412024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
6422024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
6432024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
6442024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
6452024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
6462024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
6472024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
6482024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
6492024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
6502024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
6512024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
6522024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
6532024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
6542024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
6552024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
6562024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
6572024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
6582024-12-19T00:56:31.269ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
6592024-12-19T00:56:31.269ZINFOSledAgent: datasets ensure: Updated storage file = sled-agent/src/sled_agent.rs:864 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
6602024-12-19T00:56:31.269ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7261305 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:47584 req_id = 9600fb91-8e5a-400c-8a9f-3ddd756dffba response_code = 200 uri = /datasets
6612024-12-19T00:56:31.300ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: 0018a454-75c0-41b1-be69-9476391f01e1 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
6622024-12-19T00:56:31.301ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57586
6632024-12-19T00:56:31.301ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6642024-12-19T00:56:31.301ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }
6652024-12-19T00:56:31.315ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
6662024-12-19T00:56:31.335ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6672024-12-19T00:56:31.335ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }
6682024-12-19T00:56:31.370ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6692024-12-19T00:56:31.370ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }
6702024-12-19T00:56:31.403ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6712024-12-19T00:56:31.403ZINFOSledAgent (ServiceManager): No ledger in /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:188
6722024-12-19T00:56:31.403ZINFOSledAgent (ServiceManager): No ledger in /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:188
6732024-12-19T00:56:31.403ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6742024-12-19T00:56:31.448ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6752024-12-19T00:56:31.448ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6762024-12-19T00:56:31.448ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6772024-12-19T00:56:31.501ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9 file = illumos-utils/src/zone.rs:325
6782024-12-19T00:56:31.505ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763 file = illumos-utils/src/zone.rs:325
6792024-12-19T00:56:31.509ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1 file = illumos-utils/src/zone.rs:325
6802024-12-19T00:56:31.521ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9 file = illumos-utils/src/zone.rs:363
6812024-12-19T00:56:31.526ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763 file = illumos-utils/src/zone.rs:363
6822024-12-19T00:56:31.531ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1 file = illumos-utils/src/zone.rs:363
6832024-12-19T00:56:32.317ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/de9bec7d-f8ce-44a0-879c-5e088768f91c) file = sled-agent/src/probe_manager.rs:231 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
6842024-12-19T00:56:33.180ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6852024-12-19T00:56:33.180ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
6862024-12-19T00:56:33.181ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9
6872024-12-19T00:56:33.666ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6882024-12-19T00:56:34.146ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
6892024-12-19T00:56:34.147ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6902024-12-19T00:56:34.147ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
6912024-12-19T00:56:34.155ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9") failed: Property not found. retry in 46.517864ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9
6922024-12-19T00:56:34.155ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763
6932024-12-19T00:56:34.159ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1
6942024-12-19T00:56:34.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9") failed: Property not found. retry in 86.31813ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9
6952024-12-19T00:56:34.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1") failed: Property not found. retry in 64.019127ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1
6962024-12-19T00:56:34.293ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763") failed: Property not found. retry in 61.37823ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763
6972024-12-19T00:56:34.306ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9") failed: Property not found. retry in 279.569197ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9
6982024-12-19T00:56:34.442ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1") failed: Property not found. retry in 144.017575ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1
6992024-12-19T00:56:34.544ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763") failed: Property not found. retry in 147.136271ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763
7002024-12-19T00:56:34.633ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1") failed: Property not found. retry in 210.817779ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1
7012024-12-19T00:56:34.651ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9") failed: Property not found. retry in 259.805578ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9
7022024-12-19T00:56:34.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763") failed: Property not found. retry in 208.257993ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763
7032024-12-19T00:56:34.855ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1") failed: Property not found. retry in 593.744905ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1
7042024-12-19T00:56:34.921ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9") failed: Property not found. retry in 832.691745ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9
7052024-12-19T00:56:34.943ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763") failed: Property not found. retry in 463.544835ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763
7062024-12-19T00:56:35.417ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763") failed: Property not found. retry in 1.127796047s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763
7072024-12-19T00:56:35.458ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1") failed: Property not found. retry in 1.094579881s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1
7082024-12-19T00:56:35.764ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9") failed: Property not found. retry in 874.118976ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9
7092024-12-19T00:56:36.554ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763") failed: Property not found. retry in 1.484330656s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763
7102024-12-19T00:56:36.563ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1") failed: Property not found. retry in 893.117924ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1
7112024-12-19T00:56:36.647ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9") failed: Property not found. retry in 1.497265057s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9
7122024-12-19T00:56:37.465ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1") failed: Property not found. retry in 752.1054ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1
7132024-12-19T00:56:38.048ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763") failed: Property not found. retry in 1.487277318s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763
7142024-12-19T00:56:38.156ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9") failed: Property not found. retry in 860.464795ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9
7152024-12-19T00:56:38.226ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1") failed: Property not found. retry in 1.045596438s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1
7162024-12-19T00:56:38.932ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 7357276260268799775 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
7172024-12-19T00:56:38.933ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11866606071799879623 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
7182024-12-19T00:56:39.025ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9") failed: Property not found. retry in 515.282272ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9
7192024-12-19T00:56:39.281ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1") failed: Property not found. retry in 660.5585ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1
7202024-12-19T00:56:39.916ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8023753382245910886 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
7212024-12-19T00:56:39.916ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 7374314792341526415 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
7222024-12-19T00:56:39.964ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_internal_dns_2eb2a760-6b2b-4fdd-a606-eabb932aebf9
7232024-12-19T00:56:39.964ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_internal_dns_050cdbd7-4ebc-471c-b4dc-ec1b3741f763
7242024-12-19T00:56:39.964ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_internal_dns_0018a454-75c0-41b1-be69-9476391f01e1
7252024-12-19T00:56:39.964ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7262024-12-19T00:56:39.964ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7272024-12-19T00:56:39.964ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8663781 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:57586 req_id = 2f575aa8-465d-4a97-8edf-3b4ab7d2fc1d response_code = 204 uri = /omicron-zones
7282024-12-19T00:56:39.966ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:642
7292024-12-19T00:56:40.403ZWARNSledAgent (RSS): failed to write DNS configuration (will retry in 51.420096ms) dns_config_addr = [fd00:1122:3344:1::1]:5353 error_message = Communication Error: error sending request for url (http://[fd00:1122:3344:1::1]:5353/config) file = sled-agent/src/rack_setup/service.rs:662
7302024-12-19T00:56:40.456ZWARNSledAgent (RSS): failed to write DNS configuration (will retry in 62.109451ms) dns_config_addr = [fd00:1122:3344:1::1]:5353 error_message = Communication Error: error sending request for url (http://[fd00:1122:3344:1::1]:5353/config) file = sled-agent/src/rack_setup/service.rs:662
7312024-12-19T00:56:40.537ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:642
7322024-12-19T00:56:40.590ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:642
7332024-12-19T00:56:40.637ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:678
7342024-12-19T00:56:40.637ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7352024-12-19T00:56:40.670ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7362024-12-19T00:56:40.671ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7372024-12-19T00:56:40.671ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7382024-12-19T00:56:40.700ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: 0018a454-75c0-41b1-be69-9476391f01e1 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: dfd796d3-f5de-49d4-a936-e01153b80282 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: bf880c19-9d84-4efc-886f-98b14f937093, kind: Service { id: dfd796d3-f5de-49d4-a936-e01153b80282 }, name: Name("ntp-dfd796d3-f5de-49d4-a936-e01153b80282"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 178, 125])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
7392024-12-19T00:56:40.701ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60254
7402024-12-19T00:56:40.701ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7412024-12-19T00:56:40.701ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }
7422024-12-19T00:56:40.741ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7432024-12-19T00:56:40.741ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }
7442024-12-19T00:56:40.776ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7452024-12-19T00:56:40.776ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }
7462024-12-19T00:56:40.811ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7472024-12-19T00:56:40.811ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
7482024-12-19T00:56:40.812ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
7492024-12-19T00:56:40.812ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7502024-12-19T00:56:40.823ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7512024-12-19T00:56:40.856ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7522024-12-19T00:56:40.857ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7532024-12-19T00:56:40.857ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7542024-12-19T00:56:40.887ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 178, 125]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
7552024-12-19T00:56:40.887ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = boundary_ntp
7562024-12-19T00:56:40.887ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:83
7572024-12-19T00:56:40.887ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:90
7582024-12-19T00:56:40.888ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:118
7592024-12-19T00:56:40.889ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7602024-12-19T00:56:40.912ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282 file = illumos-utils/src/zone.rs:325
7612024-12-19T00:56:40.932ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282 file = illumos-utils/src/zone.rs:363
7622024-12-19T00:56:41.610ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7632024-12-19T00:56:41.610ZINFOSledAgent (ServiceManager): Booting oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282
7642024-12-19T00:56:41.740ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282") failed: Property not found. retry in 55.190837ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282
7652024-12-19T00:56:41.803ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282") failed: Property not found. retry in 113.690663ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282
7662024-12-19T00:56:41.928ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282") failed: Property not found. retry in 177.183054ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282
7672024-12-19T00:56:42.115ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282") failed: Property not found. retry in 284.804536ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282
7682024-12-19T00:56:42.408ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282") failed: Property not found. retry in 941.364252ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282
7692024-12-19T00:56:43.361ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282") failed: Property not found. retry in 1.299310095s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282
7702024-12-19T00:56:44.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282") failed: Property not found. retry in 1.247048108s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282
7712024-12-19T00:56:45.926ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282") failed: Property not found. retry in 712.868967ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282
7722024-12-19T00:56:46.667ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282
7732024-12-19T00:56:46.667ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7742024-12-19T00:56:46.667ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7752024-12-19T00:56:46.668ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5966867 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:60254 req_id = 0da39e42-ac63-4ec3-b55b-ef1821351521 response_code = 204 uri = /omicron-zones
7762024-12-19T00:56:46.670ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:719
7772024-12-19T00:56:46.701ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
7782024-12-19T00:56:46.701ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:49323
7792024-12-19T00:56:46.713ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3915
7802024-12-19T00:56:46.713ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 11369 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:49323 req_id = 78916ea8-8732-4a0d-b00e-483b8d61fd90 response_code = 500 uri = /timesync
7812024-12-19T00:56:46.716ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
7822024-12-19T00:56:46.891ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
7832024-12-19T00:56:46.891ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:48856
7842024-12-19T00:56:48.935ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 7357276260268799775 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
7852024-12-19T00:56:48.935ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11866606071799879623 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
7862024-12-19T00:56:49.571ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 10699755921578253876 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
7872024-12-19T00:56:49.577ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 10208574803446044792 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
7882024-12-19T00:56:49.918ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8023753382245910886 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
7892024-12-19T00:56:49.918ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 7374314792341526415 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
7902024-12-19T00:56:49.966ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 5083314779294559159 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
7912024-12-19T00:56:50.901ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3915
7922024-12-19T00:56:50.901ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 4009656 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:48856 req_id = 9197a2cb-a7f8-4741-9606-a8675b6dc687 response_code = 500 uri = /timesync
7932024-12-19T00:56:50.903ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
7942024-12-19T00:56:51.049ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
7952024-12-19T00:56:51.049ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37020
7962024-12-19T00:56:51.058ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3915
7972024-12-19T00:56:51.058ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 9150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:37020 req_id = a0f390ec-c3c5-43bf-b00d-76c6ba2a69bd response_code = 500 uri = /timesync
7982024-12-19T00:56:51.061ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
7992024-12-19T00:56:51.192ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
8002024-12-19T00:56:51.192ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43523
8012024-12-19T00:56:54.943ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_dfd796d3-f5de-49d4-a936-e01153b80282': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3915
8022024-12-19T00:56:54.944ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 4009515 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43523 req_id = ab5a0158-e8b4-4969-bcfa-fb77baed39c2 response_code = 500 uri = /timesync
8032024-12-19T00:56:54.948ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
8042024-12-19T00:56:55.221ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
8052024-12-19T00:56:55.221ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:51510
8062024-12-19T00:56:55.242ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 20842 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:51510 req_id = 8759f0d2-bb31-4780-916c-0710ab36f0e1 response_code = 200 uri = /timesync
8072024-12-19T00:56:55.245ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 759117325, ip_addr: 45.63.54.13, stratum: 3, ref_time: 1734569814.944016, correction: 0.026489377 } file = sled-agent/src/rack_setup/service.rs:727
8082024-12-19T00:56:55.245ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1412
8092024-12-19T00:56:55.275ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 0018a454-75c0-41b1-be69-9476391f01e1 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: dfd796d3-f5de-49d4-a936-e01153b80282 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: bf880c19-9d84-4efc-886f-98b14f937093, kind: Service { id: dfd796d3-f5de-49d4-a936-e01153b80282 }, name: Name("ntp-dfd796d3-f5de-49d4-a936-e01153b80282"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 178, 125])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 27c4eb2e-2ae3-402e-8856-5a021868fe90 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } } } }, OmicronZoneConfig { id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 24809a5d-52aa-4891-9dcb-84b4656ea172 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } } } }, OmicronZoneConfig { id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16 (service), filesystem_pool: Some(ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
8102024-12-19T00:56:55.275ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59652
8112024-12-19T00:56:55.275ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8122024-12-19T00:56:55.275ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }
8132024-12-19T00:56:55.310ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8142024-12-19T00:56:55.310ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }
8152024-12-19T00:56:55.341ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8162024-12-19T00:56:55.341ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }
8172024-12-19T00:56:55.373ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8182024-12-19T00:56:55.373ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }
8192024-12-19T00:56:55.404ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8202024-12-19T00:56:55.404ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
8212024-12-19T00:56:55.434ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8222024-12-19T00:56:55.434ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }
8232024-12-19T00:56:55.466ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8242024-12-19T00:56:55.466ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8252024-12-19T00:56:55.498ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8262024-12-19T00:56:55.498ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8272024-12-19T00:56:55.532ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8282024-12-19T00:56:55.532ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
8292024-12-19T00:56:55.532ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
8302024-12-19T00:56:55.533ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8312024-12-19T00:56:55.564ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
8322024-12-19T00:56:55.564ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
8332024-12-19T00:56:55.564ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
8342024-12-19T00:56:55.564ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
8352024-12-19T00:56:55.564ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
8362024-12-19T00:56:55.565ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172", TimeNotSynchronized), ("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16", TimeNotSynchronized), ("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1", TimeNotSynchronized), ("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90", TimeNotSynchronized), ("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 288619 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59652 req_id = 6f56dbdd-01c9-44a0-be37-de154e026bf7 response_code = 500 uri = /omicron-zones
8372024-12-19T00:56:55.565ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "6f56dbdd-01c9-44a0-be37-de154e026bf7", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:54 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "6f56dbdd-01c9-44a0-be37-de154e026bf7" } file = sled-agent/src/rack_setup/service.rs:562 retry_after = 69.344659ms sled_address = [fd00:1122:3344:101::1]:12345
8382024-12-19T00:56:55.635ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 0018a454-75c0-41b1-be69-9476391f01e1 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: dfd796d3-f5de-49d4-a936-e01153b80282 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: bf880c19-9d84-4efc-886f-98b14f937093, kind: Service { id: dfd796d3-f5de-49d4-a936-e01153b80282 }, name: Name("ntp-dfd796d3-f5de-49d4-a936-e01153b80282"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 178, 125])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 27c4eb2e-2ae3-402e-8856-5a021868fe90 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } } } }, OmicronZoneConfig { id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 24809a5d-52aa-4891-9dcb-84b4656ea172 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } } } }, OmicronZoneConfig { id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16 (service), filesystem_pool: Some(ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
8392024-12-19T00:56:55.635ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8402024-12-19T00:56:55.636ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }
8412024-12-19T00:56:55.669ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8422024-12-19T00:56:55.669ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }
8432024-12-19T00:56:55.699ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8442024-12-19T00:56:55.699ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }
8452024-12-19T00:56:55.731ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8462024-12-19T00:56:55.731ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }
8472024-12-19T00:56:55.762ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8482024-12-19T00:56:55.762ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
8492024-12-19T00:56:55.795ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8502024-12-19T00:56:55.795ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }
8512024-12-19T00:56:55.826ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8522024-12-19T00:56:55.826ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8532024-12-19T00:56:55.859ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8542024-12-19T00:56:55.859ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8552024-12-19T00:56:55.894ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8562024-12-19T00:56:55.894ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
8572024-12-19T00:56:55.894ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
8582024-12-19T00:56:55.894ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8592024-12-19T00:56:55.927ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
8602024-12-19T00:56:55.927ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
8612024-12-19T00:56:55.927ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
8622024-12-19T00:56:55.927ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
8632024-12-19T00:56:55.927ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
8642024-12-19T00:56:55.927ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172", TimeNotSynchronized), ("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16", TimeNotSynchronized), ("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1", TimeNotSynchronized), ("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99", TimeNotSynchronized), ("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 291236 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59652 req_id = 386ebf59-af7e-4632-9fad-49f4999aa20a response_code = 500 uri = /omicron-zones
8652024-12-19T00:56:55.927ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "386ebf59-af7e-4632-9fad-49f4999aa20a", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:54 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "386ebf59-af7e-4632-9fad-49f4999aa20a" } file = sled-agent/src/rack_setup/service.rs:562 retry_after = 70.179081ms sled_address = [fd00:1122:3344:101::1]:12345
8662024-12-19T00:56:56.000ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 0018a454-75c0-41b1-be69-9476391f01e1 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: dfd796d3-f5de-49d4-a936-e01153b80282 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: bf880c19-9d84-4efc-886f-98b14f937093, kind: Service { id: dfd796d3-f5de-49d4-a936-e01153b80282 }, name: Name("ntp-dfd796d3-f5de-49d4-a936-e01153b80282"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 178, 125])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 27c4eb2e-2ae3-402e-8856-5a021868fe90 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } } } }, OmicronZoneConfig { id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 24809a5d-52aa-4891-9dcb-84b4656ea172 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } } } }, OmicronZoneConfig { id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16 (service), filesystem_pool: Some(ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
8672024-12-19T00:56:56.000ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8682024-12-19T00:56:56.000ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }
8692024-12-19T00:56:56.037ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8702024-12-19T00:56:56.037ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }
8712024-12-19T00:56:56.069ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8722024-12-19T00:56:56.069ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }
8732024-12-19T00:56:56.102ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8742024-12-19T00:56:56.102ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }
8752024-12-19T00:56:56.133ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8762024-12-19T00:56:56.133ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
8772024-12-19T00:56:56.163ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8782024-12-19T00:56:56.163ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }
8792024-12-19T00:56:56.196ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8802024-12-19T00:56:56.196ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8812024-12-19T00:56:56.228ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8822024-12-19T00:56:56.228ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8832024-12-19T00:56:56.262ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8842024-12-19T00:56:56.262ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
8852024-12-19T00:56:56.263ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
8862024-12-19T00:56:56.263ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8872024-12-19T00:56:56.295ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
8882024-12-19T00:56:56.295ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
8892024-12-19T00:56:56.295ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
8902024-12-19T00:56:56.295ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
8912024-12-19T00:56:56.295ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
8922024-12-19T00:56:56.295ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172", TimeNotSynchronized), ("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1", TimeNotSynchronized), ("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16", TimeNotSynchronized), ("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90", TimeNotSynchronized), ("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 294329 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59652 req_id = 3c83a73a-5181-41f4-a272-9f41f7593ff8 response_code = 500 uri = /omicron-zones
8932024-12-19T00:56:56.295ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "3c83a73a-5181-41f4-a272-9f41f7593ff8", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:56 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "3c83a73a-5181-41f4-a272-9f41f7593ff8" } file = sled-agent/src/rack_setup/service.rs:562 retry_after = 82.447954ms sled_address = [fd00:1122:3344:101::1]:12345
8942024-12-19T00:56:56.380ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 0018a454-75c0-41b1-be69-9476391f01e1 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: dfd796d3-f5de-49d4-a936-e01153b80282 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: bf880c19-9d84-4efc-886f-98b14f937093, kind: Service { id: dfd796d3-f5de-49d4-a936-e01153b80282 }, name: Name("ntp-dfd796d3-f5de-49d4-a936-e01153b80282"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 178, 125])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 27c4eb2e-2ae3-402e-8856-5a021868fe90 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } } } }, OmicronZoneConfig { id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 24809a5d-52aa-4891-9dcb-84b4656ea172 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } } } }, OmicronZoneConfig { id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16 (service), filesystem_pool: Some(ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
8952024-12-19T00:56:56.380ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8962024-12-19T00:56:56.380ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }
8972024-12-19T00:56:56.413ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8982024-12-19T00:56:56.413ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }
8992024-12-19T00:56:56.446ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9002024-12-19T00:56:56.446ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }
9012024-12-19T00:56:56.477ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9022024-12-19T00:56:56.477ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }
9032024-12-19T00:56:56.509ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9042024-12-19T00:56:56.509ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
9052024-12-19T00:56:56.541ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9062024-12-19T00:56:56.541ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }
9072024-12-19T00:56:56.573ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9082024-12-19T00:56:56.573ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9092024-12-19T00:56:56.605ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9102024-12-19T00:56:56.605ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9112024-12-19T00:56:56.637ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9122024-12-19T00:56:56.637ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
9132024-12-19T00:56:56.637ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
9142024-12-19T00:56:56.637ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9152024-12-19T00:56:56.670ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
9162024-12-19T00:56:56.670ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
9172024-12-19T00:56:56.670ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
9182024-12-19T00:56:56.670ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
9192024-12-19T00:56:56.670ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
9202024-12-19T00:56:56.670ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90", TimeNotSynchronized), ("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16", TimeNotSynchronized), ("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172", TimeNotSynchronized), ("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99", TimeNotSynchronized), ("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 289407 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59652 req_id = 87e85d58-12b2-4265-b322-8743b9c5f5db response_code = 500 uri = /omicron-zones
9212024-12-19T00:56:56.670ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "87e85d58-12b2-4265-b322-8743b9c5f5db", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:56 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "87e85d58-12b2-4265-b322-8743b9c5f5db" } file = sled-agent/src/rack_setup/service.rs:562 retry_after = 150.869419ms sled_address = [fd00:1122:3344:101::1]:12345
9222024-12-19T00:56:56.823ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 0018a454-75c0-41b1-be69-9476391f01e1 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: dfd796d3-f5de-49d4-a936-e01153b80282 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: bf880c19-9d84-4efc-886f-98b14f937093, kind: Service { id: dfd796d3-f5de-49d4-a936-e01153b80282 }, name: Name("ntp-dfd796d3-f5de-49d4-a936-e01153b80282"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 178, 125])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 27c4eb2e-2ae3-402e-8856-5a021868fe90 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } } } }, OmicronZoneConfig { id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 24809a5d-52aa-4891-9dcb-84b4656ea172 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } } } }, OmicronZoneConfig { id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16 (service), filesystem_pool: Some(ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
9232024-12-19T00:56:56.823ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9242024-12-19T00:56:56.823ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }
9252024-12-19T00:56:56.860ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9262024-12-19T00:56:56.860ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }
9272024-12-19T00:56:56.894ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9282024-12-19T00:56:56.894ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }
9292024-12-19T00:56:56.925ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9302024-12-19T00:56:56.925ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }
9312024-12-19T00:56:56.955ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9322024-12-19T00:56:56.955ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
9332024-12-19T00:56:56.988ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9342024-12-19T00:56:56.988ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }
9352024-12-19T00:56:57.022ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9362024-12-19T00:56:57.022ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9372024-12-19T00:56:57.054ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9382024-12-19T00:56:57.054ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9392024-12-19T00:56:57.087ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9402024-12-19T00:56:57.087ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
9412024-12-19T00:56:57.087ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
9422024-12-19T00:56:57.087ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9432024-12-19T00:56:57.120ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
9442024-12-19T00:56:57.120ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
9452024-12-19T00:56:57.120ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
9462024-12-19T00:56:57.120ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
9472024-12-19T00:56:57.120ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
9482024-12-19T00:56:57.120ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172", TimeNotSynchronized), ("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90", TimeNotSynchronized), ("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16", TimeNotSynchronized), ("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99", TimeNotSynchronized), ("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 296383 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59652 req_id = 266fb273-1651-4dee-8f92-51a372ccbb49 response_code = 500 uri = /omicron-zones
9492024-12-19T00:56:57.121ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "266fb273-1651-4dee-8f92-51a372ccbb49", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:56 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "266fb273-1651-4dee-8f92-51a372ccbb49" } file = sled-agent/src/rack_setup/service.rs:562 retry_after = 134.776177ms sled_address = [fd00:1122:3344:101::1]:12345
9502024-12-19T00:56:57.257ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 0018a454-75c0-41b1-be69-9476391f01e1 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: dfd796d3-f5de-49d4-a936-e01153b80282 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: bf880c19-9d84-4efc-886f-98b14f937093, kind: Service { id: dfd796d3-f5de-49d4-a936-e01153b80282 }, name: Name("ntp-dfd796d3-f5de-49d4-a936-e01153b80282"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 178, 125])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 27c4eb2e-2ae3-402e-8856-5a021868fe90 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } } } }, OmicronZoneConfig { id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 24809a5d-52aa-4891-9dcb-84b4656ea172 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } } } }, OmicronZoneConfig { id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16 (service), filesystem_pool: Some(ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
9512024-12-19T00:56:57.258ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9522024-12-19T00:56:57.258ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }
9532024-12-19T00:56:57.291ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9542024-12-19T00:56:57.291ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }
9552024-12-19T00:56:57.323ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9562024-12-19T00:56:57.323ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }
9572024-12-19T00:56:57.354ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9582024-12-19T00:56:57.354ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }
9592024-12-19T00:56:57.386ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9602024-12-19T00:56:57.386ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
9612024-12-19T00:56:57.418ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9622024-12-19T00:56:57.418ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }
9632024-12-19T00:56:57.449ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9642024-12-19T00:56:57.449ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9652024-12-19T00:56:57.481ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9662024-12-19T00:56:57.481ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9672024-12-19T00:56:57.515ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9682024-12-19T00:56:57.515ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
9692024-12-19T00:56:57.515ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
9702024-12-19T00:56:57.515ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9712024-12-19T00:56:57.547ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
9722024-12-19T00:56:57.547ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
9732024-12-19T00:56:57.547ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
9742024-12-19T00:56:57.547ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
9752024-12-19T00:56:57.547ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
9762024-12-19T00:56:57.547ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16", TimeNotSynchronized), ("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1", TimeNotSynchronized), ("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99", TimeNotSynchronized), ("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172", TimeNotSynchronized), ("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 288669 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59652 req_id = 6a8e0ae2-f9ce-4d57-9f83-2d103f467c23 response_code = 500 uri = /omicron-zones
9772024-12-19T00:56:57.547ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "6a8e0ae2-f9ce-4d57-9f83-2d103f467c23", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:57 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "6a8e0ae2-f9ce-4d57-9f83-2d103f467c23" } file = sled-agent/src/rack_setup/service.rs:562 retry_after = 151.617903ms sled_address = [fd00:1122:3344:101::1]:12345
9782024-12-19T00:56:57.622ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 256.52951ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
9792024-12-19T00:56:57.700ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 0018a454-75c0-41b1-be69-9476391f01e1 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: dfd796d3-f5de-49d4-a936-e01153b80282 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: bf880c19-9d84-4efc-886f-98b14f937093, kind: Service { id: dfd796d3-f5de-49d4-a936-e01153b80282 }, name: Name("ntp-dfd796d3-f5de-49d4-a936-e01153b80282"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 178, 125])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 27c4eb2e-2ae3-402e-8856-5a021868fe90 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } } } }, OmicronZoneConfig { id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 24809a5d-52aa-4891-9dcb-84b4656ea172 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } } } }, OmicronZoneConfig { id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16 (service), filesystem_pool: Some(ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
9802024-12-19T00:56:57.700ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9812024-12-19T00:56:57.700ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }
9822024-12-19T00:56:57.733ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9832024-12-19T00:56:57.733ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }
9842024-12-19T00:56:57.765ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9852024-12-19T00:56:57.765ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }
9862024-12-19T00:56:57.797ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9872024-12-19T00:56:57.797ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }
9882024-12-19T00:56:57.830ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9892024-12-19T00:56:57.830ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
9902024-12-19T00:56:57.864ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9912024-12-19T00:56:57.864ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }
9922024-12-19T00:56:57.897ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9932024-12-19T00:56:57.897ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9942024-12-19T00:56:57.929ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9952024-12-19T00:56:57.929ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9962024-12-19T00:56:57.962ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9972024-12-19T00:56:57.962ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
9982024-12-19T00:56:57.962ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
9992024-12-19T00:56:57.962ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10002024-12-19T00:56:57.996ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
10012024-12-19T00:56:57.996ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
10022024-12-19T00:56:57.996ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
10032024-12-19T00:56:57.996ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
10042024-12-19T00:56:57.996ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
10052024-12-19T00:56:57.996ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90", TimeNotSynchronized), ("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99", TimeNotSynchronized), ("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1", TimeNotSynchronized), ("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172", TimeNotSynchronized), ("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 295317 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59652 req_id = 8b5a546f-10e0-4c3f-8903-5653a3144580 response_code = 500 uri = /omicron-zones
10062024-12-19T00:56:57.996ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "8b5a546f-10e0-4c3f-8903-5653a3144580", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:57 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "8b5a546f-10e0-4c3f-8903-5653a3144580" } file = sled-agent/src/rack_setup/service.rs:562 retry_after = 382.91355ms sled_address = [fd00:1122:3344:101::1]:12345
10072024-12-19T00:56:58.382ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 0018a454-75c0-41b1-be69-9476391f01e1 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: dfd796d3-f5de-49d4-a936-e01153b80282 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: bf880c19-9d84-4efc-886f-98b14f937093, kind: Service { id: dfd796d3-f5de-49d4-a936-e01153b80282 }, name: Name("ntp-dfd796d3-f5de-49d4-a936-e01153b80282"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 178, 125])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 27c4eb2e-2ae3-402e-8856-5a021868fe90 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } } } }, OmicronZoneConfig { id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 24809a5d-52aa-4891-9dcb-84b4656ea172 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } } } }, OmicronZoneConfig { id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16 (service), filesystem_pool: Some(ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
10082024-12-19T00:56:58.382ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10092024-12-19T00:56:58.382ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }
10102024-12-19T00:56:58.416ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10112024-12-19T00:56:58.416ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }
10122024-12-19T00:56:58.447ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10132024-12-19T00:56:58.447ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }
10142024-12-19T00:56:58.478ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10152024-12-19T00:56:58.478ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }
10162024-12-19T00:56:58.510ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10172024-12-19T00:56:58.510ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
10182024-12-19T00:56:58.542ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10192024-12-19T00:56:58.542ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }
10202024-12-19T00:56:58.573ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10212024-12-19T00:56:58.573ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10222024-12-19T00:56:58.606ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10232024-12-19T00:56:58.606ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10242024-12-19T00:56:58.639ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10252024-12-19T00:56:58.639ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
10262024-12-19T00:56:58.639ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
10272024-12-19T00:56:58.639ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10282024-12-19T00:56:58.671ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
10292024-12-19T00:56:58.671ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
10302024-12-19T00:56:58.671ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
10312024-12-19T00:56:58.671ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
10322024-12-19T00:56:58.671ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
10332024-12-19T00:56:58.671ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1", TimeNotSynchronized), ("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16", TimeNotSynchronized), ("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172", TimeNotSynchronized), ("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99", TimeNotSynchronized), ("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 288349 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59652 req_id = 41f36180-aa4f-4ebc-8453-0764bd6aaddf response_code = 500 uri = /omicron-zones
10342024-12-19T00:56:58.671ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "41f36180-aa4f-4ebc-8453-0764bd6aaddf", "content-length": "124", "date": "Thu, 19 Dec 2024 00:56:58 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "41f36180-aa4f-4ebc-8453-0764bd6aaddf" } file = sled-agent/src/rack_setup/service.rs:562 retry_after = 309.723467ms sled_address = [fd00:1122:3344:101::1]:12345
10352024-12-19T00:56:58.676ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62172
10362024-12-19T00:56:58.676ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62172 req_id = 4249a578-1092-44d4-ae15-17da6e777ec0 response_code = 200 uri = /baseboard
10372024-12-19T00:56:58.983ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 0018a454-75c0-41b1-be69-9476391f01e1 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: dfd796d3-f5de-49d4-a936-e01153b80282 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: bf880c19-9d84-4efc-886f-98b14f937093, kind: Service { id: dfd796d3-f5de-49d4-a936-e01153b80282 }, name: Name("ntp-dfd796d3-f5de-49d4-a936-e01153b80282"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 178, 125])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 27c4eb2e-2ae3-402e-8856-5a021868fe90 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } } } }, OmicronZoneConfig { id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 24809a5d-52aa-4891-9dcb-84b4656ea172 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } } } }, OmicronZoneConfig { id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16 (service), filesystem_pool: Some(ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
10382024-12-19T00:56:58.984ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10392024-12-19T00:56:58.984ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }
10402024-12-19T00:56:59.017ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10412024-12-19T00:56:59.017ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }
10422024-12-19T00:56:59.048ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10432024-12-19T00:56:59.048ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }
10442024-12-19T00:56:59.080ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10452024-12-19T00:56:59.080ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }
10462024-12-19T00:56:59.112ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10472024-12-19T00:56:59.112ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
10482024-12-19T00:56:59.148ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10492024-12-19T00:56:59.148ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }
10502024-12-19T00:56:59.179ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10512024-12-19T00:56:59.179ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10522024-12-19T00:56:59.210ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10532024-12-19T00:56:59.210ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10542024-12-19T00:56:59.243ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10552024-12-19T00:56:59.243ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
10562024-12-19T00:56:59.243ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
10572024-12-19T00:56:59.244ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10582024-12-19T00:56:59.323ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10592024-12-19T00:56:59.323ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10602024-12-19T00:56:59.323ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10612024-12-19T00:56:59.323ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10622024-12-19T00:56:59.323ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10632024-12-19T00:56:59.410ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 file = illumos-utils/src/zone.rs:325
10642024-12-19T00:56:59.414ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16 file = illumos-utils/src/zone.rs:325
10652024-12-19T00:56:59.418ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90 file = illumos-utils/src/zone.rs:325
10662024-12-19T00:56:59.423ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172 file = illumos-utils/src/zone.rs:325
10672024-12-19T00:56:59.428ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1 file = illumos-utils/src/zone.rs:325
10682024-12-19T00:56:59.432ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 file = illumos-utils/src/zone.rs:363
10692024-12-19T00:56:59.438ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16 file = illumos-utils/src/zone.rs:363
10702024-12-19T00:56:59.443ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90 file = illumos-utils/src/zone.rs:363
10712024-12-19T00:56:59.448ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172 file = illumos-utils/src/zone.rs:363
10722024-12-19T00:56:59.452ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1 file = illumos-utils/src/zone.rs:363
10732024-12-19T00:57:02.310ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='9f2f1d58-18bd-480b-bc6a-222f7c7838e1'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10742024-12-19T00:57:02.310ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
10752024-12-19T00:57:02.325ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='24809a5d-52aa-4891-9dcb-84b4656ea172'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10762024-12-19T00:57:02.325ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
10772024-12-19T00:57:02.381ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10782024-12-19T00:57:02.381ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
10792024-12-19T00:57:02.457ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 32.579741ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
10802024-12-19T00:57:02.458ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='27c4eb2e-2ae3-402e-8856-5a021868fe90'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10812024-12-19T00:57:02.467ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1") failed: Property not found. retry in 42.269083ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
10822024-12-19T00:57:02.467ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
10832024-12-19T00:57:02.473ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='789f31ab-07f2-47c6-8dc0-f9cf998d2f16'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10842024-12-19T00:57:02.473ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
10852024-12-19T00:57:02.499ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 102.523941ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
10862024-12-19T00:57:02.517ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1") failed: Property not found. retry in 54.947692ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
10872024-12-19T00:57:02.525ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99") failed: Property not found. retry in 41.89224ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
10882024-12-19T00:57:02.684ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99") failed: Property not found. retry in 108.216732ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
10892024-12-19T00:57:03.296ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 136.382174ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
10902024-12-19T00:57:03.413ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1") failed: Property not found. retry in 179.852968ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
10912024-12-19T00:57:03.474ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99") failed: Property not found. retry in 104.602732ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
10922024-12-19T00:57:03.533ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 266.54879ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
10932024-12-19T00:57:03.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99") failed: Property not found. retry in 373.060679ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
10942024-12-19T00:57:04.028ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 52.344283ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
10952024-12-19T00:57:04.060ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 882.510028ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
10962024-12-19T00:57:04.082ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1") failed: Property not found. retry in 280.023527ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
10972024-12-19T00:57:04.116ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16") failed: Property not found. retry in 52.094239ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
10982024-12-19T00:57:04.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 93.808456ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
10992024-12-19T00:57:04.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16") failed: Property not found. retry in 77.940467ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
11002024-12-19T00:57:04.314ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 262.927382ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11012024-12-19T00:57:04.380ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99") failed: Property not found. retry in 825.642111ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
11022024-12-19T00:57:04.397ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16") failed: Property not found. retry in 248.418488ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
11032024-12-19T00:57:04.723ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1") failed: Property not found. retry in 481.499131ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
11042024-12-19T00:57:04.799ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16") failed: Property not found. retry in 316.720079ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
11052024-12-19T00:57:04.870ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 373.834911ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11062024-12-19T00:57:04.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 620.778385ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11072024-12-19T00:57:05.146ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16") failed: Property not found. retry in 1.129732334s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
11082024-12-19T00:57:05.243ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1") failed: Property not found. retry in 1.107961902s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
11092024-12-19T00:57:05.259ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99") failed: Property not found. retry in 1.124788903s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
11102024-12-19T00:57:05.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 788.135895ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11112024-12-19T00:57:05.614ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 731.327028ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11122024-12-19T00:57:06.085ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 1.278136349s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11132024-12-19T00:57:06.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16") failed: Property not found. retry in 591.106036ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
11142024-12-19T00:57:06.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 1.457242301s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11152024-12-19T00:57:06.363ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1") failed: Property not found. retry in 776.5328ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
11162024-12-19T00:57:06.393ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99") failed: Property not found. retry in 1.042570786s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
11172024-12-19T00:57:06.889ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16") failed: Property not found. retry in 596.383569ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
11182024-12-19T00:57:07.156ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1") failed: Property not found. retry in 1.278900768s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
11192024-12-19T00:57:07.374ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 1.171332929s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11202024-12-19T00:57:07.447ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99") failed: Property not found. retry in 1.271792444s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
11212024-12-19T00:57:07.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16") failed: Property not found. retry in 1.025089791s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
11222024-12-19T00:57:07.820ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 929.014754ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11232024-12-19T00:57:08.442ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1") failed: Property not found. retry in 794.2877ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
11242024-12-19T00:57:08.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16") failed: Property not found. retry in 1.345091349s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
11252024-12-19T00:57:08.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 873.417994ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11262024-12-19T00:57:08.726ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99") failed: Property not found. retry in 520.358444ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
11272024-12-19T00:57:08.758ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 514.900801ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11282024-12-19T00:57:09.245ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1") failed: Property not found. retry in 1.365914609s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
11292024-12-19T00:57:09.256ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99") failed: Property not found. retry in 553.958746ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
11302024-12-19T00:57:09.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 1.052359559s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11312024-12-19T00:57:09.436ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 855.473078ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11322024-12-19T00:57:09.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99") failed: Property not found. retry in 745.194722ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
11332024-12-19T00:57:09.884ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16") failed: Property not found. retry in 1.328928949s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
11342024-12-19T00:57:10.302ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 623.671711ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11352024-12-19T00:57:10.344ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 910.630419ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11362024-12-19T00:57:10.579ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99") failed: Property not found. retry in 867.598025ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
11372024-12-19T00:57:10.622ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1") failed: Property not found. retry in 815.537929ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
11382024-12-19T00:57:10.935ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 612.567162ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11392024-12-19T00:57:11.264ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 1.338149356s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11402024-12-19T00:57:11.557ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 583.801246ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11412024-12-19T00:57:12.152ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 1.101244885s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11422024-12-19T00:57:12.617ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 996.13763ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11432024-12-19T00:57:12.948ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 434.957255ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
11442024-12-19T00:57:13.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 1.426752356s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11452024-12-19T00:57:13.627ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 813.562543ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11462024-12-19T00:57:14.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 1.269046279s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11472024-12-19T00:57:14.704ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 1.175088419s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11482024-12-19T00:57:15.735ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 858.447717ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11492024-12-19T00:57:15.889ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 933.79493ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11502024-12-19T00:57:16.601ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 1.451900384s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11512024-12-19T00:57:16.832ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 1.041059426s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11522024-12-19T00:57:17.882ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 618.772604ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11532024-12-19T00:57:18.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 682.620633ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11542024-12-19T00:57:18.512ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 903.621912ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11552024-12-19T00:57:18.755ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 760.159322ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11562024-12-19T00:57:19.424ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 1.040812059s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11572024-12-19T00:57:19.523ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 1.07449529s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11582024-12-19T00:57:20.474ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90") failed: Property not found. retry in 789.448931ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11592024-12-19T00:57:20.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172") failed: Property not found. retry in 538.576624ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11602024-12-19T00:57:21.284ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_cockroachdb_fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99
11612024-12-19T00:57:21.284ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_cockroachdb_789f31ab-07f2-47c6-8dc0-f9cf998d2f16
11622024-12-19T00:57:21.284ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_cockroachdb_27c4eb2e-2ae3-402e-8856-5a021868fe90
11632024-12-19T00:57:21.284ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172
11642024-12-19T00:57:21.284ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_cockroachdb_9f2f1d58-18bd-480b-bc6a-222f7c7838e1
11652024-12-19T00:57:21.284ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
11662024-12-19T00:57:21.284ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
11672024-12-19T00:57:21.284ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 22284909 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59652 req_id = 5ba81d18-eb38-454a-9d74-c678203c2375 response_code = 204 uri = /omicron-zones
11682024-12-19T00:57:21.318ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37204
11692024-12-19T00:57:21.345ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService8/omicron6 in oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService8/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26776 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:37204 req_id = 5c48ebcd-3c39-4743-b5eb-0ba88bcb54e2 response_code = 500 uri = /cockroachdb
11702024-12-19T00:57:21.345ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "5c48ebcd-3c39-4743-b5eb-0ba88bcb54e2", "content-length": "124", "date": "Thu, 19 Dec 2024 00:57:21 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "5c48ebcd-3c39-4743-b5eb-0ba88bcb54e2" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 138.560249ms
11712024-12-19T00:57:21.511ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService8/omicron6 in oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService8/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26069 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:37204 req_id = 073de739-7ba5-4875-a8ee-9d4f90b92d7a response_code = 500 uri = /cockroachdb
11722024-12-19T00:57:21.511ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "073de739-7ba5-4875-a8ee-9d4f90b92d7a", "content-length": "124", "date": "Thu, 19 Dec 2024 00:57:21 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "073de739-7ba5-4875-a8ee-9d4f90b92d7a" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 66.503144ms
11732024-12-19T00:57:21.605ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService8/omicron6 in oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_24809a5d-52aa-4891-9dcb-84b4656ea172 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService8/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26765 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:37204 req_id = 25e2b283-c274-4ac6-a86d-ccbe61df0ae8 response_code = 500 uri = /cockroachdb
11742024-12-19T00:57:21.605ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "25e2b283-c274-4ac6-a86d-ccbe61df0ae8", "content-length": "124", "date": "Thu, 19 Dec 2024 00:57:21 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "25e2b283-c274-4ac6-a86d-ccbe61df0ae8" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 171.523818ms
11752024-12-19T00:57:21.808ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::5]:32221 file = sled-agent/src/services.rs:3758
11762024-12-19T00:57:27.773ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3776
11772024-12-19T00:57:32.562ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3799
11782024-12-19T00:57:32.563ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10784654 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:37204 req_id = 525ad7ec-07ab-415b-a339-0fe7e833e972 response_code = 204 uri = /cockroachdb
11792024-12-19T00:57:32.564ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:54445
11802024-12-19T00:57:32.564ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 279 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:54445 req_id = 541c2ebb-dbee-472b-aba2-1d96634bb641 response_code = 200 uri = /baseboard
11812024-12-19T00:57:32.568ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.050137981s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
11822024-12-19T00:57:32.602ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: 0018a454-75c0-41b1-be69-9476391f01e1 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: dfd796d3-f5de-49d4-a936-e01153b80282 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: bf880c19-9d84-4efc-886f-98b14f937093, kind: Service { id: dfd796d3-f5de-49d4-a936-e01153b80282 }, name: Name("ntp-dfd796d3-f5de-49d4-a936-e01153b80282"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 178, 125])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 27c4eb2e-2ae3-402e-8856-5a021868fe90 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } } } }, OmicronZoneConfig { id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 24809a5d-52aa-4891-9dcb-84b4656ea172 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } } } }, OmicronZoneConfig { id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16 (service), filesystem_pool: Some(ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1 (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 679d67b5-9925-49ac-b78d-613e816394dc (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: d21d0095-8e84-486f-b3a4-f9dca167b9d5, kind: Service { id: 679d67b5-9925-49ac-b78d-613e816394dc }, name: Name("external-dns-679d67b5-9925-49ac-b78d-613e816394dc"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 205])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: 5b97cf45-6db1-45ed-9f7e-111d132f7ed4, kind: Service { id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 }, name: Name("external-dns-2a880a2e-2471-4fd8-9ad5-3e0c3fbde317"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 253])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: 82094354-655d-415c-9763-28731eb50642, kind: Service { id: e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 }, name: Name("nexus-e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 60])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: ef89a8b8-3cc5-4637-aee2-3b98f04f51be, kind: Service { id: 1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 }, name: Name("nexus-1d1e5635-dd2b-4bf7-8ace-6cb3ea634995"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 75])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 1176f008-4a9d-4835-a5aa-1a86ec3c619c (service), filesystem_pool: Some(ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: a5c3ad41-155b-469b-bb3f-412814b9d0ff, kind: Service { id: 1176f008-4a9d-4835-a5aa-1a86ec3c619c }, name: Name("nexus-1176f008-4a9d-4835-a5aa-1a86ec3c619c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 188])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: b4e7d6e3-7551-4da8-8529-1286ef6ee8a3 (service), filesystem_pool: Some(ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: 10bb6caa-5666-4747-9b6c-48ed7069541b (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } } } }, OmicronZoneConfig { id: 6892dc98-d04b-4b85-b4d0-03bd53995e09 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: 68e16a8a-5014-4120-9251-64b4b355b6b0 (service), filesystem_pool: Some(ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: 089dd97c-a5d4-48df-b3ea-a61a0b0c9f43 (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 3bc6a5d7-2061-434b-96ed-851ac7abddbb (service), filesystem_pool: Some(ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External } } } }, OmicronZoneConfig { id: 0c9fcc8e-cb8d-4c52-8025-4137b7053c7b (service), filesystem_pool: Some(ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External } } } }, OmicronZoneConfig { id: bd71449b-9ea4-4e76-b752-1d23c910c439 (service), filesystem_pool: Some(ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External } } } }, OmicronZoneConfig { id: 2a938d23-12fe-4a3d-bd93-2851f8d9be67 (service), filesystem_pool: Some(ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4b6f7643-a0fa-4766-8901-1e1892739acf (service), filesystem_pool: Some(ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 8965b93d-b61b-4461-83fe-8af7a539eefa (service), filesystem_pool: Some(ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External } } } }, OmicronZoneConfig { id: 352791c3-3ea8-4a2c-abed-df833c2e15f5 (service), filesystem_pool: Some(ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9c73f9dc-da7d-470f-8c17-891618bbbaad (service), filesystem_pool: Some(ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External } } } }, OmicronZoneConfig { id: 3aaec92f-d06e-48c2-961f-d8ab46b39dfc (service), filesystem_pool: Some(ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
11832024-12-19T00:57:32.603ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54104
11842024-12-19T00:57:32.603ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
11852024-12-19T00:57:32.603ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0018a454-75c0-41b1-be69-9476391f01e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: InternalDns }, responder: ... }
11862024-12-19T00:57:32.693ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
11872024-12-19T00:57:32.693ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2eb2a760-6b2b-4fdd-a606-eabb932aebf9, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: InternalDns }, responder: ... }
11882024-12-19T00:57:32.757ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
11892024-12-19T00:57:32.757ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 050cdbd7-4ebc-471c-b4dc-ec1b3741f763, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: InternalDns }, responder: ... }
11902024-12-19T00:57:32.833ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
11912024-12-19T00:57:32.833ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 27c4eb2e-2ae3-402e-8856-5a021868fe90, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Cockroach }, responder: ... }
11922024-12-19T00:57:32.890ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
11932024-12-19T00:57:32.890ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: fff1fd4f-b526-4fcb-9d36-d2c7c7ac1a99, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
11942024-12-19T00:57:32.949ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
11952024-12-19T00:57:32.949ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 24809a5d-52aa-4891-9dcb-84b4656ea172, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Cockroach }, responder: ... }
11962024-12-19T00:57:33.017ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
11972024-12-19T00:57:33.017ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789f31ab-07f2-47c6-8dc0-f9cf998d2f16, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11982024-12-19T00:57:33.075ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
11992024-12-19T00:57:33.075ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9f2f1d58-18bd-480b-bc6a-222f7c7838e1, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
12002024-12-19T00:57:33.144ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 679d67b5-9925-49ac-b78d-613e816394dc, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
12012024-12-19T00:57:33.144ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 679d67b5-9925-49ac-b78d-613e816394dc, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: ExternalDns }, responder: ... }
12022024-12-19T00:57:33.218ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
12032024-12-19T00:57:33.218ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: ExternalDns }, responder: ... }
12042024-12-19T00:57:33.287ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 10bb6caa-5666-4747-9b6c-48ed7069541b, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:603
12052024-12-19T00:57:33.287ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 10bb6caa-5666-4747-9b6c-48ed7069541b, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Clickhouse }, responder: ... }
12062024-12-19T00:57:33.343ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3bc6a5d7-2061-434b-96ed-851ac7abddbb, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12072024-12-19T00:57:33.343ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 3bc6a5d7-2061-434b-96ed-851ac7abddbb, dataset_name: DatasetName { pool_name: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, kind: Crucible }, responder: ... }
12082024-12-19T00:57:33.409ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0c9fcc8e-cb8d-4c52-8025-4137b7053c7b, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12092024-12-19T00:57:33.409ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0c9fcc8e-cb8d-4c52-8025-4137b7053c7b, dataset_name: DatasetName { pool_name: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, kind: Crucible }, responder: ... }
12102024-12-19T00:57:33.463ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: bd71449b-9ea4-4e76-b752-1d23c910c439, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12112024-12-19T00:57:33.463ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: bd71449b-9ea4-4e76-b752-1d23c910c439, dataset_name: DatasetName { pool_name: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, kind: Crucible }, responder: ... }
12122024-12-19T00:57:33.523ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2a938d23-12fe-4a3d-bd93-2851f8d9be67, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12132024-12-19T00:57:33.523ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2a938d23-12fe-4a3d-bd93-2851f8d9be67, dataset_name: DatasetName { pool_name: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, kind: Crucible }, responder: ... }
12142024-12-19T00:57:33.581ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4b6f7643-a0fa-4766-8901-1e1892739acf, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12152024-12-19T00:57:33.581ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 4b6f7643-a0fa-4766-8901-1e1892739acf, dataset_name: DatasetName { pool_name: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, kind: Crucible }, responder: ... }
12162024-12-19T00:57:33.663ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8965b93d-b61b-4461-83fe-8af7a539eefa, dataset_name: DatasetName { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12172024-12-19T00:57:33.663ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 8965b93d-b61b-4461-83fe-8af7a539eefa, dataset_name: DatasetName { pool_name: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, kind: Crucible }, responder: ... }
12182024-12-19T00:57:33.752ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 352791c3-3ea8-4a2c-abed-df833c2e15f5, dataset_name: DatasetName { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12192024-12-19T00:57:33.752ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 352791c3-3ea8-4a2c-abed-df833c2e15f5, dataset_name: DatasetName { pool_name: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, kind: Crucible }, responder: ... }
12202024-12-19T00:57:33.820ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9c73f9dc-da7d-470f-8c17-891618bbbaad, dataset_name: DatasetName { pool_name: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12212024-12-19T00:57:33.820ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9c73f9dc-da7d-470f-8c17-891618bbbaad, dataset_name: DatasetName { pool_name: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, kind: Crucible }, responder: ... }
12222024-12-19T00:57:33.908ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3aaec92f-d06e-48c2-961f-d8ab46b39dfc, dataset_name: DatasetName { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12232024-12-19T00:57:33.908ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 3aaec92f-d06e-48c2-961f-d8ab46b39dfc, dataset_name: DatasetName { pool_name: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, kind: Crucible }, responder: ... }
12242024-12-19T00:57:33.995ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12252024-12-19T00:57:33.995ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
12262024-12-19T00:57:33.995ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
12272024-12-19T00:57:33.995ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12282024-12-19T00:57:34.189ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12292024-12-19T00:57:34.190ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12302024-12-19T00:57:34.190ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12312024-12-19T00:57:34.190ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12322024-12-19T00:57:34.190ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12332024-12-19T00:57:34.190ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12342024-12-19T00:57:34.190ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12352024-12-19T00:57:34.215ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12362024-12-19T00:57:34.215ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12372024-12-19T00:57:34.215ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12382024-12-19T00:57:34.387ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12392024-12-19T00:57:34.387ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12402024-12-19T00:57:34.387ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12412024-12-19T00:57:34.387ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12422024-12-19T00:57:34.423ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12432024-12-19T00:57:34.423ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12442024-12-19T00:57:34.423ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12452024-12-19T00:57:34.423ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12462024-12-19T00:57:34.434ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12472024-12-19T00:57:34.435ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09 file = illumos-utils/src/zone.rs:325
12482024-12-19T00:57:34.439ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67 file = illumos-utils/src/zone.rs:325
12492024-12-19T00:57:34.444ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa file = illumos-utils/src/zone.rs:325
12502024-12-19T00:57:34.448ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5 file = illumos-utils/src/zone.rs:325
12512024-12-19T00:57:34.453ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc file = illumos-utils/src/zone.rs:325
12522024-12-19T00:57:34.458ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad file = illumos-utils/src/zone.rs:325
12532024-12-19T00:57:34.463ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0 file = illumos-utils/src/zone.rs:325
12542024-12-19T00:57:34.467ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf file = illumos-utils/src/zone.rs:325
12552024-12-19T00:57:34.588ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09 file = illumos-utils/src/zone.rs:363
12562024-12-19T00:57:34.594ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67 file = illumos-utils/src/zone.rs:363
12572024-12-19T00:57:34.599ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa file = illumos-utils/src/zone.rs:363
12582024-12-19T00:57:34.603ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5 file = illumos-utils/src/zone.rs:363
12592024-12-19T00:57:34.607ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc file = illumos-utils/src/zone.rs:363
12602024-12-19T00:57:34.612ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad file = illumos-utils/src/zone.rs:363
12612024-12-19T00:57:34.617ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0 file = illumos-utils/src/zone.rs:363
12622024-12-19T00:57:34.622ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf file = illumos-utils/src/zone.rs:363
12632024-12-19T00:57:34.627ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3 file = illumos-utils/src/zone.rs:325
12642024-12-19T00:57:34.632ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b file = illumos-utils/src/zone.rs:325
12652024-12-19T00:57:34.636ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43 file = illumos-utils/src/zone.rs:325
12662024-12-19T00:57:34.642ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb file = illumos-utils/src/zone.rs:325
12672024-12-19T00:57:34.648ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b file = illumos-utils/src/zone.rs:325
12682024-12-19T00:57:34.654ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439 file = illumos-utils/src/zone.rs:325
12692024-12-19T00:57:34.716ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12702024-12-19T00:57:34.716ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3 file = illumos-utils/src/zone.rs:363
12712024-12-19T00:57:34.732ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b file = illumos-utils/src/zone.rs:363
12722024-12-19T00:57:34.755ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43 file = illumos-utils/src/zone.rs:363
12732024-12-19T00:57:34.779ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb file = illumos-utils/src/zone.rs:363
12742024-12-19T00:57:34.803ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b file = illumos-utils/src/zone.rs:363
12752024-12-19T00:57:34.812ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439 file = illumos-utils/src/zone.rs:363
12762024-12-19T00:57:34.884ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12772024-12-19T00:57:34.940ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12782024-12-19T00:57:34.995ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12792024-12-19T00:57:34.998ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12802024-12-19T00:57:34.998ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12812024-12-19T00:57:35.051ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 215, 75]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
12822024-12-19T00:57:35.051ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = nexus
12832024-12-19T00:57:35.107ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12842024-12-19T00:57:35.110ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12852024-12-19T00:57:35.110ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12862024-12-19T00:57:35.170ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 167, 188]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
12872024-12-19T00:57:35.170ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = nexus
12882024-12-19T00:57:35.173ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:83
12892024-12-19T00:57:35.173ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:90
12902024-12-19T00:57:35.175ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12912024-12-19T00:57:35.175ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12922024-12-19T00:57:35.226ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 158, 253]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
12932024-12-19T00:57:35.226ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = external_dns
12942024-12-19T00:57:35.227ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:83
12952024-12-19T00:57:35.227ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:90
12962024-12-19T00:57:35.228ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12972024-12-19T00:57:35.228ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12982024-12-19T00:57:35.303ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 245, 205]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
12992024-12-19T00:57:35.303ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = external_dns
13002024-12-19T00:57:35.305ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13012024-12-19T00:57:35.305ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13022024-12-19T00:57:35.371ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 156, 60]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
13032024-12-19T00:57:35.371ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = nexus
13042024-12-19T00:57:35.371ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:118
13052024-12-19T00:57:35.373ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13062024-12-19T00:57:35.373ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:118
13072024-12-19T00:57:35.375ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13082024-12-19T00:57:35.476ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 file = illumos-utils/src/zone.rs:325
13092024-12-19T00:57:35.488ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:83
13102024-12-19T00:57:35.488ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:90
13112024-12-19T00:57:35.488ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:83
13122024-12-19T00:57:35.488ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:90
13132024-12-19T00:57:35.489ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:83
13142024-12-19T00:57:35.489ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:90
13152024-12-19T00:57:35.489ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:118
13162024-12-19T00:57:35.492ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13172024-12-19T00:57:35.492ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:118
13182024-12-19T00:57:35.495ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:118
13192024-12-19T00:57:35.499ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13202024-12-19T00:57:35.499ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13212024-12-19T00:57:35.532ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c file = illumos-utils/src/zone.rs:325
13222024-12-19T00:57:35.595ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 file = illumos-utils/src/zone.rs:363
13232024-12-19T00:57:35.601ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 file = illumos-utils/src/zone.rs:325
13242024-12-19T00:57:35.606ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c file = illumos-utils/src/zone.rs:363
13252024-12-19T00:57:35.613ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc file = illumos-utils/src/zone.rs:325
13262024-12-19T00:57:35.619ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 file = illumos-utils/src/zone.rs:325
13272024-12-19T00:57:35.626ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 file = illumos-utils/src/zone.rs:363
13282024-12-19T00:57:35.643ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc file = illumos-utils/src/zone.rs:363
13292024-12-19T00:57:35.649ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 file = illumos-utils/src/zone.rs:363
13302024-12-19T00:57:36.964ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13312024-12-19T00:57:36.968ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
13322024-12-19T00:57:37.031ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13332024-12-19T00:57:37.031ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
13342024-12-19T00:57:37.620ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_1025d0d6-c165-4f43-bef0-cb2eeb04b8c2/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='1986276d-dd66-4bf1-ae23-f4738125a0e5'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13352024-12-19T00:57:37.620ZINFOSledAgent (ServiceManager): Booting oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
13362024-12-19T00:57:37.628ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_499e53d3-d67c-44b6-9c03-b66767baed7e/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='6c0723a6-39f5-488f-9360-cefee94bd727'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13372024-12-19T00:57:37.628ZINFOSledAgent (ServiceManager): Booting oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
13382024-12-19T00:57:37.700ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='dac9b03e-07f9-4ba8-ac8c-1e865304b271'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13392024-12-19T00:57:37.701ZINFOSledAgent (ServiceManager): Booting oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
13402024-12-19T00:57:37.717ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_f4009784-7bde-44ff-854e-49d614e9e682/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='3e435f2a-8604-431f-ba6f-1ea503319fbb'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13412024-12-19T00:57:37.717ZINFOSledAgent (ServiceManager): Booting oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
13422024-12-19T00:57:37.736ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_98a100ce-cd82-4209-8530-7b48ca829744/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='9aa39443-d57b-4650-a0cf-7186631dce5a'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13432024-12-19T00:57:37.736ZINFOSledAgent (ServiceManager): Booting oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
13442024-12-19T00:57:37.748ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='7a4eeb90-dbed-411e-bdd1-a1f3d84a6155'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13452024-12-19T00:57:37.748ZINFOSledAgent (ServiceManager): Booting oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
13462024-12-19T00:57:37.768ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_93eb8a71-6519-482e-a366-b15e30b4605e/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='d45af3d7-1d0c-4245-b0cd-7c429e85e46b'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13472024-12-19T00:57:37.768ZINFOSledAgent (ServiceManager): Booting oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
13482024-12-19T00:57:45.084ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 54.810863ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
13492024-12-19T00:57:45.373ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 46.343819ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
13502024-12-19T00:57:45.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 53.371287ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
13512024-12-19T00:57:45.632ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 74.176888ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
13522024-12-19T00:57:45.796ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 63.060667ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
13532024-12-19T00:57:45.813ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 49.174128ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
13542024-12-19T00:57:45.829ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 38.216507ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
13552024-12-19T00:57:45.840ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 71.671669ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
13562024-12-19T00:57:45.851ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 56.184828ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
13572024-12-19T00:57:45.864ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 60.047442ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
13582024-12-19T00:57:45.878ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 54.818698ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
13592024-12-19T00:57:45.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 63.76347ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
13602024-12-19T00:57:45.906ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 124.856423ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
13612024-12-19T00:57:45.917ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 83.575185ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
13622024-12-19T00:57:45.929ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 95.470139ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
13632024-12-19T00:57:45.943ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 78.628213ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
13642024-12-19T00:57:46.766ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 63.456585ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
13652024-12-19T00:57:46.779ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 109.510902ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
13662024-12-19T00:57:46.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 283.957518ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
13672024-12-19T00:57:46.805ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 103.851452ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
13682024-12-19T00:57:46.817ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 263.907918ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
13692024-12-19T00:57:46.829ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 144.125203ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
13702024-12-19T00:57:46.843ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 158.060362ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
13712024-12-19T00:57:46.857ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 180.080288ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
13722024-12-19T00:57:46.872ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 254.487034ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
13732024-12-19T00:57:46.885ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 241.49624ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
13742024-12-19T00:57:46.903ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 181.745156ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
13752024-12-19T00:57:46.921ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 599.061342ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
13762024-12-19T00:57:46.989ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 325.565753ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
13772024-12-19T00:57:47.015ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 304.502239ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
13782024-12-19T00:57:47.051ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 585.577255ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
13792024-12-19T00:57:47.090ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 316.480225ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
13802024-12-19T00:57:47.100ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 249.405604ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
13812024-12-19T00:57:47.111ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 482.462702ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
13822024-12-19T00:57:47.140ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 234.098116ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
13832024-12-19T00:57:47.153ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 585.016239ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
13842024-12-19T00:57:47.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.166604268s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
13852024-12-19T00:57:47.342ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 410.889439ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
13862024-12-19T00:57:47.364ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 669.692591ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
13872024-12-19T00:57:47.389ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 843.78191ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
13882024-12-19T00:57:47.421ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 850.76634ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
13892024-12-19T00:57:47.536ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 975.063006ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
13902024-12-19T00:57:47.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 470.80032ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
13912024-12-19T00:57:47.651ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 883.069885ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
13922024-12-19T00:57:47.754ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 409.770803ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
13932024-12-19T00:57:47.770ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 1.440045806s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
13942024-12-19T00:57:47.962ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13952024-12-19T00:57:47.962ZINFOSledAgent (ServiceManager): Booting oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
13962024-12-19T00:57:48.050ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 819.770644ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
13972024-12-19T00:57:48.096ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.252043109s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
13982024-12-19T00:57:48.178ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.379106658s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
13992024-12-19T00:57:48.248ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 1.255222811s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
14002024-12-19T00:57:48.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 40.904749ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
14012024-12-19T00:57:48.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 950.544244ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
14022024-12-19T00:57:48.315ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 137.250365ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
14032024-12-19T00:57:48.465ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 119.064337ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
14042024-12-19T00:57:48.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 806.003077ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
14052024-12-19T00:57:48.528ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 1.328451563s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
14062024-12-19T00:57:48.549ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 1.01489014s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
14072024-12-19T00:57:48.601ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 376.860066ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
14082024-12-19T00:57:48.661ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.520321886s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
14092024-12-19T00:57:48.882ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.034541361s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
14102024-12-19T00:57:48.990ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 954.270663ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
14112024-12-19T00:57:49.222ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 860.581509ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
14122024-12-19T00:57:49.250ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 1.481552824s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
14132024-12-19T00:57:49.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 626.696066ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
14142024-12-19T00:57:49.572ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 616.843594ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
14152024-12-19T00:57:49.605ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 567.37578ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
14162024-12-19T00:57:49.628ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 898.361678ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
14172024-12-19T00:57:49.643ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 726.857507ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
14182024-12-19T00:57:49.643ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='b4e7d6e3-7551-4da8-8529-1286ef6ee8a3'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14192024-12-19T00:57:49.644ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='5958423c-6ea2-4bc5-b898-d1f2fe95076e'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14202024-12-19T00:57:49.645ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
14212024-12-19T00:57:49.658ZINFOSledAgent (ServiceManager): Booting oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
14222024-12-19T00:57:49.873ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 967.35519ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
14232024-12-19T00:57:49.938ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 657.697783ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
14242024-12-19T00:57:49.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 1.240539288s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
14252024-12-19T00:57:49.985ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.117634776s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
14262024-12-19T00:57:50.062ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 71.012675ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
14272024-12-19T00:57:50.098ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 529.279108ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
14282024-12-19T00:57:50.121ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 47.268295ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
14292024-12-19T00:57:50.147ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 149.462344ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
14302024-12-19T00:57:50.181ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 122.643838ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
14312024-12-19T00:57:50.193ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 731.725286ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
14322024-12-19T00:57:50.206ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 753.642826ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
14332024-12-19T00:57:50.228ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14342024-12-19T00:57:50.229ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
14352024-12-19T00:57:50.311ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 277.134564ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
14362024-12-19T00:57:50.323ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 215.287401ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
14372024-12-19T00:57:50.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 688.048848ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
14382024-12-19T00:57:50.541ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 701.533657ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
14392024-12-19T00:57:50.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 251.246266ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
14402024-12-19T00:57:50.602ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 334.770088ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
14412024-12-19T00:57:50.613ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.022835146s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
14422024-12-19T00:57:50.623ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 25.545479ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
14432024-12-19T00:57:50.639ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 869.389002ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
14442024-12-19T00:57:50.660ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 136.73498ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
14452024-12-19T00:57:50.742ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 974.956592ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
14462024-12-19T00:57:50.808ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 180.29307ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
14472024-12-19T00:57:50.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 1.014158954s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
14482024-12-19T00:57:50.851ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 606.506073ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
14492024-12-19T00:57:50.938ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 1.253596965s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
14502024-12-19T00:57:50.951ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 519.119042ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
14512024-12-19T00:57:50.972ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.346138395s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
14522024-12-19T00:57:50.997ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 203.938709ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
14532024-12-19T00:57:51.085ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.028052007s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
14542024-12-19T00:57:51.118ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 817.638137ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
14552024-12-19T00:57:51.216ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 497.287298ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
14562024-12-19T00:57:51.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 1.456820146s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
14572024-12-19T00:57:51.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 584.507475ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
14582024-12-19T00:57:51.364ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14592024-12-19T00:57:51.431ZINFOSledAgent (ServiceManager): Booting oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
14602024-12-19T00:57:51.468ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 1.046865396s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
14612024-12-19T00:57:51.482ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 631.015639ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
14622024-12-19T00:57:51.520ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 1.261121969s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
14632024-12-19T00:57:51.652ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.329642891s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
14642024-12-19T00:57:51.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 1.370456698s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
14652024-12-19T00:57:51.734ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 523.165209ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
14662024-12-19T00:57:51.759ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 47.162918ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
14672024-12-19T00:57:51.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 70.765563ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
14682024-12-19T00:57:51.848ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 1.491821725s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
14692024-12-19T00:57:51.859ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 1.461840672s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
14702024-12-19T00:57:51.900ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 287.723101ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
14712024-12-19T00:57:51.950ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 789.938307ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
14722024-12-19T00:57:52.128ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.047836789s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
14732024-12-19T00:57:52.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 1.319090792s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
14742024-12-19T00:57:52.200ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 568.173033ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
14752024-12-19T00:57:52.211ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 955.495535ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
14762024-12-19T00:57:52.270ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 877.687574ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
14772024-12-19T00:57:52.330ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.339806074s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
14782024-12-19T00:57:52.528ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 1.432654678s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
14792024-12-19T00:57:52.695ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 1.021544132s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
14802024-12-19T00:57:52.754ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 504.128868ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
14812024-12-19T00:57:52.783ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 797.730974ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
14822024-12-19T00:57:52.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 849.749856ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
14832024-12-19T00:57:52.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.423641698s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
14842024-12-19T00:57:53.108ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 861.975607ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
14852024-12-19T00:57:53.158ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 604.329946ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
14862024-12-19T00:57:53.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 926.436673ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
14872024-12-19T00:57:53.186ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.339410655s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
14882024-12-19T00:57:53.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 818.211349ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
14892024-12-19T00:57:53.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 1.025375409s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
14902024-12-19T00:57:53.348ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 802.953948ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
14912024-12-19T00:57:53.468ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 1.484267439s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
14922024-12-19T00:57:53.590ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 1.305383698s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
14932024-12-19T00:57:53.655ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 1.087358927s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
14942024-12-19T00:57:53.680ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.156092259s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
14952024-12-19T00:57:53.726ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 769.266458ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
14962024-12-19T00:57:53.771ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 669.188582ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
14972024-12-19T00:57:53.971ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 1.211811788s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
14982024-12-19T00:57:53.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 983.450259ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
14992024-12-19T00:57:54.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.358730182s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
15002024-12-19T00:57:54.113ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 565.402354ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
15012024-12-19T00:57:54.161ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 1.384731787s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
15022024-12-19T00:57:54.364ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 1.368627724s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
15032024-12-19T00:57:54.428ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.33214731s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
15042024-12-19T00:57:54.448ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 688.456523ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
15052024-12-19T00:57:54.505ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 867.878196ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
15062024-12-19T00:57:54.535ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.327025157s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
15072024-12-19T00:57:54.689ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 632.09192ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
15082024-12-19T00:57:54.752ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 1.16440156s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
15092024-12-19T00:57:54.845ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.309976212s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
15102024-12-19T00:57:54.905ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 1.347917645s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
15112024-12-19T00:57:54.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 1.271510524s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
15122024-12-19T00:57:54.973ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 511.098134ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
15132024-12-19T00:57:55.148ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 1.14134715s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
15142024-12-19T00:57:55.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 970.29015ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
15152024-12-19T00:57:55.332ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 1.266513433s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
15162024-12-19T00:57:55.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 1.283048025s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
15172024-12-19T00:57:55.460ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.179289498s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
15182024-12-19T00:57:55.493ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 1.11738556s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
15192024-12-19T00:57:55.556ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 1.313785297s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
15202024-12-19T00:57:55.742ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 1.420148403s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
15212024-12-19T00:57:55.769ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.071661528s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
15222024-12-19T00:57:55.870ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.449913363s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
15232024-12-19T00:57:55.926ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 554.352719ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
15242024-12-19T00:57:56.166ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.222348094s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
15252024-12-19T00:57:56.175ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 746.808111ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
15262024-12-19T00:57:56.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 803.118038ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
15272024-12-19T00:57:56.262ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 995.307819ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
15282024-12-19T00:57:56.298ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 1.217663035s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
15292024-12-19T00:57:56.489ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 1.096554633s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
15302024-12-19T00:57:56.608ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 551.265452ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
15312024-12-19T00:57:56.619ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 1.250223057s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
15322024-12-19T00:57:56.648ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.350335965s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
15332024-12-19T00:57:56.675ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 1.080175553s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
15342024-12-19T00:57:56.850ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.105738121s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
15352024-12-19T00:57:56.878ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 1.180181554s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
15362024-12-19T00:57:56.930ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 1.096931352s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
15372024-12-19T00:57:57.054ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 649.663008ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
15382024-12-19T00:57:57.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 818.013395ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
15392024-12-19T00:57:57.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 544.599359ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
15402024-12-19T00:57:57.267ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 865.269455ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
15412024-12-19T00:57:57.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.231427323s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
15422024-12-19T00:57:57.398ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 734.777405ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
15432024-12-19T00:57:57.524ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 1.034940934s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
15442024-12-19T00:57:57.595ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 1.065701442s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
15452024-12-19T00:57:57.712ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 508.93125ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
15462024-12-19T00:57:57.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 946.518499ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
15472024-12-19T00:57:57.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 880.711463ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
15482024-12-19T00:57:57.878ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 1.438038531s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
15492024-12-19T00:57:57.963ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 807.252984ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
15502024-12-19T00:57:57.994ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 1.472707396s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
15512024-12-19T00:57:58.007ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.403136204s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
15522024-12-19T00:57:58.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 511.611306ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
15532024-12-19T00:57:58.068ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 1.228190955s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
15542024-12-19T00:57:58.140ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 510.444432ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
15552024-12-19T00:57:58.148ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.214303507s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
15562024-12-19T00:57:58.230ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 1.456929858s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
15572024-12-19T00:57:58.556ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 1.2044931s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
15582024-12-19T00:57:58.570ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 1.48408846s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
15592024-12-19T00:57:58.579ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 659.21687ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
15602024-12-19T00:57:58.653ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 1.157684787s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
15612024-12-19T00:57:58.662ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 1.190323948s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
15622024-12-19T00:57:58.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 747.436044ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
15632024-12-19T00:57:58.687ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 837.026633ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
15642024-12-19T00:57:58.779ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 880.628482ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
15652024-12-19T00:57:59.254ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.362670044s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
15662024-12-19T00:57:59.310ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 776.075615ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
15672024-12-19T00:57:59.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 1.13616065s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
15682024-12-19T00:57:59.374ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.285509481s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
15692024-12-19T00:57:59.411ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
15702024-12-19T00:57:59.421ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 931.039931ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
15712024-12-19T00:57:59.430ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 767.049658ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
15722024-12-19T00:57:59.430ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
15732024-12-19T00:57:59.477ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 519.004401ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
15742024-12-19T00:57:59.535ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 514.896708ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
15752024-12-19T00:57:59.671ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.488301147s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
15762024-12-19T00:57:59.697ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 1.403908828s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
15772024-12-19T00:57:59.728ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_a6644750-441b-4297-9acc-8dd23a71b4cb/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='6a3779cd-b5bc-40b2-9522-41c25adb89d1'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
15782024-12-19T00:57:59.738ZINFOSledAgent (ServiceManager): Booting oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
15792024-12-19T00:57:59.770ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 583.797787ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
15802024-12-19T00:57:59.785ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
15812024-12-19T00:57:59.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 836.024182ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
15822024-12-19T00:57:59.827ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
15832024-12-19T00:57:59.862ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 1.019349882s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
15842024-12-19T00:57:59.963ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 38.463422ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
15852024-12-19T00:57:59.993ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 72.308673ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
15862024-12-19T00:58:00.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 1.27407474s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
15872024-12-19T00:58:00.012ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 138.497294ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
15882024-12-19T00:58:00.046ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 59.573946ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
15892024-12-19T00:58:00.059ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 1.277642846s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
15902024-12-19T00:58:00.067ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 700.182827ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
15912024-12-19T00:58:00.074ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 108.89651ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
15922024-12-19T00:58:00.095ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 1.137937346s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
15932024-12-19T00:58:00.114ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 137.526162ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
15942024-12-19T00:58:00.161ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 148.543458ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
15952024-12-19T00:58:00.191ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 226.954667ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
15962024-12-19T00:58:00.207ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 1.179763041s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
15972024-12-19T00:58:00.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 238.30845ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
15982024-12-19T00:58:00.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 506.517201ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
15992024-12-19T00:58:00.361ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 937.686357ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
16002024-12-19T00:58:00.369ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 853.953586ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
16012024-12-19T00:58:00.428ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 417.821969ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
16022024-12-19T00:58:00.433ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
16032024-12-19T00:58:00.433ZINFOSledAgent (ServiceManager): Booting oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
16042024-12-19T00:58:00.475ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 1.275664462s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
16052024-12-19T00:58:00.508ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 346.333555ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
16062024-12-19T00:58:00.617ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 38.512772ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
16072024-12-19T00:58:00.628ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 571.547008ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
16082024-12-19T00:58:00.666ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 125.955482ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
16092024-12-19T00:58:00.678ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 1.326152444s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
16102024-12-19T00:58:00.688ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 755.060643ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
16112024-12-19T00:58:00.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 1.169583018s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
16122024-12-19T00:58:00.918ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 156.068008ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
16132024-12-19T00:58:00.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 645.450389ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
16142024-12-19T00:58:00.968ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 1.054683729s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
16152024-12-19T00:58:00.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 955.460855ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
16162024-12-19T00:58:00.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 589.132566ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
16172024-12-19T00:58:01.086ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 372.331159ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
16182024-12-19T00:58:01.114ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 632.941538ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
16192024-12-19T00:58:01.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 506.837535ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
16202024-12-19T00:58:01.211ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.119051338s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
16212024-12-19T00:58:01.233ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 904.912133ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
16222024-12-19T00:58:01.243ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 1.012967878s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
16232024-12-19T00:58:01.288ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 913.614906ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
16242024-12-19T00:58:01.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.203989673s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
16252024-12-19T00:58:01.346ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 1.292243137s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
16262024-12-19T00:58:01.397ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 533.744455ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
16272024-12-19T00:58:01.453ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.367130126s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
16282024-12-19T00:58:01.469ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 939.56564ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
16292024-12-19T00:58:01.596ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 804.830724ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
16302024-12-19T00:58:01.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 1.236169507s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
16312024-12-19T00:58:01.689ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 956.956443ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
16322024-12-19T00:58:01.755ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 962.203778ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
16332024-12-19T00:58:01.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 585.065659ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
16342024-12-19T00:58:01.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 1.078853269s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
16352024-12-19T00:58:01.948ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 759.010519ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
16362024-12-19T00:58:01.959ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 954.975816ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
16372024-12-19T00:58:02.013ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 885.167118ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
16382024-12-19T00:58:02.031ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 1.382048126s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
16392024-12-19T00:58:02.147ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 847.767301ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
16402024-12-19T00:58:02.211ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 1.491911417s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
16412024-12-19T00:58:02.265ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 1.114352627s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
16422024-12-19T00:58:02.339ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 874.928614ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
16432024-12-19T00:58:02.357ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 926.118226ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
16442024-12-19T00:58:02.409ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 1.078833281s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
16452024-12-19T00:58:02.417ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 1.485518184s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
16462024-12-19T00:58:02.521ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 847.683714ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
16472024-12-19T00:58:02.613ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:48276
16482024-12-19T00:58:02.617ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 293 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:48276 req_id = c0adeaad-5312-4475-b271-83116582efb9 response_code = 200 uri = /baseboard
16492024-12-19T00:58:02.648ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 1.296935023s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
16502024-12-19T00:58:02.656ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 534.380559ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
16512024-12-19T00:58:02.717ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 853.917595ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
16522024-12-19T00:58:02.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 1.20667088s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
16532024-12-19T00:58:02.828ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.42389348s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
16542024-12-19T00:58:02.851ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 910.28804ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
16552024-12-19T00:58:02.908ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 1.461041176s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
16562024-12-19T00:58:02.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 865.382028ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
16572024-12-19T00:58:03.005ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 873.282902ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
16582024-12-19T00:58:03.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 1.142388965s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
16592024-12-19T00:58:03.200ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.324297689s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
16602024-12-19T00:58:03.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 686.534601ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
16612024-12-19T00:58:03.296ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 942.329006ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
16622024-12-19T00:58:03.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 621.334386ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
16632024-12-19T00:58:03.394ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 944.392428ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
16642024-12-19T00:58:03.425ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 1.126785808s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
16652024-12-19T00:58:03.502ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 1.202481969s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
16662024-12-19T00:58:03.585ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 1.161255725s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
16672024-12-19T00:58:03.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 1.480913269s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
16682024-12-19T00:58:03.778ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 613.602301ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
16692024-12-19T00:58:03.802ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 1.350536268s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
16702024-12-19T00:58:03.891ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 1.043333716s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
16712024-12-19T00:58:03.915ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 1.442041939s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
16722024-12-19T00:58:03.926ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.044631216s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
16732024-12-19T00:58:03.944ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 721.970448ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
16742024-12-19T00:58:03.958ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 937.519442ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
16752024-12-19T00:58:04.016ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.302368674s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
16762024-12-19T00:58:04.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 1.058420692s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
16772024-12-19T00:58:04.252ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 547.820042ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
16782024-12-19T00:58:04.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.263588018s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
16792024-12-19T00:58:04.349ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 915.273593ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
16802024-12-19T00:58:04.379ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 851.177216ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
16812024-12-19T00:58:04.400ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 1.115565015s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
16822024-12-19T00:58:04.535ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 606.598734ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
16832024-12-19T00:58:04.562ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 1.4046322s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
16842024-12-19T00:58:04.680ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 503.257773ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
16852024-12-19T00:58:04.724ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 772.135829ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
16862024-12-19T00:58:04.762ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 883.534065ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
16872024-12-19T00:58:04.815ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 1.351160402s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
16882024-12-19T00:58:04.909ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 1.468097772s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
16892024-12-19T00:58:04.949ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 822.238322ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
16902024-12-19T00:58:04.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 970.705762ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
16912024-12-19T00:58:05.156ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.346596925s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
16922024-12-19T00:58:05.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 710.775278ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
16932024-12-19T00:58:05.196ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 883.263947ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
16942024-12-19T00:58:05.213ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 1.069711371s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
16952024-12-19T00:58:05.244ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 613.073027ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
16962024-12-19T00:58:05.257ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 1.177659206s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
16972024-12-19T00:58:05.279ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 1.470545266s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
16982024-12-19T00:58:05.332ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.325654314s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
16992024-12-19T00:58:05.358ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 3.071774548s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
17002024-12-19T00:58:05.369ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 1.333456023s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
17012024-12-19T00:58:05.510ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 695.228325ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
17022024-12-19T00:58:05.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 1.067360593s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
17032024-12-19T00:58:05.547ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 521.935281ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
17042024-12-19T00:58:05.664ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 871.158613ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
17052024-12-19T00:58:05.787ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 778.074511ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
17062024-12-19T00:58:06.011ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 1.009837184s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
17072024-12-19T00:58:06.084ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 1.292011179s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
17082024-12-19T00:58:06.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.086885585s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
17092024-12-19T00:58:06.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 1.049780939s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
17102024-12-19T00:58:06.115ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 1.021507741s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
17112024-12-19T00:58:06.125ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.174619066s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
17122024-12-19T00:58:06.153ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin-single"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> <propval type="astring" name="ch_address" value='[fd00:1122:3344:101::e]:9000'/> <propval type="astring" name="ch_binary" value='/opt/oxide/clickhouse/clickhouse'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
17132024-12-19T00:58:06.153ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
17142024-12-19T00:58:06.181ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 676.825114ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
17152024-12-19T00:58:06.219ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 1.049999052s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
17162024-12-19T00:58:06.297ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 924.418739ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
17172024-12-19T00:58:06.390ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 942.24632ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
17182024-12-19T00:58:06.447ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 959.181838ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
17192024-12-19T00:58:06.514ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 729.471263ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
17202024-12-19T00:58:06.524ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 69.915445ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
17212024-12-19T00:58:06.544ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 893.817253ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
17222024-12-19T00:58:06.573ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 775.167417ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
17232024-12-19T00:58:06.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 127.357774ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
17242024-12-19T00:58:06.616ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 1.273925435s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
17252024-12-19T00:58:06.668ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.306118714s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
17262024-12-19T00:58:06.714ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 1.246923042s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
17272024-12-19T00:58:06.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 162.45869ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
17282024-12-19T00:58:06.759ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 767.951929ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
17292024-12-19T00:58:06.868ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 583.002629ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
17302024-12-19T00:58:06.912ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 527.730047ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
17312024-12-19T00:58:07.032ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 1.240759949s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
17322024-12-19T00:58:07.151ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 1.175128329s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
17332024-12-19T00:58:07.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 760.722565ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
17342024-12-19T00:58:07.193ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.124970987s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
17352024-12-19T00:58:07.241ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 1.0178544s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
17362024-12-19T00:58:07.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 523.548107ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
17372024-12-19T00:58:07.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 899.361555ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
17382024-12-19T00:58:07.314ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 838.72888ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
17392024-12-19T00:58:07.346ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 544.991877ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
17402024-12-19T00:58:07.359ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 730.304618ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
17412024-12-19T00:58:07.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 853.666216ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
17422024-12-19T00:58:07.419ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 922.055682ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
17432024-12-19T00:58:07.457ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 608.472756ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
17442024-12-19T00:58:07.473ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 1.113362147s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
17452024-12-19T00:58:07.485ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 1.056516669s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
17462024-12-19T00:58:07.540ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 553.182174ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
17472024-12-19T00:58:07.796ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 599.96782ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
17482024-12-19T00:58:07.907ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 577.006639ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
17492024-12-19T00:58:07.924ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 1.124974287s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
17502024-12-19T00:58:07.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 1.006763092s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
17512024-12-19T00:58:07.979ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 1.469554711s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
17522024-12-19T00:58:07.996ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.248546656s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
17532024-12-19T00:58:08.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 831.486867ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
17542024-12-19T00:58:08.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 1.104100428s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
17552024-12-19T00:58:08.120ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 1.182273839s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
17562024-12-19T00:58:08.164ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 670.677161ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
17572024-12-19T00:58:08.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 1.46072046s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
17582024-12-19T00:58:08.259ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 992.978364ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
17592024-12-19T00:58:08.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 1.199895115s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
17602024-12-19T00:58:08.292ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 836.245456ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
17612024-12-19T00:58:08.330ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.182684021s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
17622024-12-19T00:58:08.345ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 639.498524ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
17632024-12-19T00:58:08.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 772.103739ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
17642024-12-19T00:58:08.407ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.011428103s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
17652024-12-19T00:58:08.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 1.471424341s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
17662024-12-19T00:58:08.559ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 751.445737ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
17672024-12-19T00:58:08.603ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 543.268293ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
17682024-12-19T00:58:08.849ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 635.744007ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
17692024-12-19T00:58:08.922ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 1.400070234s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
17702024-12-19T00:58:08.967ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 1.174060906s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
17712024-12-19T00:58:09.001ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 1.124370358s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
17722024-12-19T00:58:09.069ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 748.436704ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
17732024-12-19T00:58:09.144ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 836.323655ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
17742024-12-19T00:58:09.159ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 1.259366103s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
17752024-12-19T00:58:09.173ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 1.243926242s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
17762024-12-19T00:58:09.221ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 1.046692703s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
17772024-12-19T00:58:09.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.200669668s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
17782024-12-19T00:58:09.281ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 558.140974ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
17792024-12-19T00:58:09.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 753.562345ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
17802024-12-19T00:58:09.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 1.115160253s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
17812024-12-19T00:58:09.438ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.140773548s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
17822024-12-19T00:58:09.463ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 1.388679862s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
17832024-12-19T00:58:09.489ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 515.699288ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
17842024-12-19T00:58:09.503ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 1.04145976s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
17852024-12-19T00:58:09.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 850.323208ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
17862024-12-19T00:58:09.668ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 565.853194ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
17872024-12-19T00:58:09.830ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0") failed: Property not found. retry in 1.411733365s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
17882024-12-19T00:58:09.852ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 702.532133ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
17892024-12-19T00:58:09.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 1.187940798s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
17902024-12-19T00:58:09.993ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 631.242052ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
17912024-12-19T00:58:10.018ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 650.348541ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
17922024-12-19T00:58:10.082ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 948.345377ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
17932024-12-19T00:58:10.137ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 725.142182ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
17942024-12-19T00:58:10.154ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 1.460679234s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
17952024-12-19T00:58:10.248ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 745.697426ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
17962024-12-19T00:58:10.278ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 1.291169546s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
17972024-12-19T00:58:10.334ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 1.303828046s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
17982024-12-19T00:58:10.390ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf") failed: Property not found. retry in 1.167779811s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
17992024-12-19T00:58:10.428ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 500.70639ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
18002024-12-19T00:58:10.437ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c") failed: Property not found. retry in 1.461072622s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
18012024-12-19T00:58:10.454ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 982.719004ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
18022024-12-19T00:58:10.477ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 1.338416707s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
18032024-12-19T00:58:10.554ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439") failed: Property not found. retry in 992.151493ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
18042024-12-19T00:58:10.566ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09") failed: Property not found. retry in 795.841216ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
18052024-12-19T00:58:10.588ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa") failed: Property not found. retry in 1.349954043s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
18062024-12-19T00:58:10.633ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 681.772219ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
18072024-12-19T00:58:10.678ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad") failed: Property not found. retry in 674.386946ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
18082024-12-19T00:58:10.862ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 954.616036ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
18092024-12-19T00:58:10.870ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3") failed: Property not found. retry in 796.804591ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
18102024-12-19T00:58:10.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 1.426034685s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
18112024-12-19T00:58:11.005ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb") failed: Property not found. retry in 1.281820605s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
18122024-12-19T00:58:11.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b") failed: Property not found. retry in 673.703694ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
18132024-12-19T00:58:11.184ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc") failed: Property not found. retry in 1.208384361s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
18142024-12-19T00:58:11.328ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc") failed: Property not found. retry in 819.014065ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
18152024-12-19T00:58:11.451ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317") failed: Property not found. retry in 520.9537ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
18162024-12-19T00:58:11.597ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67") failed: Property not found. retry in 1.065948837s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
18172024-12-19T00:58:11.627ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43") failed: Property not found. retry in 527.613843ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
18182024-12-19T00:58:11.653ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9") failed: Property not found. retry in 1.130804238s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
18192024-12-19T00:58:11.833ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5") failed: Property not found. retry in 602.164495ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
18202024-12-19T00:58:11.848ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995") failed: Property not found. retry in 1.223559577s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
18212024-12-19T00:58:12.384ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 872.746319ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
18222024-12-19T00:58:13.272ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 1.159075134s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
18232024-12-19T00:58:14.453ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 733.107035ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
18242024-12-19T00:58:15.208ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 1.340655554s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
18252024-12-19T00:58:16.377ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 7.239113353s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
18262024-12-19T00:58:16.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b") failed: Property not found. retry in 1.215773701s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
18272024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_pantry_6892dc98-d04b-4b85-b4d0-03bd53995e09
18282024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_2a938d23-12fe-4a3d-bd93-2851f8d9be67
18292024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_8965b93d-b61b-4461-83fe-8af7a539eefa
18302024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_352791c3-3ea8-4a2c-abed-df833c2e15f5
18312024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_3aaec92f-d06e-48c2-961f-d8ab46b39dfc
18322024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_9c73f9dc-da7d-470f-8c17-891618bbbaad
18332024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_nexus_1176f008-4a9d-4835-a5aa-1a86ec3c619c
18342024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_pantry_68e16a8a-5014-4120-9251-64b4b355b6b0
18352024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_4b6f7643-a0fa-4766-8901-1e1892739acf
18362024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_external_dns_679d67b5-9925-49ac-b78d-613e816394dc
18372024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_oximeter_b4e7d6e3-7551-4da8-8529-1286ef6ee8a3
18382024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_clickhouse_10bb6caa-5666-4747-9b6c-48ed7069541b
18392024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_pantry_089dd97c-a5d4-48df-b3ea-a61a0b0c9f43
18402024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_nexus_e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9
18412024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_3bc6a5d7-2061-434b-96ed-851ac7abddbb
18422024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_0c9fcc8e-cb8d-4c52-8025-4137b7053c7b
18432024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_bd71449b-9ea4-4e76-b752-1d23c910c439
18442024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_nexus_1d1e5635-dd2b-4bf7-8ace-6cb3ea634995
18452024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_external_dns_2a880a2e-2471-4fd8-9ad5-3e0c3fbde317
18462024-12-19T00:58:17.803ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
18472024-12-19T00:58:17.809ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
18482024-12-19T00:58:17.811ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45207733 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:54104 req_id = fadc1538-4911-481f-b0d6-0465d5f3286e response_code = 204 uri = /omicron-zones
18492024-12-19T00:58:17.816ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1435
18502024-12-19T00:58:17.817ZINFOSledAgent (RSS): Writing ledger to /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
18512024-12-19T00:58:17.817ZINFOSledAgent (RSS): Writing ledger to /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
18522024-12-19T00:58:17.821ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:774
18532024-12-19T00:58:17.822ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::c]:12221 file = sled-agent/src/rack_setup/service.rs:786
18542024-12-19T00:58:17.857ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, rib_priority: None, vlan_id: None, }, ], switch: Switch0, tx_eq: None, uplink_port_fec: Some( None, ), uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:1000
18552024-12-19T00:58:17.858ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/6f1cca69-e82a-429e-8f71-b2c5888a42c1/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
18562024-12-19T00:58:17.911ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/6f1cca69-e82a-429e-8f71-b2c5888a42c1/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
18572024-12-19T00:58:18.050ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/6f1cca69-e82a-429e-8f71-b2c5888a42c1/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
18582024-12-19T00:58:18.182ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/6f1cca69-e82a-429e-8f71-b2c5888a42c1/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
18592024-12-19T00:58:18.286ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/6f1cca69-e82a-429e-8f71-b2c5888a42c1/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
18602024-12-19T00:58:18.510ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/6f1cca69-e82a-429e-8f71-b2c5888a42c1/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
18612024-12-19T00:58:18.858ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/6f1cca69-e82a-429e-8f71-b2c5888a42c1/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
18622024-12-19T00:58:19.614ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 403 Forbidden; headers: {"content-type": "application/json", "x-request-id": "02644481-546a-428c-9055-4458a6d339d7", "content-length": "113", "date": "Thu, 19 Dec 2024 00:58:19 GMT"}; value: Error { error_code: Some("Forbidden"), message: "Forbidden", request_id: "02644481-546a-428c-9055-4458a6d339d7" } file = sled-agent/src/rack_setup/service.rs:1062
18632024-12-19T00:58:20.139ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "bda8af99-d6a7-44f7-a90d-aca2bc385843", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:19 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"6f1cca69-e82a-429e-8f71-b2c5888a42c1\"", request_id: "bda8af99-d6a7-44f7-a90d-aca2bc385843" } file = sled-agent/src/rack_setup/service.rs:1062
18642024-12-19T00:58:20.641ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "f30dad15-c604-40dd-9892-c648b43e3d91", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:20 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"6f1cca69-e82a-429e-8f71-b2c5888a42c1\"", request_id: "f30dad15-c604-40dd-9892-c648b43e3d91" } file = sled-agent/src/rack_setup/service.rs:1062
18652024-12-19T00:58:21.142ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "40cf33aa-ea9a-4869-8b04-397f3d1f24ec", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:20 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"6f1cca69-e82a-429e-8f71-b2c5888a42c1\"", request_id: "40cf33aa-ea9a-4869-8b04-397f3d1f24ec" } file = sled-agent/src/rack_setup/service.rs:1062
18662024-12-19T00:58:22.447ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "f1d8c186-c63b-4813-8c48-89ae30818f21", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:21 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"de9bec7d-f8ce-44a0-879c-5e088768f91c\"", request_id: "f1d8c186-c63b-4813-8c48-89ae30818f21" } file = sled-agent/src/rack_setup/service.rs:1062
18672024-12-19T00:58:24.013ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "f1312a76-5468-41ac-a434-7ecb89e2776d", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:23 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"de9bec7d-f8ce-44a0-879c-5e088768f91c\"", request_id: "f1312a76-5468-41ac-a434-7ecb89e2776d" } file = sled-agent/src/rack_setup/service.rs:1062
18682024-12-19T00:58:25.243ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "a8b6ac37-5848-4bee-805c-8956c71a6f8c", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:24 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"de9bec7d-f8ce-44a0-879c-5e088768f91c\"", request_id: "a8b6ac37-5848-4bee-805c-8956c71a6f8c" } file = sled-agent/src/rack_setup/service.rs:1062
18692024-12-19T00:58:26.913ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "eb39ebd2-cfc0-4dde-9bc6-a3edce4df172", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:26 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"de9bec7d-f8ce-44a0-879c-5e088768f91c\"", request_id: "eb39ebd2-cfc0-4dde-9bc6-a3edce4df172" } file = sled-agent/src/rack_setup/service.rs:1062
18702024-12-19T00:58:28.493ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "a7258850-f64e-44f4-864f-d5bcef80d612", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:28 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"de9bec7d-f8ce-44a0-879c-5e088768f91c\"", request_id: "a7258850-f64e-44f4-864f-d5bcef80d612" } file = sled-agent/src/rack_setup/service.rs:1062
18712024-12-19T00:58:30.627ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "64e1a89c-4e6b-487c-948f-27152c4fb7a3", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:30 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"de9bec7d-f8ce-44a0-879c-5e088768f91c\"", request_id: "64e1a89c-4e6b-487c-948f-27152c4fb7a3" } file = sled-agent/src/rack_setup/service.rs:1062
18722024-12-19T00:58:32.652ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:50685
18732024-12-19T00:58:32.652ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:50685 req_id = 64f15201-8808-49e6-8d1e-85dafaafe9d6 response_code = 200 uri = /baseboard
18742024-12-19T00:58:33.460ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "0e347dea-b8e1-4b90-a005-5636571113af", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:33 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"de9bec7d-f8ce-44a0-879c-5e088768f91c\"", request_id: "0e347dea-b8e1-4b90-a005-5636571113af" } file = sled-agent/src/rack_setup/service.rs:1062
18752024-12-19T00:58:36.026ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:37755 remote_addr = [fd00:1122:3344:101::d]:64395 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
18762024-12-19T00:58:36.047ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 20966 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = 3b897ccb-2bde-49e4-962b-35ae23ba2c48 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
18772024-12-19T00:58:36.477ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "0e38d2af-dc2d-4ec2-9280-af7879e268dc", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:36 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"de9bec7d-f8ce-44a0-879c-5e088768f91c\"", request_id: "0e38d2af-dc2d-4ec2-9280-af7879e268dc" } file = sled-agent/src/rack_setup/service.rs:1062
18782024-12-19T00:58:39.859ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "8396fb30-3557-4e92-be03-ce0a07cd4341", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:39 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"de9bec7d-f8ce-44a0-879c-5e088768f91c\"", request_id: "8396fb30-3557-4e92-be03-ce0a07cd4341" } file = sled-agent/src/rack_setup/service.rs:1062
18792024-12-19T00:58:43.299ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "6324bd39-06d0-485e-8c39-bd39b4f574dd", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:43 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"de9bec7d-f8ce-44a0-879c-5e088768f91c\"", request_id: "6324bd39-06d0-485e-8c39-bd39b4f574dd" } file = sled-agent/src/rack_setup/service.rs:1062
18802024-12-19T00:58:48.989ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "726b07bc-568b-4489-84f6-4b4c4fa6eb27", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:48 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"de9bec7d-f8ce-44a0-879c-5e088768f91c\"", request_id: "726b07bc-568b-4489-84f6-4b4c4fa6eb27" } file = sled-agent/src/rack_setup/service.rs:1062
18812024-12-19T00:58:54.415ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "3eae5e4b-c114-44c9-964c-9fedb509bce0", "content-length": "173", "date": "Thu, 19 Dec 2024 00:58:54 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"de9bec7d-f8ce-44a0-879c-5e088768f91c\"", request_id: "3eae5e4b-c114-44c9-964c-9fedb509bce0" } file = sled-agent/src/rack_setup/service.rs:1062
18822024-12-19T00:58:58.585ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:500 new_generation = 1 old_generation = 1 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
18832024-12-19T00:58:58.677ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56845
18842024-12-19T00:58:58.678ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
18852024-12-19T00:58:58.678ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
18862024-12-19T00:58:58.678ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
18872024-12-19T00:58:58.679ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
18882024-12-19T00:58:58.691ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18892024-12-19T00:58:58.691ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a6644750-441b-4297-9acc-8dd23a71b4cb
18902024-12-19T00:58:58.716ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18912024-12-19T00:58:58.716ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa
18922024-12-19T00:58:58.738ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18932024-12-19T00:58:58.738ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_93eb8a71-6519-482e-a366-b15e30b4605e
18942024-12-19T00:58:58.761ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18952024-12-19T00:58:58.761ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_98a100ce-cd82-4209-8530-7b48ca829744
18962024-12-19T00:58:58.781ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18972024-12-19T00:58:58.781ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6
18982024-12-19T00:58:58.801ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18992024-12-19T00:58:58.801ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb
19002024-12-19T00:58:58.822ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
19012024-12-19T00:58:58.822ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_f4009784-7bde-44ff-854e-49d614e9e682
19022024-12-19T00:58:58.841ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
19032024-12-19T00:58:58.841ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_1025d0d6-c165-4f43-bef0-cb2eeb04b8c2
19042024-12-19T00:58:58.861ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
19052024-12-19T00:58:58.861ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_499e53d3-d67c-44b6-9c03-b66767baed7e
19062024-12-19T00:58:58.872ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 195439 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56845 req_id = 47bc1d18-b92b-46d7-a3fd-17373a04b5c0 response_code = 200 uri = /inventory
19072024-12-19T00:59:02.686ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:57364
19082024-12-19T00:59:02.687ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:57364 req_id = edfd3d70-e184-44d6-94b9-881d4ebd7cd5 response_code = 200 uri = /baseboard
19092024-12-19T00:59:02.956ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53514
19102024-12-19T00:59:02.958ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 5b97cf45-6db1-45ed-9f7e-111d132f7ed4, kind: Service { id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 }, name: Name("external-dns-2a880a2e-2471-4fd8-9ad5-3e0c3fbde317"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 253])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d21d0095-8e84-486f-b3a4-f9dca167b9d5, kind: Service { id: 679d67b5-9925-49ac-b78d-613e816394dc }, name: Name("external-dns-679d67b5-9925-49ac-b78d-613e816394dc"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 205])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 82094354-655d-415c-9763-28731eb50642, kind: Service { id: e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 }, name: Name("nexus-e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 60])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5c3ad41-155b-469b-bb3f-412814b9d0ff, kind: Service { id: 1176f008-4a9d-4835-a5aa-1a86ec3c619c }, name: Name("nexus-1176f008-4a9d-4835-a5aa-1a86ec3c619c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 188])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ef89a8b8-3cc5-4637-aee2-3b98f04f51be, kind: Service { id: 1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 }, name: Name("nexus-1d1e5635-dd2b-4bf7-8ace-6cb3ea634995"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 75])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19112024-12-19T00:59:02.958ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19122024-12-19T00:59:02.959ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19132024-12-19T00:59:02.959ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19142024-12-19T00:59:02.959ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
19152024-12-19T00:59:02.959ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19162024-12-19T00:59:02.959ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19172024-12-19T00:59:02.959ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2391 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53514 req_id = a0513c77-5a5c-437d-9799-8497bfdfc5b5 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19182024-12-19T00:59:02.960ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:1072
19192024-12-19T00:59:03.307ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42135
19202024-12-19T00:59:03.307ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
19212024-12-19T00:59:03.307ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
19222024-12-19T00:59:03.307ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
19232024-12-19T00:59:03.308ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
19242024-12-19T00:59:03.320ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
19252024-12-19T00:59:03.320ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a6644750-441b-4297-9acc-8dd23a71b4cb
19262024-12-19T00:59:03.344ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
19272024-12-19T00:59:03.344ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa
19282024-12-19T00:59:03.365ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
19292024-12-19T00:59:03.365ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_93eb8a71-6519-482e-a366-b15e30b4605e
19302024-12-19T00:59:03.387ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
19312024-12-19T00:59:03.387ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_98a100ce-cd82-4209-8530-7b48ca829744
19322024-12-19T00:59:03.409ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
19332024-12-19T00:59:03.409ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6
19342024-12-19T00:59:03.429ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
19352024-12-19T00:59:03.429ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb
19362024-12-19T00:59:03.449ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
19372024-12-19T00:59:03.449ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_f4009784-7bde-44ff-854e-49d614e9e682
19382024-12-19T00:59:03.469ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
19392024-12-19T00:59:03.469ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_1025d0d6-c165-4f43-bef0-cb2eeb04b8c2
19402024-12-19T00:59:03.489ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
19412024-12-19T00:59:03.489ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_499e53d3-d67c-44b6-9c03-b66767baed7e
19422024-12-19T00:59:03.500ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 192246 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42135 req_id = 644e3194-930e-41e1-a5f1-31235bb3e2d0 response_code = 200 uri = /inventory
19432024-12-19T00:59:03.658ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35966
19442024-12-19T00:59:03.658ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 5b97cf45-6db1-45ed-9f7e-111d132f7ed4, kind: Service { id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 }, name: Name("external-dns-2a880a2e-2471-4fd8-9ad5-3e0c3fbde317"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 253])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d21d0095-8e84-486f-b3a4-f9dca167b9d5, kind: Service { id: 679d67b5-9925-49ac-b78d-613e816394dc }, name: Name("external-dns-679d67b5-9925-49ac-b78d-613e816394dc"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 205])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 82094354-655d-415c-9763-28731eb50642, kind: Service { id: e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 }, name: Name("nexus-e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 60])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5c3ad41-155b-469b-bb3f-412814b9d0ff, kind: Service { id: 1176f008-4a9d-4835-a5aa-1a86ec3c619c }, name: Name("nexus-1176f008-4a9d-4835-a5aa-1a86ec3c619c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 188])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ef89a8b8-3cc5-4637-aee2-3b98f04f51be, kind: Service { id: 1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 }, name: Name("nexus-1d1e5635-dd2b-4bf7-8ace-6cb3ea634995"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 75])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19452024-12-19T00:59:03.658ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19462024-12-19T00:59:03.658ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19472024-12-19T00:59:03.658ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19482024-12-19T00:59:03.658ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
19492024-12-19T00:59:03.658ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19502024-12-19T00:59:03.659ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19512024-12-19T00:59:03.659ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 444 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35966 req_id = b44eeb15-d92f-4a49-8b1a-55ae69563395 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19522024-12-19T00:59:04.550ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52726
19532024-12-19T00:59:04.550ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 5b97cf45-6db1-45ed-9f7e-111d132f7ed4, kind: Service { id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 }, name: Name("external-dns-2a880a2e-2471-4fd8-9ad5-3e0c3fbde317"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 253])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d21d0095-8e84-486f-b3a4-f9dca167b9d5, kind: Service { id: 679d67b5-9925-49ac-b78d-613e816394dc }, name: Name("external-dns-679d67b5-9925-49ac-b78d-613e816394dc"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 205])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 82094354-655d-415c-9763-28731eb50642, kind: Service { id: e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 }, name: Name("nexus-e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 60])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5c3ad41-155b-469b-bb3f-412814b9d0ff, kind: Service { id: 1176f008-4a9d-4835-a5aa-1a86ec3c619c }, name: Name("nexus-1176f008-4a9d-4835-a5aa-1a86ec3c619c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 188])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ef89a8b8-3cc5-4637-aee2-3b98f04f51be, kind: Service { id: 1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 }, name: Name("nexus-1d1e5635-dd2b-4bf7-8ace-6cb3ea634995"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 75])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19542024-12-19T00:59:04.550ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19552024-12-19T00:59:04.550ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19562024-12-19T00:59:04.550ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19572024-12-19T00:59:04.550ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
19582024-12-19T00:59:04.550ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19592024-12-19T00:59:04.550ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19602024-12-19T00:59:04.550ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 330 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52726 req_id = 35676eee-9c78-4123-bc62-611f27c63a6b response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19612024-12-19T00:59:04.828ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57140
19622024-12-19T00:59:04.828ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 5b97cf45-6db1-45ed-9f7e-111d132f7ed4, kind: Service { id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 }, name: Name("external-dns-2a880a2e-2471-4fd8-9ad5-3e0c3fbde317"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 253])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d21d0095-8e84-486f-b3a4-f9dca167b9d5, kind: Service { id: 679d67b5-9925-49ac-b78d-613e816394dc }, name: Name("external-dns-679d67b5-9925-49ac-b78d-613e816394dc"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 205])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 82094354-655d-415c-9763-28731eb50642, kind: Service { id: e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 }, name: Name("nexus-e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 60])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5c3ad41-155b-469b-bb3f-412814b9d0ff, kind: Service { id: 1176f008-4a9d-4835-a5aa-1a86ec3c619c }, name: Name("nexus-1176f008-4a9d-4835-a5aa-1a86ec3c619c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 188])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ef89a8b8-3cc5-4637-aee2-3b98f04f51be, kind: Service { id: 1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 }, name: Name("nexus-1d1e5635-dd2b-4bf7-8ace-6cb3ea634995"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 75])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19632024-12-19T00:59:04.828ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19642024-12-19T00:59:04.828ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19652024-12-19T00:59:04.828ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19662024-12-19T00:59:04.828ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
19672024-12-19T00:59:04.828ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19682024-12-19T00:59:04.828ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19692024-12-19T00:59:04.828ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 337 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57140 req_id = 96e45748-75c3-4385-adee-c53a284f742e response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19702024-12-19T00:59:05.765ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7041 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = ff8b250e-30d6-4f85-9a52-aa9cd671e497 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
19712024-12-19T00:59:08.586ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47899
19722024-12-19T00:59:08.586ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47899 req_id = 67106176-0345-4158-908f-0b44a84b9292 response_code = 200 uri = /vpc-routes
19732024-12-19T00:59:08.717ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
19742024-12-19T00:59:08.717ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 880 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47899 req_id = eda95d3d-6638-4eaf-b50f-40510b10df71 response_code = 204 uri = /eip-gateways
19752024-12-19T00:59:08.812ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2984 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47899 req_id = de76db35-4356-4f31-a6e4-e488bbef1a8e response_code = 204 uri = /vpc-routes
19762024-12-19T00:59:21.737ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50410
19772024-12-19T00:59:21.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50410 req_id = 2bc75dbf-cfce-4668-916d-e45a8061e8e2 response_code = 200 uri = /v2p
19782024-12-19T00:59:21.789ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46804
19792024-12-19T00:59:21.789ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46804 req_id = 01cac1ee-fdc9-48a9-bd47-079ac89449bb response_code = 200 uri = /vpc-routes
19802024-12-19T00:59:21.812ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37768
19812024-12-19T00:59:21.812ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37768 req_id = cf622b03-8c78-4ae6-9ec1-db5ac2129005 response_code = 200 uri = /v2p
19822024-12-19T00:59:21.838ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34264
19832024-12-19T00:59:21.838ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 22 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34264 req_id = 5138865d-7f55-49e5-a0b9-208dc4517aea response_code = 200 uri = /vpc-routes
19842024-12-19T00:59:21.858ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39874
19852024-12-19T00:59:21.859ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39874 req_id = f7f00a99-d0b4-4b88-841f-9f10701513d6 response_code = 200 uri = /vpc-routes
19862024-12-19T00:59:21.905ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
19872024-12-19T00:59:21.905ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34264 req_id = 7e042837-7739-4f25-a630-401f004911ea response_code = 204 uri = /eip-gateways
19882024-12-19T00:59:21.911ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
19892024-12-19T00:59:21.911ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 32 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46804 req_id = d8244b36-b118-49c1-948a-8df32f80dc76 response_code = 204 uri = /eip-gateways
19902024-12-19T00:59:21.915ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
19912024-12-19T00:59:21.915ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39874 req_id = 3710bd10-c9a5-492f-9c03-b3e8956f6d17 response_code = 204 uri = /eip-gateways
19922024-12-19T00:59:21.916ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34264 req_id = edd635ef-4126-4c5d-9be9-92f31349d641 response_code = 204 uri = /vpc-routes
19932024-12-19T00:59:21.921ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46804 req_id = ac6aacbe-9e47-4d12-a431-852b7302f027 response_code = 204 uri = /vpc-routes
19942024-12-19T00:59:21.925ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39874 req_id = 026b377b-76b1-4e44-90a4-0e2f1f563cc6 response_code = 204 uri = /vpc-routes
19952024-12-19T00:59:22.077ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33168
19962024-12-19T00:59:22.077ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
19972024-12-19T00:59:22.135ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
19982024-12-19T00:59:22.193ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115758 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:33168 req_id = 7f206ca1-9fa7-42f0-870b-5fca9386e2af response_code = 204 uri = /switch-ports
19992024-12-19T00:59:22.193ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37666
20002024-12-19T00:59:22.193ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51284
20012024-12-19T00:59:22.193ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20022024-12-19T00:59:22.197ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33168 req_id = 7b8a9762-f1b6-4f43-bbc6-1e990cd0a856 response_code = 200 uri = /network-bootstore-config
20032024-12-19T00:59:22.198ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 670 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37666 req_id = 9bc05eda-7c11-4aed-b70b-dd29b89c0d8b response_code = 200 uri = /v2p
20042024-12-19T00:59:22.240ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
20052024-12-19T00:59:22.240ZINFOSledAgent (bootstore): Writing ledger to /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
20062024-12-19T00:59:22.241ZINFOSledAgent (bootstore): Writing ledger to /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
20072024-12-19T00:59:22.243ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
20082024-12-19T00:59:22.243ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3013 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33168 req_id = b75f6878-6971-496c-aa75-f365cc1b2b6a response_code = 204 uri = /network-bootstore-config
20092024-12-19T00:59:22.243ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20102024-12-19T00:59:22.269ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37208
20112024-12-19T00:59:22.305ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20122024-12-19T00:59:22.305ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111595 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:51284 req_id = 7bdf6597-cad4-4776-85f6-bb78fe6b52bd response_code = 204 uri = /switch-ports
20132024-12-19T00:59:22.309ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 786 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51284 req_id = 54f756e9-87ac-49d5-bb3b-68e1ca4aad8f response_code = 200 uri = /network-bootstore-config
20142024-12-19T00:59:22.341ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20152024-12-19T00:59:22.402ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 132564 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37208 req_id = 90e13e20-cc11-43be-8f4a-32641823e24a response_code = 204 uri = /switch-ports
20162024-12-19T00:59:22.402ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37208 req_id = e116e18d-b277-4951-9db8-14934905d169 response_code = 200 uri = /network-bootstore-config
20172024-12-19T00:59:32.721ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:64745
20182024-12-19T00:59:32.721ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 25 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:64745 req_id = ba6e874d-f1ef-4d59-9bc1-9a8388d24ae7 response_code = 200 uri = /baseboard
20192024-12-19T00:59:35.766ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7595 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = 73a6a030-4b64-431a-9f9f-2a340c0611f8 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
20202024-12-19T00:59:51.626ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56094
20212024-12-19T00:59:51.626ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56094 req_id = 2523ddcc-d364-4a6f-b7f6-a740014af69d response_code = 200 uri = /vpc-routes
20222024-12-19T00:59:51.650ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46410
20232024-12-19T00:59:51.650ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46410 req_id = 7a3eaa48-9f1a-49c7-9ba2-54dd9d0c0656 response_code = 200 uri = /vpc-routes
20242024-12-19T00:59:51.674ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
20252024-12-19T00:59:51.674ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46410 req_id = daf5557b-e803-449c-858b-ed5a475cb620 response_code = 204 uri = /eip-gateways
20262024-12-19T00:59:51.678ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
20272024-12-19T00:59:51.678ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56094 req_id = dc3be174-c5ab-4464-89d3-199d55285e9e response_code = 204 uri = /eip-gateways
20282024-12-19T00:59:51.688ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 260 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46410 req_id = 8d3b4708-d253-4926-a039-fadaf954b5bb response_code = 204 uri = /vpc-routes
20292024-12-19T00:59:51.689ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56094 req_id = 601737b6-321f-425c-afe6-7acba6874ad6 response_code = 204 uri = /vpc-routes
20302024-12-19T00:59:51.752ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62500
20312024-12-19T00:59:51.752ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62500 req_id = 55caa0d6-74f9-4874-a948-9d90f7fa5cc5 response_code = 200 uri = /v2p
20322024-12-19T00:59:51.784ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48745
20332024-12-19T00:59:51.784ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48745 req_id = 78c93acf-9afe-490f-95af-a8c24716e423 response_code = 200 uri = /v2p
20342024-12-19T00:59:51.786ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62257
20352024-12-19T00:59:51.786ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20362024-12-19T00:59:51.823ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20372024-12-19T00:59:51.879ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92769 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:62257 req_id = b22374fb-f94d-431f-9dbf-d304d3d3e532 response_code = 204 uri = /switch-ports
20382024-12-19T00:59:51.879ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54484
20392024-12-19T00:59:51.879ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 263 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62257 req_id = 7200dd0d-8eb6-4d16-a7f1-5d3a11a52b30 response_code = 200 uri = /network-bootstore-config
20402024-12-19T00:59:51.879ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20412024-12-19T00:59:51.918ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20422024-12-19T00:59:51.923ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64454
20432024-12-19T00:59:51.923ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64454 req_id = 4e06c531-2758-479c-9e44-feb378b0039a response_code = 200 uri = /vpc-routes
20442024-12-19T00:59:51.980ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100877 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:54484 req_id = 7e0a5aed-d2d4-4e44-b868-aba073b25a19 response_code = 204 uri = /switch-ports
20452024-12-19T00:59:51.981ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54484 req_id = b4f41aea-82e1-4e5f-b138-fb1eb6225198 response_code = 200 uri = /network-bootstore-config
20462024-12-19T00:59:51.993ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53652
20472024-12-19T00:59:51.994ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53652 req_id = a5c79296-4c63-405a-9f60-9fc88e87e85b response_code = 200 uri = /v2p
20482024-12-19T00:59:52.076ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
20492024-12-19T00:59:52.076ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64454 req_id = 95989866-293c-4e96-a317-dbbd4c71476f response_code = 204 uri = /eip-gateways
20502024-12-19T00:59:52.085ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64454 req_id = 1d766008-b250-4c27-bfec-18e09d87e457 response_code = 204 uri = /vpc-routes
20512024-12-19T00:59:52.241ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42887
20522024-12-19T00:59:52.241ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20532024-12-19T00:59:52.278ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20542024-12-19T00:59:52.329ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88498 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:42887 req_id = dbc3b4df-4130-4178-8ce2-e3082848ae1e response_code = 204 uri = /switch-ports
20552024-12-19T00:59:52.329ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42887 req_id = 3c5ee042-e7c3-44c3-a35a-51185e5c0d7a response_code = 200 uri = /network-bootstore-config
20562024-12-19T01:00:02.753ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:55942
20572024-12-19T01:00:02.754ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:55942 req_id = 42821e2d-e53d-4e5a-b9e4-dcb1753984c8 response_code = 200 uri = /baseboard
20582024-12-19T01:00:05.766ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7540 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = 1260a86f-8b89-4194-8f76-2d63406e2d02 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
20592024-12-19T01:00:21.774ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34006
20602024-12-19T01:00:21.774ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34006 req_id = ce710629-0290-413c-9728-3fb8d8d2897a response_code = 200 uri = /vpc-routes
20612024-12-19T01:00:21.817ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
20622024-12-19T01:00:21.817ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34006 req_id = 937ef43e-93d0-4ed6-9d6f-c308c8150efe response_code = 204 uri = /eip-gateways
20632024-12-19T01:00:21.822ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44860
20642024-12-19T01:00:21.822ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44860 req_id = 3836fc2b-58a1-4205-bcc0-5ef445a05fcd response_code = 200 uri = /vpc-routes
20652024-12-19T01:00:21.829ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 272 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34006 req_id = 53d64f7b-4880-40e4-adb4-5e2969afff4b response_code = 204 uri = /vpc-routes
20662024-12-19T01:00:21.842ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57612
20672024-12-19T01:00:21.842ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57612 req_id = e651e003-2183-44a6-a681-f6b7cf65cdb7 response_code = 200 uri = /v2p
20682024-12-19T01:00:21.859ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
20692024-12-19T01:00:21.859ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44860 req_id = 43650ab2-7592-4ae1-b12c-3edd0a120455 response_code = 204 uri = /eip-gateways
20702024-12-19T01:00:21.869ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45736
20712024-12-19T01:00:21.869ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45736 req_id = 9018ad01-a9b4-4457-9e06-caffccefc1f4 response_code = 200 uri = /v2p
20722024-12-19T01:00:21.889ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58664
20732024-12-19T01:00:21.889ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58664 req_id = f7542788-a731-4875-8a93-0c10fe1870ab response_code = 200 uri = /vpc-routes
20742024-12-19T01:00:21.938ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36682
20752024-12-19T01:00:21.938ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20762024-12-19T01:00:21.980ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20772024-12-19T01:00:22.040ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101705 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:36682 req_id = 15f99205-f9fe-498e-a0ef-56cfc32a1b4f response_code = 204 uri = /switch-ports
20782024-12-19T01:00:22.040ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
20792024-12-19T01:00:22.040ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 151 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58664 req_id = a5ef9375-1ec3-4e0d-84a5-15b55c7796b3 response_code = 204 uri = /eip-gateways
20802024-12-19T01:00:22.040ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47028
20812024-12-19T01:00:22.040ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53252
20822024-12-19T01:00:22.041ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 253 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36682 req_id = 1a9afc0f-90b1-4e29-bb58-a3c5b375a2c6 response_code = 200 uri = /network-bootstore-config
20832024-12-19T01:00:22.041ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 490 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44860 req_id = ef5f3b53-9682-48c2-95a9-54820458c59e response_code = 204 uri = /vpc-routes
20842024-12-19T01:00:22.041ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20852024-12-19T01:00:22.041ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47028 req_id = 6ad3ac28-a9d8-486f-b674-e64d2490b36d response_code = 200 uri = /v2p
20862024-12-19T01:00:22.085ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20872024-12-19T01:00:22.139ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98614 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:53252 req_id = dbda9aaa-293b-45ce-a53c-bd83fc999748 response_code = 204 uri = /switch-ports
20882024-12-19T01:00:22.140ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53252 req_id = cd710002-e34e-47e6-ac44-c7c82840e9a1 response_code = 200 uri = /network-bootstore-config
20892024-12-19T01:00:22.159ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 251 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58664 req_id = 878212b9-1efe-454c-9a55-1c019e899e4b response_code = 204 uri = /vpc-routes
20902024-12-19T01:00:22.159ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58427
20912024-12-19T01:00:22.159ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20922024-12-19T01:00:22.194ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20932024-12-19T01:00:22.247ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87870 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:58427 req_id = 5a18b6b4-53a7-45c0-bd0e-1abd6bd31e68 response_code = 204 uri = /switch-ports
20942024-12-19T01:00:22.247ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 187 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58427 req_id = d67e19a1-73a9-4d9d-98f5-1781c5b9fe56 response_code = 200 uri = /network-bootstore-config
20952024-12-19T01:00:32.788ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:42785
20962024-12-19T01:00:32.788ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:42785 req_id = 5a28c706-6b67-448c-999f-074df27b5661 response_code = 200 uri = /baseboard
20972024-12-19T01:00:35.765ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6585 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = 425a4936-c3db-41c2-a501-665614b67670 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
20982024-12-19T01:00:51.538ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56855
20992024-12-19T01:00:51.538ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56855 req_id = 1d06ceb7-760c-492f-9d22-a6f656b92717 response_code = 200 uri = /vpc-routes
21002024-12-19T01:00:51.595ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64012
21012024-12-19T01:00:51.595ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64012 req_id = a77e9108-676f-4e32-abbf-d37c749d0e7c response_code = 200 uri = /vpc-routes
21022024-12-19T01:00:51.616ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
21032024-12-19T01:00:51.616ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56855 req_id = c84dd927-d12e-4514-91aa-7fbd2d8571a3 response_code = 204 uri = /eip-gateways
21042024-12-19T01:00:51.665ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
21052024-12-19T01:00:51.665ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64012 req_id = 48f33496-52a4-4e8c-b9ad-547acb93bea1 response_code = 204 uri = /eip-gateways
21062024-12-19T01:00:51.675ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 198 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64012 req_id = 7a1a12cb-c3c7-4d77-8384-33f21a8362b7 response_code = 204 uri = /vpc-routes
21072024-12-19T01:00:51.720ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55452
21082024-12-19T01:00:51.720ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21092024-12-19T01:00:51.762ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21102024-12-19T01:00:51.821ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101094 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:55452 req_id = 9d8ef72c-8aa2-4359-80c7-778b0ac87260 response_code = 204 uri = /switch-ports
21112024-12-19T01:00:51.821ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47945
21122024-12-19T01:00:51.821ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34597
21132024-12-19T01:00:51.822ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53269
21142024-12-19T01:00:51.822ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 311 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55452 req_id = d15bbe98-4c2a-43af-bb65-16c3663bc24a response_code = 200 uri = /network-bootstore-config
21152024-12-19T01:00:51.822ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 579 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56855 req_id = 682cd0c4-a431-4b58-84bc-833c368abc34 response_code = 204 uri = /vpc-routes
21162024-12-19T01:00:51.822ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 251 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47945 req_id = 7e8f2d9e-bc97-4e40-86f1-4ba1d3137851 response_code = 200 uri = /v2p
21172024-12-19T01:00:51.822ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 240 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34597 req_id = 6c65ff19-ea11-46f8-8ed9-f2b2f0a4e967 response_code = 200 uri = /v2p
21182024-12-19T01:00:51.823ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21192024-12-19T01:00:51.852ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65315
21202024-12-19T01:00:51.853ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 261 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65315 req_id = 0eb9a197-1422-40f6-b6d0-e6297d5f7131 response_code = 200 uri = /v2p
21212024-12-19T01:00:51.860ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21222024-12-19T01:00:51.869ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55461
21232024-12-19T01:00:51.869ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55461 req_id = 5bb8910b-01f1-4dee-b290-39498164471d response_code = 200 uri = /vpc-routes
21242024-12-19T01:00:51.917ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
21252024-12-19T01:00:51.917ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55461 req_id = df52a92b-9bc9-4b74-a0fd-1ad04b9860c4 response_code = 204 uri = /eip-gateways
21262024-12-19T01:00:51.924ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101588 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53269 req_id = 102a0aa0-b6fc-4e97-a05e-d2e45f7ffe91 response_code = 204 uri = /switch-ports
21272024-12-19T01:00:51.924ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53269 req_id = 707b9a14-fe3b-425e-869f-76e60b6be448 response_code = 200 uri = /network-bootstore-config
21282024-12-19T01:00:51.938ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59710
21292024-12-19T01:00:51.938ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21302024-12-19T01:00:51.976ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21312024-12-19T01:00:52.032ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93632 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59710 req_id = 01cde73d-4018-4d6c-b8bd-e8c9ca810739 response_code = 204 uri = /switch-ports
21322024-12-19T01:00:52.032ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59710 req_id = e3094741-1a1e-45ab-8e85-4632ec43e63b response_code = 200 uri = /network-bootstore-config
21332024-12-19T01:00:52.038ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 415 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55461 req_id = b1d3e16c-47b9-46e3-b021-5075296369a0 response_code = 204 uri = /vpc-routes
21342024-12-19T01:01:02.822ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47481
21352024-12-19T01:01:02.822ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 27 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47481 req_id = 8c10664a-1dcc-4dd9-ba10-682daed7cec9 response_code = 200 uri = /baseboard
21362024-12-19T01:01:05.767ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7319 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = befa3d16-fc88-457b-abe3-af75752056da response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
21372024-12-19T01:01:21.881ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60071
21382024-12-19T01:01:21.881ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60071 req_id = bf40a67c-85b5-408c-aa45-908b330fd65b response_code = 200 uri = /vpc-routes
21392024-12-19T01:01:21.959ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
21402024-12-19T01:01:21.959ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60071 req_id = 7814a653-1605-42ad-b5e2-9b1848776f0a response_code = 204 uri = /eip-gateways
21412024-12-19T01:01:21.985ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57432
21422024-12-19T01:01:21.985ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21432024-12-19T01:01:22.029ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21442024-12-19T01:01:22.100ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114883 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:57432 req_id = a878fa5a-155d-4176-83db-c0cd74a621de response_code = 204 uri = /switch-ports
21452024-12-19T01:01:22.101ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38167
21462024-12-19T01:01:22.101ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38893
21472024-12-19T01:01:22.101ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53641
21482024-12-19T01:01:22.101ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48994
21492024-12-19T01:01:22.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 391 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57432 req_id = cf700bc0-9e83-4f10-a983-1dbf095c6a03 response_code = 200 uri = /network-bootstore-config
21502024-12-19T01:01:22.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 842 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60071 req_id = 07b35799-b056-4b53-8bf3-9cb86676c572 response_code = 204 uri = /vpc-routes
21512024-12-19T01:01:22.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 308 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48994 req_id = a6bc7061-b77c-4e40-ad01-0d25babf51a2 response_code = 200 uri = /vpc-routes
21522024-12-19T01:01:22.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 450 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38167 req_id = df69e807-5923-4326-a10b-80fadea4d689 response_code = 200 uri = /vpc-routes
21532024-12-19T01:01:22.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 304 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53641 req_id = 5d145a34-b7f2-417c-8b50-fb676212690e response_code = 200 uri = /v2p
21542024-12-19T01:01:22.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 363 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38893 req_id = c9f95338-2570-48de-b383-e8bf8ab888e5 response_code = 200 uri = /v2p
21552024-12-19T01:01:22.166ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57361
21562024-12-19T01:01:22.166ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21572024-12-19T01:01:22.215ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21582024-12-19T01:01:22.297ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 130887 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:57361 req_id = fa3bd795-ad70-4746-beb0-58f233c6d21c response_code = 204 uri = /switch-ports
21592024-12-19T01:01:22.297ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57819
21602024-12-19T01:01:22.297ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38723
21612024-12-19T01:01:22.297ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
21622024-12-19T01:01:22.297ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 257 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38167 req_id = 6d59ed03-6d1c-4527-9888-fe9657aba2d0 response_code = 204 uri = /eip-gateways
21632024-12-19T01:01:22.297ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
21642024-12-19T01:01:22.297ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 321 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57361 req_id = 8dab7953-6c46-49bc-b02e-0347810838f4 response_code = 200 uri = /network-bootstore-config
21652024-12-19T01:01:22.297ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 288 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48994 req_id = c788d1ff-e466-4b7a-9101-a54dc7706fdb response_code = 204 uri = /eip-gateways
21662024-12-19T01:01:22.298ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21672024-12-19T01:01:22.298ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 282 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57819 req_id = e25ea844-3f95-4d2d-931f-a23a7ee43db8 response_code = 200 uri = /v2p
21682024-12-19T01:01:22.309ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 417 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48994 req_id = 9063f944-1ef4-4c47-a0dd-a77593332dc1 response_code = 204 uri = /vpc-routes
21692024-12-19T01:01:22.311ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 200 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38167 req_id = 722e9059-1405-4275-9c62-22e37f34c9b3 response_code = 204 uri = /vpc-routes
21702024-12-19T01:01:22.353ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21712024-12-19T01:01:22.430ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 132110 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:38723 req_id = e8fbfd7e-bb3e-45cf-afdd-7076498a5227 response_code = 204 uri = /switch-ports
21722024-12-19T01:01:22.430ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 165 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38723 req_id = b175f0fe-bef4-43ae-bea0-61570acfa5ea response_code = 200 uri = /network-bootstore-config
21732024-12-19T01:01:23.748ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
21742024-12-19T01:01:32.856ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62536
21752024-12-19T01:01:32.856ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62536 req_id = af439cb8-728a-4179-b8a4-d5f7c7631b7a response_code = 200 uri = /baseboard
21762024-12-19T01:01:35.768ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8429 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = ed5e984d-74f3-4c70-9640-6330666e61df response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
21772024-12-19T01:01:51.541ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57547
21782024-12-19T01:01:51.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57547 req_id = 87e23ce2-4f0f-4b6b-8b51-aaf75727db3e response_code = 200 uri = /vpc-routes
21792024-12-19T01:01:51.580ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38433
21802024-12-19T01:01:51.580ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38433 req_id = 9a0dcd44-8507-42fb-a4da-ae5052192568 response_code = 200 uri = /vpc-routes
21812024-12-19T01:01:51.599ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
21822024-12-19T01:01:51.599ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57547 req_id = 74ce8d51-fe4b-46c5-b9f6-37de3b0af4b6 response_code = 204 uri = /eip-gateways
21832024-12-19T01:01:51.613ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57547 req_id = 2b11c949-ad4f-4f8b-b77b-9f2da8fb1818 response_code = 204 uri = /vpc-routes
21842024-12-19T01:01:51.642ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
21852024-12-19T01:01:51.642ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38433 req_id = 145f5d46-f44f-4bf0-8c26-fe345a37cf72 response_code = 204 uri = /eip-gateways
21862024-12-19T01:01:51.651ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59348
21872024-12-19T01:01:51.651ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21882024-12-19T01:01:51.702ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21892024-12-19T01:01:51.760ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108867 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:59348 req_id = 6adbf552-4560-4f86-905b-312450ddd518 response_code = 204 uri = /switch-ports
21902024-12-19T01:01:51.760ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38180
21912024-12-19T01:01:51.760ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37575
21922024-12-19T01:01:51.760ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63156
21932024-12-19T01:01:51.761ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21942024-12-19T01:01:51.761ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 487 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59348 req_id = 71c359bc-576e-4f0b-bc54-a658feee497a response_code = 200 uri = /network-bootstore-config
21952024-12-19T01:01:51.761ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 373 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38180 req_id = 52c3dff1-87c1-4e89-85ea-4e8df1008092 response_code = 200 uri = /v2p
21962024-12-19T01:01:51.761ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 340 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63156 req_id = dddf2317-b287-4f2a-8003-0f92d80221fc response_code = 200 uri = /v2p
21972024-12-19T01:01:51.761ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 696 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38433 req_id = a512e307-f537-4ad0-9e89-0bb91d59ad80 response_code = 204 uri = /vpc-routes
21982024-12-19T01:01:51.799ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21992024-12-19T01:01:51.819ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35878
22002024-12-19T01:01:51.819ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35878 req_id = 935ac035-81a9-49c3-b60b-3abe7088aaee response_code = 200 uri = /vpc-routes
22012024-12-19T01:01:51.859ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98094 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:37575 req_id = 033a9d3f-458f-4568-aea5-652003344baf response_code = 204 uri = /switch-ports
22022024-12-19T01:01:51.859ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37575 req_id = adb504b4-4c9d-427e-ac61-7a388c082b15 response_code = 200 uri = /network-bootstore-config
22032024-12-19T01:01:51.865ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
22042024-12-19T01:01:51.865ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35878 req_id = 08718908-49bb-42d9-aead-f65ce179d9d8 response_code = 204 uri = /eip-gateways
22052024-12-19T01:01:51.876ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 310 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35878 req_id = d7dad6c7-cc6a-4f4b-b6c9-202175a723e3 response_code = 204 uri = /vpc-routes
22062024-12-19T01:01:51.955ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63063
22072024-12-19T01:01:51.956ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63063 req_id = d8d24f86-2965-48bb-8cce-8438abae1476 response_code = 200 uri = /v2p
22082024-12-19T01:01:51.959ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54956
22092024-12-19T01:01:51.959ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22102024-12-19T01:01:51.998ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22112024-12-19T01:01:52.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97739 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54956 req_id = 7210340d-0d27-40c9-a2d4-206e7b2587b8 response_code = 204 uri = /switch-ports
22122024-12-19T01:01:52.058ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54956 req_id = f0bddceb-0e2b-4298-be70-8713296ebe20 response_code = 200 uri = /network-bootstore-config
22132024-12-19T01:02:02.889ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:41461
22142024-12-19T01:02:02.889ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:41461 req_id = 55062366-a57f-4970-b0b9-6abee2cb8281 response_code = 200 uri = /baseboard
22152024-12-19T01:02:05.767ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7028 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = 84cfe6da-e5c8-46b2-b770-fa99e50001a0 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
22162024-12-19T01:02:21.745ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51995
22172024-12-19T01:02:21.745ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51995 req_id = cc0e1ee1-df61-4778-865b-ab27c070b9bb response_code = 200 uri = /vpc-routes
22182024-12-19T01:02:21.800ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35590
22192024-12-19T01:02:21.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 30 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35590 req_id = d65707f2-f270-4bb3-9284-a8903f30991b response_code = 200 uri = /vpc-routes
22202024-12-19T01:02:21.821ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
22212024-12-19T01:02:21.821ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35590 req_id = 5740f8d7-567c-4d94-b88a-7e8e109fd2df response_code = 204 uri = /eip-gateways
22222024-12-19T01:02:21.828ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35590 req_id = 1a60182d-fbd0-48ad-8cdb-8aba9d119e49 response_code = 204 uri = /vpc-routes
22232024-12-19T01:02:21.837ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60169
22242024-12-19T01:02:21.837ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 30 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60169 req_id = 19bc22de-c23e-42f6-a69c-d3762c619d61 response_code = 200 uri = /vpc-routes
22252024-12-19T01:02:21.855ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49661
22262024-12-19T01:02:21.855ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49661 req_id = abe5c278-ec00-43d6-b916-f18562b68962 response_code = 200 uri = /v2p
22272024-12-19T01:02:21.884ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
22282024-12-19T01:02:21.884ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60169 req_id = 28ced21c-9dc6-4cda-ac1b-7c6160893965 response_code = 204 uri = /eip-gateways
22292024-12-19T01:02:21.892ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60169 req_id = 941e78af-9c1d-4853-b3c8-8c46d0491443 response_code = 204 uri = /vpc-routes
22302024-12-19T01:02:21.912ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
22312024-12-19T01:02:21.912ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51995 req_id = 53e74b79-61a4-4c05-a009-8b2c068135a2 response_code = 204 uri = /eip-gateways
22322024-12-19T01:02:21.920ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51995 req_id = dc93fbe7-a931-449f-83a6-64b32d13b76b response_code = 204 uri = /vpc-routes
22332024-12-19T01:02:21.935ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55009
22342024-12-19T01:02:21.935ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55009 req_id = 88b61b2f-f184-4863-80d5-e9278f831a03 response_code = 200 uri = /v2p
22352024-12-19T01:02:21.956ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35513
22362024-12-19T01:02:21.956ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22372024-12-19T01:02:22.000ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22382024-12-19T01:02:22.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100971 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:35513 req_id = eb504553-d40f-4cdb-b618-aa4d4e74c0db response_code = 204 uri = /switch-ports
22392024-12-19T01:02:22.058ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53221
22402024-12-19T01:02:22.058ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22412024-12-19T01:02:22.093ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22422024-12-19T01:02:22.150ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92403 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:53221 req_id = e8e6885a-6b44-4263-95f4-66f3f8fb8717 response_code = 204 uri = /switch-ports
22432024-12-19T01:02:22.150ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49446
22442024-12-19T01:02:22.150ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35513 req_id = 887608ef-b3d4-482e-a8f2-f937ba0656ee response_code = 200 uri = /network-bootstore-config
22452024-12-19T01:02:22.150ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58794
22462024-12-19T01:02:22.151ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 271 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53221 req_id = 6434fc42-fbf5-426e-8269-ccee62ba13c0 response_code = 200 uri = /network-bootstore-config
22472024-12-19T01:02:22.151ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49446 req_id = c28d35f2-8c2a-43be-804c-b989124c69c3 response_code = 200 uri = /v2p
22482024-12-19T01:02:22.151ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22492024-12-19T01:02:22.190ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22502024-12-19T01:02:22.245ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93917 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:58794 req_id = 50b50ee6-2f51-499d-bb2c-32e6c8226490 response_code = 204 uri = /switch-ports
22512024-12-19T01:02:22.245ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58794 req_id = e4e98eeb-9cb8-4aa6-9f43-519132d357ec response_code = 200 uri = /network-bootstore-config
22522024-12-19T01:02:32.922ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62353
22532024-12-19T01:02:32.922ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62353 req_id = 46a6afb2-cd5e-455b-819e-9ab115f80fba response_code = 200 uri = /baseboard
22542024-12-19T01:02:35.767ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7401 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = b240248d-262e-4b0b-b35b-a4fb99271865 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
22552024-12-19T01:02:51.593ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57618
22562024-12-19T01:02:51.593ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57618 req_id = 1765d03b-4129-4290-80a2-71cbc205a3fc response_code = 200 uri = /vpc-routes
22572024-12-19T01:02:51.618ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
22582024-12-19T01:02:51.618ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57618 req_id = bd857a65-2c6d-433a-b364-15b899772716 response_code = 204 uri = /eip-gateways
22592024-12-19T01:02:51.628ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53172
22602024-12-19T01:02:51.628ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53172 req_id = acee05a7-c0b5-411f-b78f-5eb725ffe805 response_code = 200 uri = /vpc-routes
22612024-12-19T01:02:51.633ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 288 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57618 req_id = 0a7768cb-a9d5-4e74-a36a-9fb93c3c7c29 response_code = 204 uri = /vpc-routes
22622024-12-19T01:02:51.644ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
22632024-12-19T01:02:51.644ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53172 req_id = 9443ac0f-48ca-49ea-9693-cd94a7ccc825 response_code = 204 uri = /eip-gateways
22642024-12-19T01:02:51.647ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46013
22652024-12-19T01:02:51.648ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46013 req_id = 41989f3a-f61d-40a2-8219-5b431da0b56d response_code = 200 uri = /v2p
22662024-12-19T01:02:51.662ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 226 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53172 req_id = f4ac4271-033e-47d6-aeaa-ee1f2d6e3894 response_code = 204 uri = /vpc-routes
22672024-12-19T01:02:51.698ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40433
22682024-12-19T01:02:51.698ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40433 req_id = 46fe2460-90f1-43e2-85f2-91a4412534fb response_code = 200 uri = /v2p
22692024-12-19T01:02:51.736ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60993
22702024-12-19T01:02:51.736ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22712024-12-19T01:02:51.778ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22722024-12-19T01:02:51.838ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101845 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:60993 req_id = ba29a011-1a0b-4eda-bd49-a71ada2ab7b7 response_code = 204 uri = /switch-ports
22732024-12-19T01:02:51.839ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35576
22742024-12-19T01:02:51.839ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22752024-12-19T01:02:51.839ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 276 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60993 req_id = 9ace94d0-8025-4e5c-bdaf-ad29d3890297 response_code = 200 uri = /network-bootstore-config
22762024-12-19T01:02:51.851ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54845
22772024-12-19T01:02:51.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54845 req_id = eb909084-9e83-4e53-bb19-2ebad45b629e response_code = 200 uri = /vpc-routes
22782024-12-19T01:02:51.883ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22792024-12-19T01:02:51.900ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
22802024-12-19T01:02:51.900ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 666 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54845 req_id = 8452402d-f227-4810-8542-a94b487567e1 response_code = 204 uri = /eip-gateways
22812024-12-19T01:02:51.942ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50563
22822024-12-19T01:02:51.942ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 269 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50563 req_id = 856c903e-5c80-422e-be2a-b8a4195c34a7 response_code = 200 uri = /v2p
22832024-12-19T01:02:51.950ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111403 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:35576 req_id = 1d6c407a-4441-4c5a-8de1-dc62996ce92a response_code = 204 uri = /switch-ports
22842024-12-19T01:02:51.951ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35576 req_id = b569a269-d349-47ed-9a28-b1f905b2eef0 response_code = 200 uri = /network-bootstore-config
22852024-12-19T01:02:51.952ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 332 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54845 req_id = f171446e-1a04-4174-b98a-605aa512f4b5 response_code = 204 uri = /vpc-routes
22862024-12-19T01:02:52.064ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39928
22872024-12-19T01:02:52.064ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22882024-12-19T01:02:52.101ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22892024-12-19T01:02:52.154ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89697 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:39928 req_id = 21cb3abf-2754-46a0-955e-cbb212b9c57b response_code = 204 uri = /switch-ports
22902024-12-19T01:02:52.154ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39928 req_id = 2821a8dd-97d0-492a-b70b-5f14c04f0d39 response_code = 200 uri = /network-bootstore-config
22912024-12-19T01:03:02.955ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:51413
22922024-12-19T01:03:02.956ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:51413 req_id = c85f980a-bcf8-4352-9e16-7f6ba7f64226 response_code = 200 uri = /baseboard
22932024-12-19T01:03:05.767ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7201 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = a3254d9e-5ee3-4638-8064-b874c2938214 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
22942024-12-19T01:03:21.692ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:32773
22952024-12-19T01:03:21.695ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:32773 req_id = 5d7699e9-06e5-4b8b-8a36-1001f59dcd92 response_code = 200 uri = /vpc-routes
22962024-12-19T01:03:21.771ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
22972024-12-19T01:03:21.771ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:32773 req_id = 088c78a7-6abe-4fcf-b2bf-c820696fd814 response_code = 204 uri = /eip-gateways
22982024-12-19T01:03:21.783ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 334 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:32773 req_id = a28479c6-81e5-41e5-aa41-cf18e6c9f31b response_code = 204 uri = /vpc-routes
22992024-12-19T01:03:21.798ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59567
23002024-12-19T01:03:21.798ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59567 req_id = 15f69524-5e4e-4ba2-a072-a3cc611d3c0b response_code = 200 uri = /vpc-routes
23012024-12-19T01:03:21.819ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53641
23022024-12-19T01:03:21.819ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 171 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53641 req_id = 53eebd43-038e-4e81-bfa9-d269e26c0c25 response_code = 200 uri = /v2p
23032024-12-19T01:03:21.869ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
23042024-12-19T01:03:21.869ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59567 req_id = b9732994-29b1-48e2-96ae-755388c71dba response_code = 204 uri = /eip-gateways
23052024-12-19T01:03:21.883ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 203 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59567 req_id = 1b2d3bed-0f49-4bc3-9878-02b06b72d668 response_code = 204 uri = /vpc-routes
23062024-12-19T01:03:21.948ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41914
23072024-12-19T01:03:21.948ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
23082024-12-19T01:03:21.992ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
23092024-12-19T01:03:22.061ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113031 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:41914 req_id = 146a46d0-8663-4d43-bf42-55877e4b6ca8 response_code = 204 uri = /switch-ports
23102024-12-19T01:03:22.061ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40560
23112024-12-19T01:03:22.061ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34508
23122024-12-19T01:03:22.061ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39023
23132024-12-19T01:03:22.062ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34508 req_id = a955a170-486b-4dd2-aa5d-1ed356b6eb75 response_code = 200 uri = /vpc-routes
23142024-12-19T01:03:22.062ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 278 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41914 req_id = ef788574-1abc-43a2-b90e-3c3a98303398 response_code = 200 uri = /network-bootstore-config
23152024-12-19T01:03:22.062ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 5b97cf45-6db1-45ed-9f7e-111d132f7ed4, kind: Service { id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 }, name: Name("external-dns-2a880a2e-2471-4fd8-9ad5-3e0c3fbde317"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 253])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d21d0095-8e84-486f-b3a4-f9dca167b9d5, kind: Service { id: 679d67b5-9925-49ac-b78d-613e816394dc }, name: Name("external-dns-679d67b5-9925-49ac-b78d-613e816394dc"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 205])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 82094354-655d-415c-9763-28731eb50642, kind: Service { id: e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 }, name: Name("nexus-e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 60])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5c3ad41-155b-469b-bb3f-412814b9d0ff, kind: Service { id: 1176f008-4a9d-4835-a5aa-1a86ec3c619c }, name: Name("nexus-1176f008-4a9d-4835-a5aa-1a86ec3c619c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 188])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ef89a8b8-3cc5-4637-aee2-3b98f04f51be, kind: Service { id: 1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 }, name: Name("nexus-1d1e5635-dd2b-4bf7-8ace-6cb3ea634995"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 75])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
23162024-12-19T01:03:22.062ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 323 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40560 req_id = 354cce8e-58c3-4cf0-b817-f3fa6745e846 response_code = 200 uri = /v2p
23172024-12-19T01:03:22.062ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23182024-12-19T01:03:22.062ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23192024-12-19T01:03:22.062ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23202024-12-19T01:03:22.062ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
23212024-12-19T01:03:22.062ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23222024-12-19T01:03:22.062ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23232024-12-19T01:03:22.062ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 583 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39023 req_id = 505a5f8e-f23c-47ca-a849-200f9aac5a49 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23242024-12-19T01:03:22.073ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34879
23252024-12-19T01:03:22.073ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34879 req_id = 35ee5ebf-f128-448e-b42b-d4efdb0b9483 response_code = 200 uri = /v2p
23262024-12-19T01:03:22.088ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
23272024-12-19T01:03:22.088ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34508 req_id = 563be41c-e3cd-44b2-b59b-58d5d8c386ca response_code = 204 uri = /eip-gateways
23282024-12-19T01:03:22.097ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 305 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34508 req_id = 78853f87-1a51-4c63-9b43-f0865c38a794 response_code = 204 uri = /vpc-routes
23292024-12-19T01:03:22.132ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52763
23302024-12-19T01:03:22.132ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
23312024-12-19T01:03:22.172ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
23322024-12-19T01:03:22.233ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100963 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:52763 req_id = e6ea3cbc-c25a-4913-bf0b-922f634ff629 response_code = 204 uri = /switch-ports
23332024-12-19T01:03:22.234ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43761
23342024-12-19T01:03:22.234ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40179
23352024-12-19T01:03:22.234ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
23362024-12-19T01:03:22.234ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 284 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52763 req_id = ae0986bc-b994-4ccd-a793-da25ceaa816f response_code = 200 uri = /network-bootstore-config
23372024-12-19T01:03:22.237ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 5b97cf45-6db1-45ed-9f7e-111d132f7ed4, kind: Service { id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 }, name: Name("external-dns-2a880a2e-2471-4fd8-9ad5-3e0c3fbde317"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 253])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d21d0095-8e84-486f-b3a4-f9dca167b9d5, kind: Service { id: 679d67b5-9925-49ac-b78d-613e816394dc }, name: Name("external-dns-679d67b5-9925-49ac-b78d-613e816394dc"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 205])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 82094354-655d-415c-9763-28731eb50642, kind: Service { id: e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 }, name: Name("nexus-e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 60])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5c3ad41-155b-469b-bb3f-412814b9d0ff, kind: Service { id: 1176f008-4a9d-4835-a5aa-1a86ec3c619c }, name: Name("nexus-1176f008-4a9d-4835-a5aa-1a86ec3c619c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 188])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ef89a8b8-3cc5-4637-aee2-3b98f04f51be, kind: Service { id: 1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 }, name: Name("nexus-1d1e5635-dd2b-4bf7-8ace-6cb3ea634995"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 75])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
23382024-12-19T01:03:22.237ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23392024-12-19T01:03:22.237ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23402024-12-19T01:03:22.237ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23412024-12-19T01:03:22.237ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
23422024-12-19T01:03:22.238ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23432024-12-19T01:03:22.238ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23442024-12-19T01:03:22.238ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3827 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40179 req_id = fdfd9a8a-18b8-46a1-9139-523104c6b48c response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23452024-12-19T01:03:22.250ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:32861
23462024-12-19T01:03:22.251ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 5b97cf45-6db1-45ed-9f7e-111d132f7ed4, kind: Service { id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 }, name: Name("external-dns-2a880a2e-2471-4fd8-9ad5-3e0c3fbde317"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 253])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d21d0095-8e84-486f-b3a4-f9dca167b9d5, kind: Service { id: 679d67b5-9925-49ac-b78d-613e816394dc }, name: Name("external-dns-679d67b5-9925-49ac-b78d-613e816394dc"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 205])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 82094354-655d-415c-9763-28731eb50642, kind: Service { id: e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 }, name: Name("nexus-e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 60])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5c3ad41-155b-469b-bb3f-412814b9d0ff, kind: Service { id: 1176f008-4a9d-4835-a5aa-1a86ec3c619c }, name: Name("nexus-1176f008-4a9d-4835-a5aa-1a86ec3c619c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 188])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ef89a8b8-3cc5-4637-aee2-3b98f04f51be, kind: Service { id: 1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 }, name: Name("nexus-1d1e5635-dd2b-4bf7-8ace-6cb3ea634995"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 75])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
23472024-12-19T01:03:22.251ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23482024-12-19T01:03:22.251ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23492024-12-19T01:03:22.251ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23502024-12-19T01:03:22.251ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
23512024-12-19T01:03:22.251ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23522024-12-19T01:03:22.251ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23532024-12-19T01:03:22.251ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 659 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:32861 req_id = faf6b17f-4275-4c2d-adf4-503ac57921b7 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23542024-12-19T01:03:22.270ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
23552024-12-19T01:03:22.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89133 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:43761 req_id = d0f011fd-1907-41a0-a40d-d0078d8dfad7 response_code = 204 uri = /switch-ports
23562024-12-19T01:03:22.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43761 req_id = 9a86ef78-b45c-4bf2-87c6-741f3996d6e9 response_code = 200 uri = /network-bootstore-config
23572024-12-19T01:03:32.988ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:60279
23582024-12-19T01:03:32.989ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:60279 req_id = 13f992cb-6f06-45d0-af41-b1e9a8a6a985 response_code = 200 uri = /baseboard
23592024-12-19T01:03:35.768ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8586 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = f7fc9e95-9d5a-4dae-9daa-3010fd920dcf response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
23602024-12-19T01:03:51.630ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58862
23612024-12-19T01:03:51.630ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58862 req_id = 18183ca1-f957-4633-9b98-ddeff720a6c0 response_code = 200 uri = /vpc-routes
23622024-12-19T01:03:51.669ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34155
23632024-12-19T01:03:51.669ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34155 req_id = 2953d262-6932-4dcf-b017-1e7afcb7a03e response_code = 200 uri = /vpc-routes
23642024-12-19T01:03:51.680ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
23652024-12-19T01:03:51.680ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58862 req_id = 936a4cde-85d3-4a81-8048-c41eca9263cc response_code = 204 uri = /eip-gateways
23662024-12-19T01:03:51.689ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 187 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58862 req_id = f4443575-761e-4de7-808e-f5e4f2dfe461 response_code = 204 uri = /vpc-routes
23672024-12-19T01:03:51.693ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62324
23682024-12-19T01:03:51.693ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62324 req_id = 36617326-e007-4a15-9e53-4508655906d1 response_code = 200 uri = /v2p
23692024-12-19T01:03:51.720ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
23702024-12-19T01:03:51.720ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34155 req_id = b09fec95-8530-4abb-936b-603c52516939 response_code = 204 uri = /eip-gateways
23712024-12-19T01:03:51.728ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 183 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34155 req_id = ce1d7184-8674-47e5-bc7b-9e095ef76f56 response_code = 204 uri = /vpc-routes
23722024-12-19T01:03:51.735ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49303
23732024-12-19T01:03:51.735ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
23742024-12-19T01:03:51.773ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
23752024-12-19T01:03:51.825ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90556 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:49303 req_id = 4f3e04df-b7e1-4e4e-8610-4a1cacdf5121 response_code = 204 uri = /switch-ports
23762024-12-19T01:03:51.825ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47395
23772024-12-19T01:03:51.825ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51944
23782024-12-19T01:03:51.825ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58021
23792024-12-19T01:03:51.826ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 232 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58021 req_id = 55dde65d-4e08-4bc1-9b44-e6a90c40d73b response_code = 200 uri = /v2p
23802024-12-19T01:03:51.826ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 327 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49303 req_id = b59ce000-5632-414c-aa8c-93c9c8a4a0d7 response_code = 200 uri = /network-bootstore-config
23812024-12-19T01:03:51.826ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
23822024-12-19T01:03:51.826ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 272 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51944 req_id = 78eaf0b2-aa42-43f5-9ed5-eeaf42c526ed response_code = 200 uri = /v2p
23832024-12-19T01:03:51.865ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
23842024-12-19T01:03:51.906ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63977
23852024-12-19T01:03:51.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63977 req_id = 6468ff08-bd55-4dc2-b308-c64f46d340a2 response_code = 200 uri = /vpc-routes
23862024-12-19T01:03:51.920ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94106 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47395 req_id = ae3286ff-02d2-48b3-b06d-717ba5c94105 response_code = 204 uri = /switch-ports
23872024-12-19T01:03:51.920ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47395 req_id = 1c3514b5-3cd3-4f76-99bc-4f559dbfdcd8 response_code = 200 uri = /network-bootstore-config
23882024-12-19T01:03:51.950ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
23892024-12-19T01:03:51.950ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63977 req_id = 349153e8-b1ce-4868-ac49-1e88ca1a7e45 response_code = 204 uri = /eip-gateways
23902024-12-19T01:03:51.958ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 289 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63977 req_id = 345cd78e-4612-44d2-8ad5-ccb62ad93668 response_code = 204 uri = /vpc-routes
23912024-12-19T01:03:51.972ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50946
23922024-12-19T01:03:51.972ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
23932024-12-19T01:03:52.009ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
23942024-12-19T01:03:52.064ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91512 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50946 req_id = 7f4551c8-cf36-4b22-8181-83d1d07b9f58 response_code = 204 uri = /switch-ports
23952024-12-19T01:03:52.064ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50946 req_id = 0f0ce2ed-c79a-487f-a861-e7b77978ad64 response_code = 200 uri = /network-bootstore-config
23962024-12-19T01:04:03.023ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:57351
23972024-12-19T01:04:03.023ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:57351 req_id = 76b88c5b-eb37-4c83-8eb8-d37daedde247 response_code = 200 uri = /baseboard
23982024-12-19T01:04:05.767ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7543 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = dc959a7d-16dc-4fa1-a56f-82758ea4b973 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
23992024-12-19T01:04:21.776ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44691
24002024-12-19T01:04:21.776ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44691 req_id = c2897e19-cac2-415a-8149-61dcea26531b response_code = 200 uri = /vpc-routes
24012024-12-19T01:04:21.813ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33016
24022024-12-19T01:04:21.813ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33016 req_id = 34aa2ca6-b209-4b68-b808-8d3c25819551 response_code = 200 uri = /vpc-routes
24032024-12-19T01:04:21.832ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
24042024-12-19T01:04:21.832ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44691 req_id = 73f431b1-1655-46de-9383-5355f41658b2 response_code = 204 uri = /eip-gateways
24052024-12-19T01:04:21.843ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 221 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44691 req_id = e6e2e890-8646-4140-94c3-4f613290882a response_code = 204 uri = /vpc-routes
24062024-12-19T01:04:21.867ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44538
24072024-12-19T01:04:21.867ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44538 req_id = 207f09ca-ba17-433e-9aa1-b0f0c3e8586e response_code = 200 uri = /v2p
24082024-12-19T01:04:21.868ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
24092024-12-19T01:04:21.868ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:33016 req_id = d1045be7-ca75-421e-9865-816ef13d6be4 response_code = 204 uri = /eip-gateways
24102024-12-19T01:04:21.879ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 259 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:33016 req_id = 537e7846-d210-4209-b146-acb95ed3ecbd response_code = 204 uri = /vpc-routes
24112024-12-19T01:04:21.900ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50367
24122024-12-19T01:04:21.900ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 28 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50367 req_id = 663d08c6-1e6d-4bde-96c9-cf25b353e045 response_code = 200 uri = /vpc-routes
24132024-12-19T01:04:21.903ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34649
24142024-12-19T01:04:21.904ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24152024-12-19T01:04:21.946ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24162024-12-19T01:04:22.015ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111533 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:34649 req_id = 84d3d252-c2e4-4ca8-98e7-3cf55164135b response_code = 204 uri = /switch-ports
24172024-12-19T01:04:22.015ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53437
24182024-12-19T01:04:22.015ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47144
24192024-12-19T01:04:22.016ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
24202024-12-19T01:04:22.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 270 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50367 req_id = 5af31d6c-b160-4acc-bb43-2a349823ef1f response_code = 204 uri = /eip-gateways
24212024-12-19T01:04:22.016ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24222024-12-19T01:04:22.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 309 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53437 req_id = a561388f-f75f-4eda-931b-890e430454b2 response_code = 200 uri = /v2p
24232024-12-19T01:04:22.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 379 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34649 req_id = e365ccb1-60cb-486b-a947-7b440d9a91a6 response_code = 200 uri = /network-bootstore-config
24242024-12-19T01:04:22.028ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 309 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50367 req_id = db7c7993-ed5f-4440-8f2c-926b686f15da response_code = 204 uri = /vpc-routes
24252024-12-19T01:04:22.053ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24262024-12-19T01:04:22.067ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49082
24272024-12-19T01:04:22.068ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 257 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49082 req_id = cd21562d-cc4a-40fe-b7cf-eb758be1f110 response_code = 200 uri = /v2p
24282024-12-19T01:04:22.110ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94346 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47144 req_id = 97fdfe00-c885-4b1d-8a7e-c46814c0ab1d response_code = 204 uri = /switch-ports
24292024-12-19T01:04:22.110ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47144 req_id = 584a3c13-d965-43ad-a894-a64da17c26f6 response_code = 200 uri = /network-bootstore-config
24302024-12-19T01:04:22.116ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63443
24312024-12-19T01:04:22.117ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24322024-12-19T01:04:22.151ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24332024-12-19T01:04:22.204ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87461 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63443 req_id = 94a64d67-fdb0-4e3d-bc90-3bad4f6a13eb response_code = 204 uri = /switch-ports
24342024-12-19T01:04:22.204ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63443 req_id = 1825c877-a2ad-432b-9864-0f8f2dc583c0 response_code = 200 uri = /network-bootstore-config
24352024-12-19T01:04:33.058ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:55143
24362024-12-19T01:04:33.059ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 77 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:55143 req_id = da23f338-c596-4c98-a103-5c3bec57a9f0 response_code = 200 uri = /baseboard
24372024-12-19T01:04:35.769ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8193 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = c5cd3660-0e71-4926-8d5b-624ff397d25b response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
24382024-12-19T01:04:51.597ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52043
24392024-12-19T01:04:51.598ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52043 req_id = b0a63aeb-0626-42bb-8795-9bf1f529528a response_code = 200 uri = /vpc-routes
24402024-12-19T01:04:51.659ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
24412024-12-19T01:04:51.659ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52043 req_id = 3d13747f-f8d1-4c96-bdb7-f735f647ea5d response_code = 204 uri = /eip-gateways
24422024-12-19T01:04:51.669ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 202 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52043 req_id = b5b43898-cd87-4568-a2e3-19f6c619faa3 response_code = 204 uri = /vpc-routes
24432024-12-19T01:04:51.683ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61190
24442024-12-19T01:04:51.683ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61190 req_id = 94e68b7d-59a2-4fee-925f-3a7cb950ca66 response_code = 200 uri = /v2p
24452024-12-19T01:04:51.691ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40051
24462024-12-19T01:04:51.691ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40051 req_id = d5748e05-433e-480c-a477-eeffd1b4e9d3 response_code = 200 uri = /vpc-routes
24472024-12-19T01:04:51.711ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
24482024-12-19T01:04:51.711ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40051 req_id = 98921bff-0f6a-436f-bc78-a898d4fe2ea4 response_code = 204 uri = /eip-gateways
24492024-12-19T01:04:51.715ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51413
24502024-12-19T01:04:51.715ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24512024-12-19T01:04:51.755ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24522024-12-19T01:04:51.811ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95747 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:51413 req_id = 2314827c-13cb-4ee1-ad9c-2df5675b6155 response_code = 204 uri = /switch-ports
24532024-12-19T01:04:51.811ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62511
24542024-12-19T01:04:51.811ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57925
24552024-12-19T01:04:51.811ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62511 req_id = 6a1cc9e6-c373-4c00-b856-c98ee2046cee response_code = 200 uri = /vpc-routes
24562024-12-19T01:04:51.812ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 331 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51413 req_id = f8a77141-0349-4781-8c09-30ae39947f3b response_code = 200 uri = /network-bootstore-config
24572024-12-19T01:04:51.812ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 331 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57925 req_id = d3e81856-eba1-4691-8333-a97b597a10ca response_code = 200 uri = /v2p
24582024-12-19T01:04:51.812ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 539 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40051 req_id = 55c8305c-6974-4d12-b498-9870cd0cfbb3 response_code = 204 uri = /vpc-routes
24592024-12-19T01:04:51.825ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42733
24602024-12-19T01:04:51.825ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42733 req_id = e31e030b-0492-45e6-bd67-367d07d7d099 response_code = 200 uri = /v2p
24612024-12-19T01:04:51.845ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44783
24622024-12-19T01:04:51.845ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24632024-12-19T01:04:51.887ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24642024-12-19T01:04:51.939ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94018 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:44783 req_id = c14283b1-cab6-4c10-95ee-7850d9be2911 response_code = 204 uri = /switch-ports
24652024-12-19T01:04:51.939ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59010
24662024-12-19T01:04:51.940ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
24672024-12-19T01:04:51.940ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 290 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44783 req_id = 494be668-b3d9-41f7-9ffe-1da380f28639 response_code = 200 uri = /network-bootstore-config
24682024-12-19T01:04:51.940ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 234 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62511 req_id = 8308f01a-b7bb-4715-beb6-a5464161b225 response_code = 204 uri = /eip-gateways
24692024-12-19T01:04:51.940ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24702024-12-19T01:04:51.949ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 387 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62511 req_id = 7b7318c7-3416-4cfc-8656-9ebe51e9635e response_code = 204 uri = /vpc-routes
24712024-12-19T01:04:51.977ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24722024-12-19T01:04:52.033ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93251 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59010 req_id = d51a80d6-334d-4258-8b12-42049d7fed0c response_code = 204 uri = /switch-ports
24732024-12-19T01:04:52.033ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59010 req_id = d34fdca4-15d6-4d8f-9cdb-ef07991ad5bc response_code = 200 uri = /network-bootstore-config
24742024-12-19T01:05:03.094ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:53821
24752024-12-19T01:05:03.094ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:53821 req_id = 79a1b775-6e0a-45a2-afd1-b8a78b147928 response_code = 200 uri = /baseboard
24762024-12-19T01:05:05.769ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8515 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = 7271854b-bfdd-45b5-a2b5-daf8056e6882 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
24772024-12-19T01:05:21.655ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40933
24782024-12-19T01:05:21.655ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40933 req_id = 1c66f4ea-25da-4162-b273-32640391256d response_code = 200 uri = /vpc-routes
24792024-12-19T01:05:21.714ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
24802024-12-19T01:05:21.714ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40933 req_id = 20cf115b-9a34-4f27-8c6a-d504f2b7d7e5 response_code = 204 uri = /eip-gateways
24812024-12-19T01:05:21.718ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37871
24822024-12-19T01:05:21.718ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 33 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37871 req_id = 3e5c4215-e167-475f-8bcb-7f4973b53db5 response_code = 200 uri = /vpc-routes
24832024-12-19T01:05:21.728ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 280 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40933 req_id = 3bfd8031-b7d4-486c-9f45-7b441281f592 response_code = 204 uri = /vpc-routes
24842024-12-19T01:05:21.773ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50954
24852024-12-19T01:05:21.773ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50954 req_id = d61a2096-0a4a-4df2-8b53-165244f31192 response_code = 200 uri = /vpc-routes
24862024-12-19T01:05:21.807ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
24872024-12-19T01:05:21.807ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:37871 req_id = d49cf695-8294-40bc-88dd-62237350dcc0 response_code = 204 uri = /eip-gateways
24882024-12-19T01:05:21.831ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 269 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:37871 req_id = 491024b1-db00-4ade-a966-42c3cb865326 response_code = 204 uri = /vpc-routes
24892024-12-19T01:05:21.859ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
24902024-12-19T01:05:21.859ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50954 req_id = bd928aff-46c4-414d-b196-d61a9c7e3336 response_code = 204 uri = /eip-gateways
24912024-12-19T01:05:21.871ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 264 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50954 req_id = 57304039-1026-4b1d-b77c-951109f47e9a response_code = 204 uri = /vpc-routes
24922024-12-19T01:05:21.922ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44127
24932024-12-19T01:05:21.923ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44127 req_id = 3ef6aaa9-bf5f-4144-aecb-92c5ab95f291 response_code = 200 uri = /v2p
24942024-12-19T01:05:21.952ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50944
24952024-12-19T01:05:21.952ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24962024-12-19T01:05:21.987ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24972024-12-19T01:05:22.049ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97169 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:50944 req_id = bea6bfef-2af8-4faa-8308-638522ad6dae response_code = 204 uri = /switch-ports
24982024-12-19T01:05:22.049ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35730
24992024-12-19T01:05:22.049ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60881
25002024-12-19T01:05:22.049ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49473
25012024-12-19T01:05:22.050ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 347 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50944 req_id = f0107f55-40af-4228-abf9-392a5a351cd1 response_code = 200 uri = /network-bootstore-config
25022024-12-19T01:05:22.050ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25032024-12-19T01:05:22.050ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 292 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60881 req_id = 065de90c-95b0-43d0-ac06-71d2f6972cea response_code = 200 uri = /v2p
25042024-12-19T01:05:22.050ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 261 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35730 req_id = 39a946a0-a29e-4917-9913-69cc3499843a response_code = 200 uri = /v2p
25052024-12-19T01:05:22.090ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25062024-12-19T01:05:22.095ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40611
25072024-12-19T01:05:22.147ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97405 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:49473 req_id = 21e0d32f-d03f-4046-9df1-4774e7de7a0a response_code = 204 uri = /switch-ports
25082024-12-19T01:05:22.147ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25092024-12-19T01:05:22.151ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 669 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49473 req_id = 4417c03f-f60d-457b-b86f-af1ca1ecdcff response_code = 200 uri = /network-bootstore-config
25102024-12-19T01:05:22.184ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25112024-12-19T01:05:22.238ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142835 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:40611 req_id = 50ff0f5c-b6c6-450a-b607-0a0e36f6c2bb response_code = 204 uri = /switch-ports
25122024-12-19T01:05:22.238ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40611 req_id = 16be3cfd-3276-4efa-8632-521560c583f2 response_code = 200 uri = /network-bootstore-config
25132024-12-19T01:05:33.126ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63375
25142024-12-19T01:05:33.126ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63375 req_id = 9147306d-76f3-435a-9b1a-8c68a44107a2 response_code = 200 uri = /baseboard
25152024-12-19T01:05:35.769ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8020 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = ff31be02-dda4-48b4-a68d-3ef4f5ced7d9 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
25162024-12-19T01:05:51.583ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42836
25172024-12-19T01:05:51.583ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42836 req_id = c6c64a68-2fee-43c4-8d3a-d54b05a1f662 response_code = 200 uri = /vpc-routes
25182024-12-19T01:05:51.609ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43373
25192024-12-19T01:05:51.609ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43373 req_id = dfee0cb2-6c33-4bc1-b0e3-f902ded29a9d response_code = 200 uri = /vpc-routes
25202024-12-19T01:05:51.632ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
25212024-12-19T01:05:51.632ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42836 req_id = 521643bb-1d53-46fc-8ced-ba40f23f5a66 response_code = 204 uri = /eip-gateways
25222024-12-19T01:05:51.640ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
25232024-12-19T01:05:51.640ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43373 req_id = 2c7acc6c-9d2f-4804-ad2d-f9a5ecd7ba2d response_code = 204 uri = /eip-gateways
25242024-12-19T01:05:51.642ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 211 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42836 req_id = 550525a2-2192-4dda-a9f5-8ca49630d9a9 response_code = 204 uri = /vpc-routes
25252024-12-19T01:05:51.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 217 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43373 req_id = 6f76cd4a-6b7d-4d2a-8910-61ab6b475ed4 response_code = 204 uri = /vpc-routes
25262024-12-19T01:05:51.694ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35155
25272024-12-19T01:05:51.694ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35155 req_id = 47c522db-2f4b-45fa-8a1b-4b2c14004559 response_code = 200 uri = /v2p
25282024-12-19T01:05:51.700ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44605
25292024-12-19T01:05:51.700ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25302024-12-19T01:05:51.737ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25312024-12-19T01:05:51.791ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90688 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:44605 req_id = 35da1d58-ce81-47bc-8c2d-2ce58e41952e response_code = 204 uri = /switch-ports
25322024-12-19T01:05:51.791ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57718
25332024-12-19T01:05:51.791ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43116
25342024-12-19T01:05:51.792ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 289 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44605 req_id = d9a1919d-4bcc-4ea6-88d4-a6400d04b134 response_code = 200 uri = /network-bootstore-config
25352024-12-19T01:05:51.792ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25362024-12-19T01:05:51.792ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 259 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57718 req_id = f879e2af-546e-4053-a7be-b10da738b4ef response_code = 200 uri = /v2p
25372024-12-19T01:05:51.803ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60831
25382024-12-19T01:05:51.803ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 168 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60831 req_id = ff1b6846-856d-4d90-a76a-d46a7b218f45 response_code = 200 uri = /vpc-routes
25392024-12-19T01:05:51.833ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25402024-12-19T01:05:51.847ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
25412024-12-19T01:05:51.848ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 665 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60831 req_id = 79d20984-6a62-4348-8d53-c5ab82fa0c21 response_code = 204 uri = /eip-gateways
25422024-12-19T01:05:51.849ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57560
25432024-12-19T01:05:51.849ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57560 req_id = 7336259a-4d85-4d01-ba74-55b5fe79a3d6 response_code = 200 uri = /v2p
25442024-12-19T01:05:51.859ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 947 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60831 req_id = bdadf74c-694d-480e-a58b-fa54eb023f25 response_code = 204 uri = /vpc-routes
25452024-12-19T01:05:51.890ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98762 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:43116 req_id = 518069de-ffff-422f-bd31-56b09b3f2039 response_code = 204 uri = /switch-ports
25462024-12-19T01:05:51.891ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43116 req_id = 6c667f25-c8ae-403f-bf87-144f2a981e28 response_code = 200 uri = /network-bootstore-config
25472024-12-19T01:05:51.903ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43374
25482024-12-19T01:05:51.903ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25492024-12-19T01:05:51.937ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25502024-12-19T01:05:51.990ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86833 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:43374 req_id = 235fb44c-2a41-41f3-8829-1539e213b191 response_code = 204 uri = /switch-ports
25512024-12-19T01:05:51.990ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43374 req_id = 0ba1d77d-b8b0-474c-82d3-e3f5269529ee response_code = 200 uri = /network-bootstore-config
25522024-12-19T01:06:03.160ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61110
25532024-12-19T01:06:03.160ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61110 req_id = b065870c-15e0-4fcd-b0d8-2e02b627a3cd response_code = 200 uri = /baseboard
25542024-12-19T01:06:05.769ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8259 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = 497e3a69-7e3b-410b-93de-b4b67f32e241 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
25552024-12-19T01:06:08.262ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:197
25562024-12-19T01:06:08.262ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
25572024-12-19T01:06:08.295ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1857, tv_nsec: 139416748 } time_to_next_cleanup = 599.99999562s
25582024-12-19T01:06:21.749ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39686
25592024-12-19T01:06:21.749ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39686 req_id = 0287f5eb-52db-44c0-98b8-ac3a1a810923 response_code = 200 uri = /vpc-routes
25602024-12-19T01:06:21.787ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47489
25612024-12-19T01:06:21.787ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47489 req_id = abce1d0a-99cb-4470-be68-652a0934a70d response_code = 200 uri = /vpc-routes
25622024-12-19T01:06:21.806ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
25632024-12-19T01:06:21.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39686 req_id = 391efc5a-03b3-405b-a1ac-df96c258e5ac response_code = 204 uri = /eip-gateways
25642024-12-19T01:06:21.816ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 214 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39686 req_id = f5a5305d-3251-421e-9c99-d26bf4312b6d response_code = 204 uri = /vpc-routes
25652024-12-19T01:06:21.834ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
25662024-12-19T01:06:21.834ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47489 req_id = 2c8577f7-3842-48a9-a518-27e2681af934 response_code = 204 uri = /eip-gateways
25672024-12-19T01:06:21.891ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43715
25682024-12-19T01:06:21.891ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43715 req_id = 2c46285a-0fe7-4a3c-84a6-ea75c9281e16 response_code = 200 uri = /v2p
25692024-12-19T01:06:21.900ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34753
25702024-12-19T01:06:21.900ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34753 req_id = d517cc14-4c2c-4fa8-9edb-609302a6e186 response_code = 200 uri = /vpc-routes
25712024-12-19T01:06:21.902ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 205 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47489 req_id = 49623f09-167e-41c5-9133-faeb4d744b07 response_code = 204 uri = /vpc-routes
25722024-12-19T01:06:21.915ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
25732024-12-19T01:06:21.915ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 52 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34753 req_id = e9f34041-0fc3-4704-9ad1-826369ddda85 response_code = 204 uri = /eip-gateways
25742024-12-19T01:06:21.961ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62008
25752024-12-19T01:06:21.961ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62008 req_id = 19dd7191-052f-4765-bd3c-7fccaa772c10 response_code = 200 uri = /v2p
25762024-12-19T01:06:21.965ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63018
25772024-12-19T01:06:21.965ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25782024-12-19T01:06:22.001ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25792024-12-19T01:06:22.060ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94254 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:63018 req_id = 4a3525ad-2cfa-487f-8a81-bf27bbe88e44 response_code = 204 uri = /switch-ports
25802024-12-19T01:06:22.060ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33362
25812024-12-19T01:06:22.060ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54654
25822024-12-19T01:06:22.060ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 275 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33362 req_id = 7b9e0c06-960c-4824-b789-8876b34a6ac4 response_code = 200 uri = /v2p
25832024-12-19T01:06:22.060ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25842024-12-19T01:06:22.060ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 323 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63018 req_id = ef8d6690-64dc-4be3-8829-1f447f6f7ab0 response_code = 200 uri = /network-bootstore-config
25852024-12-19T01:06:22.065ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5056 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34753 req_id = 797fccba-e6a5-43bc-bb20-e0ef849c27bc response_code = 204 uri = /vpc-routes
25862024-12-19T01:06:22.098ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60767
25872024-12-19T01:06:22.100ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25882024-12-19T01:06:22.156ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25892024-12-19T01:06:22.156ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96098 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:54654 req_id = f7d1ee51-085e-4d52-9b13-3ded5cfc94cb response_code = 204 uri = /switch-ports
25902024-12-19T01:06:22.160ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 330 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54654 req_id = 55c73113-c9b9-4bee-9482-e8a30b36e784 response_code = 200 uri = /network-bootstore-config
25912024-12-19T01:06:22.190ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25922024-12-19T01:06:22.245ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 146803 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:60767 req_id = 34e69288-c583-4a99-9df9-2e8b797493d0 response_code = 204 uri = /switch-ports
25932024-12-19T01:06:22.245ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60767 req_id = 7a638351-9f81-48f5-b2c4-5266f1fe1feb response_code = 200 uri = /network-bootstore-config
25942024-12-19T01:06:23.750ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
25952024-12-19T01:06:33.193ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:57645
25962024-12-19T01:06:33.193ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:57645 req_id = 67392e36-3ee3-4efe-bbb4-e2caa545c8e1 response_code = 200 uri = /baseboard
25972024-12-19T01:06:35.767ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7330 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = 376cff70-4f8c-4788-952a-f8ac52293484 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
25982024-12-19T01:06:51.628ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52426
25992024-12-19T01:06:51.628ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52426 req_id = c1ac7eb1-47a0-49f2-ab09-6a76bfcf3057 response_code = 200 uri = /vpc-routes
26002024-12-19T01:06:51.631ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58461
26012024-12-19T01:06:51.632ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 32 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58461 req_id = c00d0548-f824-4cd4-b8bd-4a2e3a07d34b response_code = 200 uri = /vpc-routes
26022024-12-19T01:06:51.678ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64103
26032024-12-19T01:06:51.678ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 183 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64103 req_id = 37740431-e141-44a5-aaa0-56b810ff6248 response_code = 200 uri = /v2p
26042024-12-19T01:06:51.686ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
26052024-12-19T01:06:51.686ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58461 req_id = da259fb5-d496-4527-89e8-c472241fbab3 response_code = 204 uri = /eip-gateways
26062024-12-19T01:06:51.694ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
26072024-12-19T01:06:51.694ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52426 req_id = a44be53b-866d-49ad-b728-90c00da1fe3f response_code = 204 uri = /eip-gateways
26082024-12-19T01:06:51.698ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 292 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58461 req_id = 983c33d2-2f25-467e-9f0b-857cbbdb9280 response_code = 204 uri = /vpc-routes
26092024-12-19T01:06:51.703ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54043
26102024-12-19T01:06:51.704ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26112024-12-19T01:06:51.753ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26122024-12-19T01:06:51.830ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126411 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54043 req_id = 00cfbc74-4474-407d-9f72-fb43741c0e51 response_code = 204 uri = /switch-ports
26132024-12-19T01:06:51.830ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40082
26142024-12-19T01:06:51.830ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57469
26152024-12-19T01:06:51.831ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38770
26162024-12-19T01:06:51.831ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 327 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54043 req_id = ed21e1c5-3ad7-4d4b-9de1-dd79a2fed7ae response_code = 200 uri = /network-bootstore-config
26172024-12-19T01:06:51.831ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 535 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52426 req_id = bc947aa0-7840-41dc-b2be-d0416c42f172 response_code = 204 uri = /vpc-routes
26182024-12-19T01:06:51.831ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26192024-12-19T01:06:51.831ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 286 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38770 req_id = 61db352c-8b44-480c-93ef-b36c77d69634 response_code = 200 uri = /v2p
26202024-12-19T01:06:51.831ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57469 req_id = 02f30dbf-8711-4582-91e3-42f7182b73ff response_code = 200 uri = /vpc-routes
26212024-12-19T01:06:51.837ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57568
26222024-12-19T01:06:51.837ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 308 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57568 req_id = 5bfa3313-f0c2-452a-993e-30c98cd36057 response_code = 200 uri = /v2p
26232024-12-19T01:06:51.882ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26242024-12-19T01:06:51.891ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
26252024-12-19T01:06:51.891ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 204 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57469 req_id = beb8e26d-bd55-4234-a228-f5c8a9aa776d response_code = 204 uri = /eip-gateways
26262024-12-19T01:06:51.905ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 985 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57469 req_id = b1386efe-2c53-4707-ac49-3d150123e40f response_code = 204 uri = /vpc-routes
26272024-12-19T01:06:51.913ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62245
26282024-12-19T01:06:51.960ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26292024-12-19T01:06:51.960ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 128984 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40082 req_id = a9fc29f4-2ed9-4201-a6cf-3ecacbd47781 response_code = 204 uri = /switch-ports
26302024-12-19T01:06:51.964ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 777 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40082 req_id = 8a235f00-1cfb-405d-808e-dffde590e49b response_code = 200 uri = /network-bootstore-config
26312024-12-19T01:06:51.996ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26322024-12-19T01:06:52.045ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131819 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62245 req_id = eab3a234-e418-4518-b235-d20576c6fe48 response_code = 204 uri = /switch-ports
26332024-12-19T01:06:52.045ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62245 req_id = 98cd90c3-e718-4110-8a84-1f505526b267 response_code = 200 uri = /network-bootstore-config
26342024-12-19T01:07:03.227ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:53981
26352024-12-19T01:07:03.227ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 30 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:53981 req_id = 56da62dc-7990-4a00-93e6-c0c9cbbb11ab response_code = 200 uri = /baseboard
26362024-12-19T01:07:05.768ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7992 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = 17fb4fc8-1ac8-4572-b2a4-efc5ad420890 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
26372024-12-19T01:07:21.632ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44424
26382024-12-19T01:07:21.632ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44424 req_id = 9f0664a0-34fb-48ba-b23d-8e0d8655c069 response_code = 200 uri = /vpc-routes
26392024-12-19T01:07:21.748ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
26402024-12-19T01:07:21.748ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44424 req_id = b0aa8401-6681-4207-a89e-dc07fc746654 response_code = 204 uri = /eip-gateways
26412024-12-19T01:07:21.759ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 353 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44424 req_id = 0cbdd8bc-1b8f-4056-812f-9e47d3bb03e1 response_code = 204 uri = /vpc-routes
26422024-12-19T01:07:21.771ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42056
26432024-12-19T01:07:21.771ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42056 req_id = e71190c4-5fd4-4cfb-97f4-7b7af14dc16b response_code = 200 uri = /vpc-routes
26442024-12-19T01:07:21.831ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
26452024-12-19T01:07:21.831ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42056 req_id = e96fa5db-1ff6-4095-8687-83ea0782f533 response_code = 204 uri = /eip-gateways
26462024-12-19T01:07:21.846ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 264 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42056 req_id = d94533bd-7079-4740-892b-6d470ae317fb response_code = 204 uri = /vpc-routes
26472024-12-19T01:07:21.934ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39124
26482024-12-19T01:07:21.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39124 req_id = 67a71817-4c97-425d-a33d-ec4da7843bc8 response_code = 200 uri = /v2p
26492024-12-19T01:07:21.936ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56969
26502024-12-19T01:07:21.936ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26512024-12-19T01:07:21.971ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26522024-12-19T01:07:22.027ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90879 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:56969 req_id = 43201335-6f4b-4222-a1a2-d80879a31933 response_code = 204 uri = /switch-ports
26532024-12-19T01:07:22.027ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46411
26542024-12-19T01:07:22.027ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53700
26552024-12-19T01:07:22.027ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53700 req_id = 7bf4be0e-cab9-40c5-aba1-3ea0a0dc6634 response_code = 200 uri = /vpc-routes
26562024-12-19T01:07:22.027ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 230 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46411 req_id = ec1c97e1-c565-4074-8f87-90c7305d2b9a response_code = 200 uri = /v2p
26572024-12-19T01:07:22.027ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 243 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56969 req_id = 1e6b1f61-10de-4872-96bd-5207f530ada8 response_code = 200 uri = /network-bootstore-config
26582024-12-19T01:07:22.036ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33498
26592024-12-19T01:07:22.037ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26602024-12-19T01:07:22.081ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26612024-12-19T01:07:22.146ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109088 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:33498 req_id = 4b2ae27a-9f25-47a4-b5cd-5b96088867ae response_code = 204 uri = /switch-ports
26622024-12-19T01:07:22.146ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
26632024-12-19T01:07:22.146ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53700 req_id = 615d3141-79ab-423f-80f0-c0e60d3831b9 response_code = 204 uri = /eip-gateways
26642024-12-19T01:07:22.146ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33498 req_id = 89ed2d78-ab33-49e0-b6cb-f7dfdbbaa13d response_code = 200 uri = /network-bootstore-config
26652024-12-19T01:07:22.174ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61583
26662024-12-19T01:07:22.174ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61583 req_id = 8a650802-5ced-4a8f-9832-de97c7a2ae79 response_code = 200 uri = /v2p
26672024-12-19T01:07:22.182ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 294 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53700 req_id = d82d9b86-578d-4631-81e8-75555df16624 response_code = 204 uri = /vpc-routes
26682024-12-19T01:07:22.190ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44031
26692024-12-19T01:07:22.190ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26702024-12-19T01:07:22.226ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26712024-12-19T01:07:22.278ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88179 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:44031 req_id = c993914b-88df-40eb-b9e4-2bf843d61325 response_code = 204 uri = /switch-ports
26722024-12-19T01:07:22.279ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44031 req_id = 9a7dcc79-a732-4ba5-b09a-fbb98f115fb1 response_code = 200 uri = /network-bootstore-config
26732024-12-19T01:07:33.260ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:57737
26742024-12-19T01:07:33.260ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:57737 req_id = cab82b3b-72d5-46c3-9589-8ecc1937697d response_code = 200 uri = /baseboard
26752024-12-19T01:07:35.770ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8701 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = edc8fbd8-fc59-4083-ac8d-55670acd7c5e response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
26762024-12-19T01:07:51.568ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34729
26772024-12-19T01:07:51.568ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34729 req_id = 55b37e38-0dd8-4b6b-99c8-a664a7729dea response_code = 200 uri = /vpc-routes
26782024-12-19T01:07:51.642ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
26792024-12-19T01:07:51.642ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34729 req_id = 192382da-4fe8-4736-8084-3697e7eddffe response_code = 204 uri = /eip-gateways
26802024-12-19T01:07:51.655ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 346 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34729 req_id = 0780f1d4-5970-4fac-9c6b-fb053e0d7f29 response_code = 204 uri = /vpc-routes
26812024-12-19T01:07:51.658ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64820
26822024-12-19T01:07:51.658ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64820 req_id = 0b2cf86c-718a-4529-aecf-ab1a59a1c007 response_code = 200 uri = /vpc-routes
26832024-12-19T01:07:51.717ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
26842024-12-19T01:07:51.717ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64820 req_id = 8b562e94-083d-46a8-8e1e-89f6d7b59478 response_code = 204 uri = /eip-gateways
26852024-12-19T01:07:51.719ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38894
26862024-12-19T01:07:51.719ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26872024-12-19T01:07:51.775ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26882024-12-19T01:07:51.865ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 146085 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:38894 req_id = ac65b9f7-52fc-4c87-8789-ab4c4d11c778 response_code = 204 uri = /switch-ports
26892024-12-19T01:07:51.865ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61721
26902024-12-19T01:07:51.866ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37105
26912024-12-19T01:07:51.866ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36035
26922024-12-19T01:07:51.866ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38306
26932024-12-19T01:07:51.866ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 347 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38894 req_id = 9c7f94cc-a376-4b0f-b9f1-2d2d10b69881 response_code = 200 uri = /network-bootstore-config
26942024-12-19T01:07:51.866ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39582
26952024-12-19T01:07:51.866ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 498 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64820 req_id = 7a8bbce9-8521-429f-9717-d3f7ab1a175e response_code = 204 uri = /vpc-routes
26962024-12-19T01:07:51.866ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36035 req_id = 1facb874-f5c2-4373-ae7f-079913a111f6 response_code = 200 uri = /vpc-routes
26972024-12-19T01:07:51.866ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26982024-12-19T01:07:51.866ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 340 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37105 req_id = 76ffbe0a-5ae0-422c-9b34-22898bdf5ea5 response_code = 200 uri = /v2p
26992024-12-19T01:07:51.866ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 304 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38306 req_id = b9d47147-4d3d-4a2e-9d92-f12dff7411ab response_code = 200 uri = /v2p
27002024-12-19T01:07:51.866ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 294 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39582 req_id = 17ede0c3-145d-4253-ba98-0434200068a7 response_code = 200 uri = /v2p
27012024-12-19T01:07:51.900ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58819
27022024-12-19T01:07:51.921ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
27032024-12-19T01:07:51.926ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
27042024-12-19T01:07:51.926ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 855 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36035 req_id = e89ffd32-e7c9-43fd-9487-6ceccad3980e response_code = 204 uri = /eip-gateways
27052024-12-19T01:07:51.941ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 812 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36035 req_id = 444cb204-c347-4988-93be-4e1dc29b9106 response_code = 204 uri = /vpc-routes
27062024-12-19T01:07:51.983ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 116997 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:61721 req_id = 10fd49bc-49b4-49a8-87c6-2cb8cdb575e2 response_code = 204 uri = /switch-ports
27072024-12-19T01:07:51.983ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
27082024-12-19T01:07:51.988ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5178 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61721 req_id = 5f7e3166-a27e-4294-8320-d13dcfc16394 response_code = 200 uri = /network-bootstore-config
27092024-12-19T01:07:52.022ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
27102024-12-19T01:07:52.075ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 175297 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:58819 req_id = 9d1c25fb-4995-4ebe-8eaa-88faafa4ec20 response_code = 204 uri = /switch-ports
27112024-12-19T01:07:52.076ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58819 req_id = 7f8be8fd-4a7a-45cc-9c0b-a6c788ea42bf response_code = 200 uri = /network-bootstore-config
27122024-12-19T01:08:03.294ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:38970
27132024-12-19T01:08:03.294ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:38970 req_id = 66f7d70e-8500-4ed4-94cc-de7fa45179e8 response_code = 200 uri = /baseboard
27142024-12-19T01:08:05.771ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9473 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = f4aa0ea0-3ce0-4915-8b73-7a34fbe64d1a response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
27152024-12-19T01:08:21.809ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51199
27162024-12-19T01:08:21.809ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51199 req_id = d53f048d-724d-433d-bd65-d37934634c91 response_code = 200 uri = /vpc-routes
27172024-12-19T01:08:21.864ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56835
27182024-12-19T01:08:21.864ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56835 req_id = 7f828cf0-9aa1-4d25-9e37-52ece4fb891d response_code = 200 uri = /vpc-routes
27192024-12-19T01:08:21.869ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
27202024-12-19T01:08:21.869ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51199 req_id = 23295367-8762-4b4d-8462-5d7e046b094f response_code = 204 uri = /eip-gateways
27212024-12-19T01:08:21.896ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42176
27222024-12-19T01:08:21.896ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42176 req_id = 8404d3bd-afe2-4a72-be1c-debf7ef6807d response_code = 200 uri = /vpc-routes
27232024-12-19T01:08:21.945ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
27242024-12-19T01:08:21.945ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56835 req_id = 52103d87-9919-43b5-94d4-08bf0e51a683 response_code = 204 uri = /eip-gateways
27252024-12-19T01:08:21.958ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 436 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56835 req_id = 530d26dc-4472-478e-b3d2-9c88a437189d response_code = 204 uri = /vpc-routes
27262024-12-19T01:08:21.977ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61310
27272024-12-19T01:08:21.977ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
27282024-12-19T01:08:21.977ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
27292024-12-19T01:08:21.977ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
27302024-12-19T01:08:21.978ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
27312024-12-19T01:08:21.993ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27322024-12-19T01:08:21.993ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a6644750-441b-4297-9acc-8dd23a71b4cb
27332024-12-19T01:08:21.999ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
27342024-12-19T01:08:21.999ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 772 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42176 req_id = ea7b2e75-e1b8-42e6-8e8f-a42f80e734aa response_code = 204 uri = /eip-gateways
27352024-12-19T01:08:22.009ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 403 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42176 req_id = 7634f457-075a-45fd-816f-1af898f3affa response_code = 204 uri = /vpc-routes
27362024-12-19T01:08:22.023ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54799
27372024-12-19T01:08:22.023ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54799 req_id = 6cfd61f8-6dcd-421d-a26b-fbfad4592369 response_code = 200 uri = /v2p
27382024-12-19T01:08:22.024ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27392024-12-19T01:08:22.024ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa
27402024-12-19T01:08:22.031ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 610 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51199 req_id = d604684f-1bce-4d09-a3c2-4101a8ebd446 response_code = 204 uri = /vpc-routes
27412024-12-19T01:08:22.054ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27422024-12-19T01:08:22.055ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_93eb8a71-6519-482e-a366-b15e30b4605e
27432024-12-19T01:08:22.063ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41125
27442024-12-19T01:08:22.063ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
27452024-12-19T01:08:22.082ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27462024-12-19T01:08:22.083ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_98a100ce-cd82-4209-8530-7b48ca829744
27472024-12-19T01:08:22.097ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40554
27482024-12-19T01:08:22.106ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64996
27492024-12-19T01:08:22.106ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
27502024-12-19T01:08:22.110ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27512024-12-19T01:08:22.110ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6
27522024-12-19T01:08:22.115ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59857
27532024-12-19T01:08:22.116ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4638 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40554 req_id = 43437f90-872d-4b84-b4f3-8d9fb74d983f response_code = 200 uri = /v2p
27542024-12-19T01:08:22.116ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 346 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59857 req_id = 9d5ae035-29ea-487a-b64d-365ca8b7d3c1 response_code = 200 uri = /v2p
27552024-12-19T01:08:22.119ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53558
27562024-12-19T01:08:22.129ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
27572024-12-19T01:08:22.129ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
27582024-12-19T01:08:22.129ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
27592024-12-19T01:08:22.129ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
27602024-12-19T01:08:22.134ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
27612024-12-19T01:08:22.139ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34462
27622024-12-19T01:08:22.144ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27632024-12-19T01:08:22.144ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb
27642024-12-19T01:08:22.159ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27652024-12-19T01:08:22.159ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a6644750-441b-4297-9acc-8dd23a71b4cb
27662024-12-19T01:08:22.165ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59693
27672024-12-19T01:08:22.165ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 5b97cf45-6db1-45ed-9f7e-111d132f7ed4, kind: Service { id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 }, name: Name("external-dns-2a880a2e-2471-4fd8-9ad5-3e0c3fbde317"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 253])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d21d0095-8e84-486f-b3a4-f9dca167b9d5, kind: Service { id: 679d67b5-9925-49ac-b78d-613e816394dc }, name: Name("external-dns-679d67b5-9925-49ac-b78d-613e816394dc"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 205])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 82094354-655d-415c-9763-28731eb50642, kind: Service { id: e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 }, name: Name("nexus-e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 60])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5c3ad41-155b-469b-bb3f-412814b9d0ff, kind: Service { id: 1176f008-4a9d-4835-a5aa-1a86ec3c619c }, name: Name("nexus-1176f008-4a9d-4835-a5aa-1a86ec3c619c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 188])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ef89a8b8-3cc5-4637-aee2-3b98f04f51be, kind: Service { id: 1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 }, name: Name("nexus-1d1e5635-dd2b-4bf7-8ace-6cb3ea634995"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 75])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27682024-12-19T01:08:22.165ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27692024-12-19T01:08:22.165ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27702024-12-19T01:08:22.165ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27712024-12-19T01:08:22.165ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
27722024-12-19T01:08:22.165ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27732024-12-19T01:08:22.165ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27742024-12-19T01:08:22.165ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 409 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:59693 req_id = 1b7af96c-49a8-412d-ae99-a006a5ab46a1 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27752024-12-19T01:08:22.175ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
27762024-12-19T01:08:22.175ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/config/omicron-zones.json file = common/src/ledger.rs:177
27772024-12-19T01:08:22.184ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39275
27782024-12-19T01:08:22.184ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
27792024-12-19T01:08:22.189ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27802024-12-19T01:08:22.189ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_f4009784-7bde-44ff-854e-49d614e9e682
27812024-12-19T01:08:22.195ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 5b97cf45-6db1-45ed-9f7e-111d132f7ed4, kind: Service { id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 }, name: Name("external-dns-2a880a2e-2471-4fd8-9ad5-3e0c3fbde317"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 253])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d21d0095-8e84-486f-b3a4-f9dca167b9d5, kind: Service { id: 679d67b5-9925-49ac-b78d-613e816394dc }, name: Name("external-dns-679d67b5-9925-49ac-b78d-613e816394dc"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 205])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 82094354-655d-415c-9763-28731eb50642, kind: Service { id: e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 }, name: Name("nexus-e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 60])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5c3ad41-155b-469b-bb3f-412814b9d0ff, kind: Service { id: 1176f008-4a9d-4835-a5aa-1a86ec3c619c }, name: Name("nexus-1176f008-4a9d-4835-a5aa-1a86ec3c619c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 188])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ef89a8b8-3cc5-4637-aee2-3b98f04f51be, kind: Service { id: 1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 }, name: Name("nexus-1d1e5635-dd2b-4bf7-8ace-6cb3ea634995"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 75])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27822024-12-19T01:08:22.195ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27832024-12-19T01:08:22.195ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27842024-12-19T01:08:22.195ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27852024-12-19T01:08:22.195ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131923 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:41125 req_id = be632602-0344-46f9-b59e-94d28e2721c5 response_code = 204 uri = /switch-ports
27862024-12-19T01:08:22.195ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
27872024-12-19T01:08:22.195ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27882024-12-19T01:08:22.195ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27892024-12-19T01:08:22.195ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5099 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39275 req_id = b68a86cf-79e9-425c-a8ae-4582240985a5 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27902024-12-19T01:08:22.195ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/config/omicron-zones.json file = common/src/ledger.rs:177
27912024-12-19T01:08:22.196ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 325 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41125 req_id = c737c5f2-3f19-43bf-889f-d32dccae3cc7 response_code = 200 uri = /network-bootstore-config
27922024-12-19T01:08:22.206ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27932024-12-19T01:08:22.206ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa
27942024-12-19T01:08:22.210ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53480
27952024-12-19T01:08:22.221ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a6644750-441b-4297-9acc-8dd23a71b4cb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27962024-12-19T01:08:22.221ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a6644750-441b-4297-9acc-8dd23a71b4cb
27972024-12-19T01:08:22.229ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
27982024-12-19T01:08:22.239ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27992024-12-19T01:08:22.239ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_1025d0d6-c165-4f43-bef0-cb2eeb04b8c2
28002024-12-19T01:08:22.260ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ac442b57-464c-4650-8ad2-3f5b4744c9fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28012024-12-19T01:08:22.260ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_ac442b57-464c-4650-8ad2-3f5b4744c9fa
28022024-12-19T01:08:22.273ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28032024-12-19T01:08:22.273ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_93eb8a71-6519-482e-a366-b15e30b4605e
28042024-12-19T01:08:22.289ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28052024-12-19T01:08:22.289ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_499e53d3-d67c-44b6-9c03-b66767baed7e
28062024-12-19T01:08:22.294ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
28072024-12-19T01:08:22.295ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43660
28082024-12-19T01:08:22.302ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 152645 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:34462 req_id = 38e89899-e459-48aa-be38-0e12776874a2 response_code = 204 uri = /switch-ports
28092024-12-19T01:08:22.303ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 5b97cf45-6db1-45ed-9f7e-111d132f7ed4, kind: Service { id: 2a880a2e-2471-4fd8-9ad5-3e0c3fbde317 }, name: Name("external-dns-2a880a2e-2471-4fd8-9ad5-3e0c3fbde317"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 253])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d21d0095-8e84-486f-b3a4-f9dca167b9d5, kind: Service { id: 679d67b5-9925-49ac-b78d-613e816394dc }, name: Name("external-dns-679d67b5-9925-49ac-b78d-613e816394dc"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 205])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 82094354-655d-415c-9763-28731eb50642, kind: Service { id: e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9 }, name: Name("nexus-e1a83ba4-ec9f-4cc1-ae29-0ab5f6b211f9"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 60])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a5c3ad41-155b-469b-bb3f-412814b9d0ff, kind: Service { id: 1176f008-4a9d-4835-a5aa-1a86ec3c619c }, name: Name("nexus-1176f008-4a9d-4835-a5aa-1a86ec3c619c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 188])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ef89a8b8-3cc5-4637-aee2-3b98f04f51be, kind: Service { id: 1d1e5635-dd2b-4bf7-8ace-6cb3ea634995 }, name: Name("nexus-1d1e5635-dd2b-4bf7-8ace-6cb3ea634995"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 75])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
28102024-12-19T01:08:22.303ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
28112024-12-19T01:08:22.303ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28122024-12-19T01:08:22.303ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28132024-12-19T01:08:22.303ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
28142024-12-19T01:08:22.303ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
28152024-12-19T01:08:22.303ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28162024-12-19T01:08:22.303ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1075 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43660 req_id = e029fb33-3a01-4f62-b234-608ce3972cfb response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
28172024-12-19T01:08:22.303ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 303 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34462 req_id = 8d5dbe2d-5bf5-410d-98be-685837d65802 response_code = 200 uri = /network-bootstore-config
28182024-12-19T01:08:22.305ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28192024-12-19T01:08:22.305ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_98a100ce-cd82-4209-8530-7b48ca829744
28202024-12-19T01:08:22.318ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 93eb8a71-6519-482e-a366-b15e30b4605e (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28212024-12-19T01:08:22.318ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_93eb8a71-6519-482e-a366-b15e30b4605e
28222024-12-19T01:08:22.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 345856 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61310 req_id = ebe7c3a3-8b82-4ce2-82f6-ad3b6c7d9b05 response_code = 200 uri = /inventory
28232024-12-19T01:08:22.335ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
28242024-12-19T01:08:22.343ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 98a100ce-cd82-4209-8530-7b48ca829744 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28252024-12-19T01:08:22.344ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_98a100ce-cd82-4209-8530-7b48ca829744
28262024-12-19T01:08:22.358ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28272024-12-19T01:08:22.358ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6
28282024-12-19T01:08:22.374ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a98ada5c-5d21-42de-9ddc-c9598d5076c6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28292024-12-19T01:08:22.374ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a98ada5c-5d21-42de-9ddc-c9598d5076c6
28302024-12-19T01:08:22.390ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28312024-12-19T01:08:22.390ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb
28322024-12-19T01:08:22.408ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197472 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53480 req_id = 663886c5-cde5-4cb2-a2fc-935f367d69d3 response_code = 204 uri = /switch-ports
28332024-12-19T01:08:22.413ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 745 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53480 req_id = f7142f6d-d1a6-4430-817a-b4af42038ffd response_code = 200 uri = /network-bootstore-config
28342024-12-19T01:08:22.417ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28352024-12-19T01:08:22.417ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_f4009784-7bde-44ff-854e-49d614e9e682
28362024-12-19T01:08:22.429ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28372024-12-19T01:08:22.429ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb
28382024-12-19T01:08:22.448ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28392024-12-19T01:08:22.448ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_1025d0d6-c165-4f43-bef0-cb2eeb04b8c2
28402024-12-19T01:08:22.463ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: f4009784-7bde-44ff-854e-49d614e9e682 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28412024-12-19T01:08:22.463ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_f4009784-7bde-44ff-854e-49d614e9e682
28422024-12-19T01:08:22.484ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 1025d0d6-c165-4f43-bef0-cb2eeb04b8c2 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28432024-12-19T01:08:22.484ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_1025d0d6-c165-4f43-bef0-cb2eeb04b8c2
28442024-12-19T01:08:22.497ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28452024-12-19T01:08:22.497ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_499e53d3-d67c-44b6-9c03-b66767baed7e
28462024-12-19T01:08:22.513ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 499e53d3-d67c-44b6-9c03-b66767baed7e (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28472024-12-19T01:08:22.514ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_499e53d3-d67c-44b6-9c03-b66767baed7e
28482024-12-19T01:08:22.518ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 406935 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64996 req_id = 15de92ec-5b82-4708-ab67-03144c725c85 response_code = 200 uri = /inventory
28492024-12-19T01:08:22.526ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 406781 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53558 req_id = 13a48cc7-cb4a-44db-af5e-39ae9c566156 response_code = 200 uri = /inventory
28502024-12-19T01:08:33.327ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:49880
28512024-12-19T01:08:33.327ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:49880 req_id = 5c6bc0fa-5ff2-4578-99d2-058d4b2c0cd0 response_code = 200 uri = /baseboard
28522024-12-19T01:08:35.770ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8322 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = 33b4ed9c-13fa-450b-88ba-f66f7eb3e908 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
28532024-12-19T01:08:51.664ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41438
28542024-12-19T01:08:51.664ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41438 req_id = be5f1314-3745-4e85-86fe-22a668a13dde response_code = 200 uri = /vpc-routes
28552024-12-19T01:08:51.686ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
28562024-12-19T01:08:51.686ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41438 req_id = 1e742f02-8063-4073-80f0-a95806cc82d9 response_code = 204 uri = /eip-gateways
28572024-12-19T01:08:51.695ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 314 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41438 req_id = af5696d1-1998-43f4-869f-d7efef8dd4e3 response_code = 204 uri = /vpc-routes
28582024-12-19T01:08:51.714ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57580
28592024-12-19T01:08:51.714ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57580 req_id = e9ee68d3-3e33-4f97-b972-ff7e9d2b2285 response_code = 200 uri = /v2p
28602024-12-19T01:08:51.729ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62390
28612024-12-19T01:08:51.729ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 32 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62390 req_id = 9af2f529-f8dd-4529-818d-d2cc38affbda response_code = 200 uri = /vpc-routes
28622024-12-19T01:08:51.732ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34567
28632024-12-19T01:08:51.732ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34567 req_id = 54aaaeaf-e67b-4782-975b-19d21db85e9a response_code = 200 uri = /v2p
28642024-12-19T01:08:51.755ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37048
28652024-12-19T01:08:51.755ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
28662024-12-19T01:08:51.795ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
28672024-12-19T01:08:51.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97151 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:37048 req_id = 95ab82d7-f561-4fe2-be55-6590e35b6148 response_code = 204 uri = /switch-ports
28682024-12-19T01:08:51.853ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63192
28692024-12-19T01:08:51.853ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
28702024-12-19T01:08:51.853ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 252 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62390 req_id = 59d3ac08-1a6f-49b7-b5d4-67235679335a response_code = 204 uri = /eip-gateways
28712024-12-19T01:08:51.853ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 272 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37048 req_id = e92df47f-8e26-46a8-8421-83ce3fb13021 response_code = 200 uri = /network-bootstore-config
28722024-12-19T01:08:51.853ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 177 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63192 req_id = 0a8b84d0-58f5-4ac7-915f-1b3fa184d301 response_code = 200 uri = /vpc-routes
28732024-12-19T01:08:51.861ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56575
28742024-12-19T01:08:51.861ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56575 req_id = c3db87eb-b9ca-4148-a59f-953521ff75dd response_code = 200 uri = /v2p
28752024-12-19T01:08:51.867ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 315 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62390 req_id = 867ca4bc-4bc5-4a18-acaf-b388c611fbf2 response_code = 204 uri = /vpc-routes
28762024-12-19T01:08:51.873ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34672
28772024-12-19T01:08:51.873ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
28782024-12-19T01:08:51.912ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
28792024-12-19T01:08:51.964ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90802 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:34672 req_id = 0cebeb73-a343-4ecf-85c7-498d1944fc06 response_code = 204 uri = /switch-ports
28802024-12-19T01:08:51.964ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35286
28812024-12-19T01:08:51.965ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
28822024-12-19T01:08:51.965ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
28832024-12-19T01:08:51.965ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 288 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34672 req_id = 4bb51229-f3f2-490f-8178-fce530dae7f0 response_code = 200 uri = /network-bootstore-config
28842024-12-19T01:08:51.965ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 215 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63192 req_id = 3826d70a-ee4a-4f9d-a9de-539c465b5b97 response_code = 204 uri = /eip-gateways
28852024-12-19T01:08:51.975ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 424 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63192 req_id = 2bd5a92e-4e4d-4e16-809d-1d2fc0980d45 response_code = 204 uri = /vpc-routes
28862024-12-19T01:08:51.999ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
28872024-12-19T01:08:52.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92509 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:35286 req_id = 632fc424-7dda-4315-ac4a-cf86a7cd7669 response_code = 204 uri = /switch-ports
28882024-12-19T01:08:52.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 171 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35286 req_id = 2e308905-5ccd-4229-a1fa-26b40da49113 response_code = 200 uri = /network-bootstore-config
28892024-12-19T01:09:03.360ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:42848
28902024-12-19T01:09:03.360ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:42848 req_id = e73fc2fb-1617-4b42-8e1a-54d04cecb439 response_code = 200 uri = /baseboard
28912024-12-19T01:09:05.770ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8433 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = 3ef7c25e-3a5e-45b7-93d7-aae618f39775 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
28922024-12-19T01:09:21.690ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44680
28932024-12-19T01:09:21.690ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44680 req_id = 65f0def2-4f77-4ef2-8a50-b86be1a0204a response_code = 200 uri = /vpc-routes
28942024-12-19T01:09:21.762ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56671
28952024-12-19T01:09:21.762ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56671 req_id = 08ed6255-6176-4302-bfe5-59bb96cb9f7b response_code = 200 uri = /vpc-routes
28962024-12-19T01:09:21.791ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63535
28972024-12-19T01:09:21.791ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 185 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63535 req_id = f23f71c5-ef14-4eef-a288-e3c89e27b2cc response_code = 200 uri = /v2p
28982024-12-19T01:09:21.797ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
28992024-12-19T01:09:21.797ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44680 req_id = 3cede885-6a35-48fe-813b-eb0f404a50af response_code = 204 uri = /eip-gateways
29002024-12-19T01:09:21.816ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 529 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44680 req_id = 2e8ebdf1-181a-42e1-8d2e-133d64f1a40a response_code = 204 uri = /vpc-routes
29012024-12-19T01:09:21.893ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
29022024-12-19T01:09:21.893ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56671 req_id = 3b430e1f-f5b2-4a04-93da-c681a741f43d response_code = 204 uri = /eip-gateways
29032024-12-19T01:09:21.909ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 253 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56671 req_id = 06e51342-fda1-4067-a500-3a56f4347fc3 response_code = 204 uri = /vpc-routes
29042024-12-19T01:09:21.914ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42279
29052024-12-19T01:09:21.914ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42279 req_id = 39e3700f-d94b-44f7-865a-d74444fefc9a response_code = 200 uri = /vpc-routes
29062024-12-19T01:09:21.958ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65358
29072024-12-19T01:09:21.959ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
29082024-12-19T01:09:22.013ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
29092024-12-19T01:09:22.086ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127479 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:65358 req_id = 7690d6ad-d3b0-47d0-8bb3-ded349449429 response_code = 204 uri = /switch-ports
29102024-12-19T01:09:22.086ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33752
29112024-12-19T01:09:22.086ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
29122024-12-19T01:09:22.086ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42279 req_id = 081fcc52-488f-4ce3-bb8d-3a674ba72bdc response_code = 204 uri = /eip-gateways
29132024-12-19T01:09:22.086ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54584
29142024-12-19T01:09:22.087ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 296 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65358 req_id = 9c6cc305-0213-4bf4-8329-275849e7e856 response_code = 200 uri = /network-bootstore-config
29152024-12-19T01:09:22.087ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 202 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54584 req_id = c5710115-b0c9-4f95-9909-8f8daf827efe response_code = 200 uri = /v2p
29162024-12-19T01:09:22.087ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
29172024-12-19T01:09:22.119ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63062
29182024-12-19T01:09:22.119ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63062 req_id = ad528f07-6090-4866-bb71-465f55520fef response_code = 200 uri = /v2p
29192024-12-19T01:09:22.123ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
29202024-12-19T01:09:22.128ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 298 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42279 req_id = 06d75672-ed3f-45f6-8f7d-c233abe89a4a response_code = 204 uri = /vpc-routes
29212024-12-19T01:09:22.175ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88762 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:33752 req_id = 11a89db4-b218-4e31-88d3-bd179f35a9a3 response_code = 204 uri = /switch-ports
29222024-12-19T01:09:22.176ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33752 req_id = ba8721e8-47ab-430d-b254-552248f9f552 response_code = 200 uri = /network-bootstore-config
29232024-12-19T01:09:22.187ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61333
29242024-12-19T01:09:22.188ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
29252024-12-19T01:09:22.223ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
29262024-12-19T01:09:22.279ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91041 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:61333 req_id = f7d977f4-947c-4481-a3fb-068c23d4eca5 response_code = 204 uri = /switch-ports
29272024-12-19T01:09:22.279ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 231 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61333 req_id = a7c576dc-b4db-454e-a5e3-de93955ba3a2 response_code = 200 uri = /network-bootstore-config
29282024-12-19T01:09:33.393ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:42970
29292024-12-19T01:09:33.393ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:42970 req_id = 8b8af7d1-bd8e-413e-92c8-93ce98781985 response_code = 200 uri = /baseboard
29302024-12-19T01:09:34.786ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37182
29312024-12-19T01:09:34.786ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37182 req_id = 6f28eec9-027d-4f3c-a7ed-a1e0131bb520 response_code = 200 uri = /v2p
29322024-12-19T01:09:34.788ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:874 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(456258) }
29332024-12-19T01:09:34.789ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2095 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37182 req_id = 0b13400d-6008-4da2-906a-480f1db0a93b response_code = 204 uri = /v2p
29342024-12-19T01:09:34.928ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53046
29352024-12-19T01:09:34.931ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:555 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13, kind: Instance { id: 3351ff91-e235-41a8-aa20-009c343a2e26 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(456258), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13, kind: Instance { id: 3351ff91-e235-41a8-aa20-009c343a2e26 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(456258), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13, kind: Instance { id: 3351ff91-e235-41a8-aa20-009c343a2e26 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(456258), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(456258))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13, kind: Instance { id: 3351ff91-e235-41a8-aa20-009c343a2e26 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(456258), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-a8e2cbe66ab", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: ebfdbe1b-b4cf-4837-a1bd-1be22b27cb0c, read_only_parent: Some(Volume { block_size: 512, id: 31439b03-9d4a-4fd7-aaf3-a67ea004e78d, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 77cea9d0-51df-4e60-91c9-19bb39c5ed23, key: Some("ogkMo59x2d+1Cugz6GN2NzdkuKAGYmY5tvC4gnTgzT4="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19001", "[fd00:1122:3344:101::12]:19002", "[fd00:1122:3344:101::18]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: ebfdbe1b-b4cf-4837-a1bd-1be22b27cb0c, key: Some("Fu3iUwmPeoDTreb8whDhcWpU1yB5U5drYf9T5dcAStc="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19003", "[fd00:1122:3344:101::1a]:19001", "[fd00:1122:3344:101::17]:19002"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-a8e2cbe66ab" }] }), cloud_init_bytes: Some(..) } instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 metadata = InstanceMetadata { silo_id: 2bb141b0-74b5-460d-b8be-8fc9331a3779, project_id: 403bdcb1-ef82-4052-98e6-aed235a6917e } migration_id = None propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-12-19T01:09:34.928262324Z }
29362024-12-19T01:09:34.932ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:588 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 migration_id = None propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
29372024-12-19T01:09:34.932ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:1281 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 migration_id = None propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13, kind: Instance { id: 3351ff91-e235-41a8-aa20-009c343a2e26 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(456258), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13, kind: Instance { id: 3351ff91-e235-41a8-aa20-009c343a2e26 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(456258), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13, kind: Instance { id: 3351ff91-e235-41a8-aa20-009c343a2e26 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(456258), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(456258))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13, kind: Instance { id: 3351ff91-e235-41a8-aa20-009c343a2e26 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(456258), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-a8e2cbe66ab", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: ebfdbe1b-b4cf-4837-a1bd-1be22b27cb0c, read_only_parent: Some(Volume { block_size: 512, id: 31439b03-9d4a-4fd7-aaf3-a67ea004e78d, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 77cea9d0-51df-4e60-91c9-19bb39c5ed23, key: Some("ogkMo59x2d+1Cugz6GN2NzdkuKAGYmY5tvC4gnTgzT4="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19001", "[fd00:1122:3344:101::12]:19002", "[fd00:1122:3344:101::18]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: ebfdbe1b-b4cf-4837-a1bd-1be22b27cb0c, key: Some("Fu3iUwmPeoDTreb8whDhcWpU1yB5U5drYf9T5dcAStc="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19003", "[fd00:1122:3344:101::1a]:19001", "[fd00:1122:3344:101::17]:19002"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-a8e2cbe66ab" }] }), cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-12-19T01:09:34.928262324Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
29382024-12-19T01:09:34.932ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3581 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53046 req_id = 95d29a81-bafe-40fd-a947-68cb70228b70 response_code = 200 uri = /vmms/807c9dac-06bf-4827-8ab6-4e7d2843a439
29392024-12-19T01:09:35.034ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38570
29402024-12-19T01:09:35.035ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 934 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38570 req_id = b4369dfc-7749-4e3c-aae1-3fd2eae61294 response_code = 200 uri = /vmms/807c9dac-06bf-4827-8ab6-4e7d2843a439/state
29412024-12-19T01:09:35.036ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 240, 0, 0]), slot: 0, vni: Vni { inner: 456258 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
29422024-12-19T01:09:35.036ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
29432024-12-19T01:09:35.076ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439 file = illumos-utils/src/zone.rs:325 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
29442024-12-19T01:09:35.097ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439 file = illumos-utils/src/zone.rs:363 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
29452024-12-19T01:09:35.197ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54433
29462024-12-19T01:09:35.197ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54433 req_id = c5477602-4253-4010-a493-6764d710aed4 response_code = 200 uri = /vpc-routes
29472024-12-19T01:09:35.216ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13: {10.151.2.175: {4adfe2d7-5ea8-460f-955e-a23ce87027ea}, 10.151.2.176: {4adfe2d7-5ea8-460f-955e-a23ce87027ea}}} }
29482024-12-19T01:09:35.771ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8317 local_addr = [fd00:1122:3344:101::1]:37755 method = GET remote_addr = [fd00:1122:3344:101::d]:64395 req_id = 258b97eb-9c7a-4f19-9d57-25cd725dc913 response_code = 200 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c uri = /de9bec7d-f8ce-44a0-879c-5e088768f91c
29492024-12-19T01:09:36.421ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
29502024-12-19T01:09:36.421ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439 zone file = illumos-utils/src/running_zone.rs:488 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 zone = oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439
29512024-12-19T01:09:36.721ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439") failed: Property not found. retry in 32.396394ms file = illumos-utils/src/svc.rs:36 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 zone = oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439
29522024-12-19T01:09:36.762ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439") failed: Property not found. retry in 55.66869ms file = illumos-utils/src/svc.rs:36 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 zone = oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439
29532024-12-19T01:09:36.843ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439") failed: Property not found. retry in 289.591804ms file = illumos-utils/src/svc.rs:36 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 zone = oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439
29542024-12-19T01:09:37.146ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439") failed: Property not found. retry in 253.770137ms file = illumos-utils/src/svc.rs:36 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 zone = oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439
29552024-12-19T01:09:37.411ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439") failed: Property not found. retry in 671.377512ms file = illumos-utils/src/svc.rs:36 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 zone = oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439
29562024-12-19T01:09:38.095ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439") failed: Property not found. retry in 892.527699ms file = illumos-utils/src/svc.rs:36 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 zone = oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439
29572024-12-19T01:09:39.001ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439") failed: Property not found. retry in 1.46822621s file = illumos-utils/src/svc.rs:36 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 zone = oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439
29582024-12-19T01:09:40.482ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439") failed: Property not found. retry in 719.661163ms file = illumos-utils/src/svc.rs:36 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 zone = oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439
29592024-12-19T01:09:41.212ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439") failed: Property not found. retry in 1.322917259s file = illumos-utils/src/svc.rs:36 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 zone = oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439
29602024-12-19T01:09:42.562ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439 file = sled-agent/src/instance.rs:1786 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
29612024-12-19T01:09:42.572ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1795 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
29622024-12-19T01:09:42.611ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1819 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
29632024-12-19T01:09:42.749ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:932 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
29642024-12-19T01:09:42.751ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1243 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
29652024-12-19T01:09:42.751ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7535176 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54433 req_id = 9c62fd5d-57a7-4497-8637-355f84947733 response_code = 204 uri = /eip-gateways
29662024-12-19T01:09:42.751ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-12-19T01:09:42.751239645Z } file = sled-agent/src/instance.rs:780 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
29672024-12-19T01:09:42.751ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-12-19T01:09:42.751239645Z } propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
29682024-12-19T01:09:42.751ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-12-19T01:09:42.751239645Z }, migration_in: None, migration_out: None }
29692024-12-19T01:09:42.769ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-12-19T01:09:42.768991286Z } file = sled-agent/src/instance.rs:780 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
29702024-12-19T01:09:42.769ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-19T01:09:42.768991286Z } propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
29712024-12-19T01:09:42.769ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-19T01:09:42.768991286Z }, migration_in: None, migration_out: None }
29722024-12-19T01:09:42.834ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 373 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54433 req_id = de914aae-e9b0-499c-a879-0e53a201ff58 response_code = 204 uri = /vpc-routes
29732024-12-19T01:09:51.592ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44813
29742024-12-19T01:09:51.598ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44813 req_id = fd840928-8f91-4b8d-95e9-cdfc9389a3b1 response_code = 200 uri = /vmms/807c9dac-06bf-4827-8ab6-4e7d2843a439/state
29752024-12-19T01:09:51.600ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60263
29762024-12-19T01:09:51.606ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60263 req_id = 24ce683b-eb23-4e3a-93e5-0873f33a4454 response_code = 200 uri = /vmms/807c9dac-06bf-4827-8ab6-4e7d2843a439/state
29772024-12-19T01:09:51.609ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65350
29782024-12-19T01:09:51.609ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65350 req_id = 3e3e2227-d48b-49ec-b189-503a7fcda2f3 response_code = 200 uri = /vpc-routes
29792024-12-19T01:09:51.676ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13: {10.151.2.175: {4adfe2d7-5ea8-460f-955e-a23ce87027ea}, 10.151.2.176: {4adfe2d7-5ea8-460f-955e-a23ce87027ea}}} }
29802024-12-19T01:09:51.676ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:65350 req_id = c2b1b43b-8b94-471f-a692-4f504be2f86f response_code = 204 uri = /eip-gateways
29812024-12-19T01:09:51.679ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46395
29822024-12-19T01:09:51.679ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46395 req_id = 9cbf0ee6-876d-4261-9bfa-0c61b9eb195c response_code = 200 uri = /vpc-routes
29832024-12-19T01:09:51.698ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 264 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:65350 req_id = d8f1b8d1-2c1f-4364-8fb3-9ed0be93ce40 response_code = 204 uri = /vpc-routes
29842024-12-19T01:09:51.700ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13: {10.151.2.176: {4adfe2d7-5ea8-460f-955e-a23ce87027ea}, 10.151.2.175: {4adfe2d7-5ea8-460f-955e-a23ce87027ea}}} }
29852024-12-19T01:09:51.700ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46395 req_id = 3d6357d3-9012-4b2c-8c0a-7f8478e9f996 response_code = 204 uri = /eip-gateways
29862024-12-19T01:09:51.725ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 254 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46395 req_id = 6257468b-86da-4d8f-940a-f32304c016d5 response_code = 204 uri = /vpc-routes
29872024-12-19T01:09:51.770ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49905
29882024-12-19T01:09:51.772ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49905 req_id = 11366696-b293-4d3f-8116-3b7e86f11d44 response_code = 200 uri = /vmms/807c9dac-06bf-4827-8ab6-4e7d2843a439/state
29892024-12-19T01:09:51.796ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55247
29902024-12-19T01:09:51.796ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
29912024-12-19T01:09:51.843ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
29922024-12-19T01:09:51.905ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108816 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:55247 req_id = 51bd5b0d-7e8c-4fd6-a77b-5e1a047fba2c response_code = 204 uri = /switch-ports
29932024-12-19T01:09:51.905ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48497
29942024-12-19T01:09:51.905ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40521
29952024-12-19T01:09:51.905ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61615
29962024-12-19T01:09:51.906ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47000
29972024-12-19T01:09:51.906ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36915
29982024-12-19T01:09:51.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 357 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55247 req_id = d2e03a9e-a0c9-4fdc-9237-fe3082ec9140 response_code = 200 uri = /network-bootstore-config
29992024-12-19T01:09:51.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 267 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48497 req_id = 748d0c35-6695-432d-b19a-3019913dcdef response_code = 200 uri = /vpc-routes
30002024-12-19T01:09:51.906ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
30012024-12-19T01:09:51.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 262 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36915 req_id = e45f7f66-2b50-4f57-8c8a-f2421fa11c59 response_code = 200 uri = /v2p
30022024-12-19T01:09:51.911ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47000 req_id = 06276d54-5cf3-410a-9231-6c544e9b48d4 response_code = 200 uri = /v2p
30032024-12-19T01:09:51.911ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5412 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61615 req_id = 231e0963-23cb-4b17-a6b2-5c81ee3fb6b2 response_code = 200 uri = /v2p
30042024-12-19T01:09:51.933ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62788
30052024-12-19T01:09:51.947ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
30062024-12-19T01:09:51.952ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13: {10.151.2.176: {4adfe2d7-5ea8-460f-955e-a23ce87027ea}, 10.151.2.175: {4adfe2d7-5ea8-460f-955e-a23ce87027ea}}} }
30072024-12-19T01:09:51.952ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 779 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48497 req_id = c5a24195-35a5-47d5-95af-4ee6613f5998 response_code = 204 uri = /eip-gateways
30082024-12-19T01:09:51.967ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 452 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48497 req_id = bd84183d-911e-4aa8-9385-04d4ad685c0e response_code = 204 uri = /vpc-routes
30092024-12-19T01:09:52.008ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
30102024-12-19T01:09:52.008ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101836 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40521 req_id = 811c6571-45cb-445c-9d5e-ed7050dcc2e2 response_code = 204 uri = /switch-ports
30112024-12-19T01:09:52.012ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 671 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40521 req_id = db39de7d-80ab-42e4-9017-3decb53d81b1 response_code = 200 uri = /network-bootstore-config
30122024-12-19T01:09:52.043ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
30132024-12-19T01:09:52.094ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 160739 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62788 req_id = ff33dd5f-913e-4e76-aa14-4000bf0af40d response_code = 204 uri = /switch-ports
30142024-12-19T01:09:52.094ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62788 req_id = 9de0d099-a0e3-4075-bc06-9ee212897bc1 response_code = 200 uri = /network-bootstore-config
30152024-12-19T01:09:52.659ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60591
30162024-12-19T01:09:52.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 973 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60591 req_id = b44ac208-abe1-425a-b577-f256786a5453 response_code = 200 uri = /vmms/807c9dac-06bf-4827-8ab6-4e7d2843a439/state
30172024-12-19T01:09:52.661ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-12-19T01:09:52.661029483Z } file = sled-agent/src/instance.rs:780 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
30182024-12-19T01:09:52.661ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-12-19T01:09:52.661029483Z } propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
30192024-12-19T01:09:52.661ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-12-19T01:09:52.661029483Z }, migration_in: None, migration_out: None }
30202024-12-19T01:09:52.673ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-12-19T01:09:52.673303139Z } file = sled-agent/src/instance.rs:780 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
30212024-12-19T01:09:52.673ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-12-19T01:09:52.673303139Z } propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
30222024-12-19T01:09:52.673ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:816 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
30232024-12-19T01:09:52.673ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
30242024-12-19T01:09:52.673ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:602 id = 15341334488679396811 n_samples = 6 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
30252024-12-19T01:09:52.673ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:602 id = 13230004881984272096 n_samples = 6 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
30262024-12-19T01:09:52.673ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
30272024-12-19T01:09:52.673ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/8fd565e7-8dbe-4e0e-b9f6-62127926c764/debug/bundle/zone", "/pool/int/c0070a0a-b66c-49d5-a725-e7820b89ee73/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/a6644750-441b-4297-9acc-8dd23a71b4cb/crypt/debug", "/pool/ext/ac442b57-464c-4650-8ad2-3f5b4744c9fa/crypt/debug", "/pool/ext/93eb8a71-6519-482e-a366-b15e30b4605e/crypt/debug", "/pool/ext/98a100ce-cd82-4209-8530-7b48ca829744/crypt/debug", "/pool/ext/a98ada5c-5d21-42de-9ddc-c9598d5076c6/crypt/debug", "/pool/ext/6652bba4-bb1a-4de5-9e9c-cc3c7d421cdb/crypt/debug", "/pool/ext/f4009784-7bde-44ff-854e-49d614e9e682/crypt/debug", "/pool/ext/1025d0d6-c165-4f43-bef0-cb2eeb04b8c2/crypt/debug", "/pool/ext/499e53d3-d67c-44b6-9c03-b66767baed7e/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439
30282024-12-19T01:09:53.309ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1142 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439", bundle_id: 42e444ae-aad3-4be1-9202-43035cc1fd4c }, time_created: 2024-12-19T01:09:52.674406104Z, version: 0, cause: TerminatedInstance }
30292024-12-19T01:09:53.309ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_807c9dac-06bf-4827-8ab6-4e7d2843a439 file = sled-agent/src/instance.rs:1032 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
30302024-12-19T01:09:55.057ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:279 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
30312024-12-19T01:09:55.073ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:1000 id = RouterId { vni: Vni(456258), kind: System }
30322024-12-19T01:09:55.073ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:1000 id = RouterId { vni: Vni(456258), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
30332024-12-19T01:09:55.074ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-12-19T01:09:55.073992606Z }, migration_in: None, migration_out: None }
30342024-12-19T01:09:55.098ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-12-19T01:09:55.073992606Z }, migration_in: None, migration_out: None }
30352024-12-19T01:09:55.103ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:969 instance_id = 3351ff91-e235-41a8-aa20-009c343a2e26 propolis_id = 807c9dac-06bf-4827-8ab6-4e7d2843a439
30362024-12-19T01:09:55.547ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47586
30372024-12-19T01:09:55.547ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47586 req_id = dae9fef0-0155-4e92-a52f-49dbdff572ab response_code = 200 uri = /vpc-routes
30382024-12-19T01:09:55.589ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {2aa1db5d-9b5c-4679-b0fc-a689dfdf0c13: {10.151.2.175: {4adfe2d7-5ea8-460f-955e-a23ce87027ea}, 10.151.2.176: {4adfe2d7-5ea8-460f-955e-a23ce87027ea}}} }
30392024-12-19T01:09:55.589ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 188 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47586 req_id = 779c3879-d4f3-4a47-b312-8cba2e320831 response_code = 204 uri = /eip-gateways
30402024-12-19T01:09:55.600ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 307 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47586 req_id = c640d76d-7db1-4619-adb2-b45097a07cbd response_code = 204 uri = /vpc-routes
30412024-12-19T01:09:55.654ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42930
30422024-12-19T01:09:55.654ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42930 req_id = aeadc56e-3e06-4a70-a26c-de559e27da0d response_code = 200 uri = /v2p
30432024-12-19T01:09:56.960ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46732
30442024-12-19T01:09:56.960ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 195 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46732 req_id = 9b5ab8a2-7803-450a-a202-9dc3ba2fd003 response_code = 200 uri = /v2p
30452024-12-19T01:09:57.001ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55207
30462024-12-19T01:09:57.001ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55207 req_id = bead398c-e327-410d-90e2-e9d5f5d66b05 response_code = 200 uri = /vpc-routes
30472024-12-19T01:09:57.042ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
30482024-12-19T01:09:57.043ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1243 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c
30492024-12-19T01:09:57.043ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55207 req_id = fd392ae8-9e9b-46f3-929a-76f4369b054a response_code = 204 uri = /eip-gateways
30502024-12-19T01:09:57.050ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 291 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55207 req_id = e035af8c-f081-432a-8e16-1f2e584052ff response_code = 204 uri = /vpc-routes
30512024-12-19T01:09:57.624ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45931
30522024-12-19T01:09:57.624ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45931 req_id = 8c252cda-81b3-4cbb-b207-bc62a6d0e033 response_code = 200 uri = /vpc-routes
30532024-12-19T01:09:57.640ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = de9bec7d-f8ce-44a0-879c-5e088768f91c values = ExternalIpGatewayMap { mappings: {} }
30542024-12-19T01:09:57.640ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45931 req_id = da439c58-eb6c-478e-b316-fe3677ba7a8f response_code = 204 uri = /eip-gateways
30552024-12-19T01:09:57.647ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 200 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45931 req_id = 66b9f204-4eac-4729-bb8f-5e45475708e4 response_code = 204 uri = /vpc-routes