Max level shown:
1 [ Aug 12 22:30:22 Enabled. ]
2 [ Aug 12 22:30:22 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Aug 12 22:30:22 Method "start" exited with status 0. ]
4 [ Aug 12 22:30:22 Rereading configuration. ]
5 [ Aug 12 22:30:22 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-08-12T22:30:22.592ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-08-12T22:30:22.593ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-08-12T22:30:22.644ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-08-12T22:30:22.644ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-08-12T22:30:22.761ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-08-12T22:30:22.770ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-08-12T22:30:22.778ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-08-12T22:30:23.379ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:a02a::, width: 64 }
152024-08-12T22:30:23.519ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:125
162024-08-12T22:30:23.519ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:137
172024-08-12T22:30:23.519ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:150
182024-08-12T22:30:23.520ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:172 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192024-08-12T22:30:23.520ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:716
202024-08-12T22:30:23.520ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
212024-08-12T22:30:23.523ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "sock", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:623
222024-08-12T22:30:23.523ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:765
232024-08-12T22:30:23.523ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:186
242024-08-12T22:30:23.523ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:824
252024-08-12T22:30:23.523ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:242 vdev = /scratch/m2_0.vdev
262024-08-12T22:30:23.523ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-08-12T22:30:23.523ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:405
282024-08-12T22:30:23.523ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-08-12T22:30:23.523ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-08-12T22:30:23.534ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:360
312024-08-12T22:30:23.534ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool)
322024-08-12T22:30:23.661ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:304 zpool = ZpoolName { id: 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool), kind: Internal }
332024-08-12T22:30:23.897ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:405
342024-08-12T22:30:23.897ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:242 vdev = /scratch/m2_1.vdev
352024-08-12T22:30:23.897ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:601 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-08-12T22:30:23.897ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:622 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-08-12T22:30:23.897ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:405
382024-08-12T22:30:23.897ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-08-12T22:30:23.897ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-08-12T22:30:23.905ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:360
412024-08-12T22:30:23.905ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool)
422024-08-12T22:30:24.017ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:304 zpool = ZpoolName { id: b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool), kind: Internal }
432024-08-12T22:30:24.250ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:242 vdev = /scratch/u2_0.vdev
442024-08-12T22:30:24.250ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
462024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
472024-08-12T22:30:24.251ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:405
482024-08-12T22:30:24.251ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
492024-08-12T22:30:24.251ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:242 vdev = /scratch/u2_1.vdev
502024-08-12T22:30:24.251ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:405
512024-08-12T22:30:24.251ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
522024-08-12T22:30:24.251ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
532024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
542024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
552024-08-12T22:30:24.251ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:242 vdev = /scratch/u2_2.vdev
562024-08-12T22:30:24.251ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:405
572024-08-12T22:30:24.251ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
582024-08-12T22:30:24.251ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
592024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
602024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
612024-08-12T22:30:24.251ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:242 vdev = /scratch/u2_3.vdev
622024-08-12T22:30:24.251ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
632024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
642024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
652024-08-12T22:30:24.251ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:405
662024-08-12T22:30:24.251ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
672024-08-12T22:30:24.251ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:242 vdev = /scratch/u2_4.vdev
682024-08-12T22:30:24.251ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
692024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
702024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
712024-08-12T22:30:24.251ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:405
722024-08-12T22:30:24.251ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
732024-08-12T22:30:24.251ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:242 vdev = /scratch/u2_5.vdev
742024-08-12T22:30:24.251ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:405
752024-08-12T22:30:24.251ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
762024-08-12T22:30:24.251ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
772024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
782024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
792024-08-12T22:30:24.251ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:242 vdev = /scratch/u2_6.vdev
802024-08-12T22:30:24.251ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:405
812024-08-12T22:30:24.251ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
822024-08-12T22:30:24.251ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
832024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
842024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
852024-08-12T22:30:24.251ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:242 vdev = /scratch/u2_7.vdev
862024-08-12T22:30:24.251ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
872024-08-12T22:30:24.251ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:405
882024-08-12T22:30:24.251ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
892024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
902024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
912024-08-12T22:30:24.251ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:242 vdev = /scratch/u2_8.vdev
922024-08-12T22:30:24.251ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
932024-08-12T22:30:24.251ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:405
942024-08-12T22:30:24.251ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
952024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
962024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
972024-08-12T22:30:24.251ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:96
982024-08-12T22:30:24.251ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:98
992024-08-12T22:30:24.251ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1002024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1012024-08-12T22:30:24.251ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1022024-08-12T22:30:24.251ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
1032024-08-12T22:30:24.251ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:210
1042024-08-12T22:30:24.251ZINFOSledAgent (bootstore): No ledger in /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1052024-08-12T22:30:24.251ZINFOSledAgent (bootstore): No ledger in /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1062024-08-12T22:30:24.251ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-sock-i86pc
1072024-08-12T22:30:24.251ZINFOSledAgent (bootstore): No ledger in /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1082024-08-12T22:30:24.251ZINFOSledAgent (bootstore): No ledger in /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1092024-08-12T22:30:24.251ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-sock-i86pc
1102024-08-12T22:30:24.251ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:215
1112024-08-12T22:30:24.251ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:230
1122024-08-12T22:30:24.261ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:729
1132024-08-12T22:30:24.261ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:292 next_cleanup = Instant { tv_sec: 1689, tv_nsec: 156703906 } time_to_next_cleanup = 599.999929958s
1142024-08-12T22:30:24.261ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3726
1152024-08-12T22:30:24.261ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
1162024-08-12T22:30:24.261ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-08-12T22:30:24.261ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-08-12T22:30:24.264ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:205 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
1192024-08-12T22:30:24.264ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:44 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
1202024-08-12T22:30:24.265ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1017
1212024-08-12T22:30:24.265ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
1222024-08-12T22:30:24.265ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
1232024-08-12T22:30:24.265ZINFOSledAgent (RSS): No ledger in /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1242024-08-12T22:30:24.265ZINFOSledAgent (RSS): No ledger in /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1252024-08-12T22:30:24.265ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:1077
1262024-08-12T22:30:24.265ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
1272024-08-12T22:30:24.265ZINFOSledAgent (RSS): No ledger in /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/rss-sled-plan.json file = common/src/ledger.rs:188
1282024-08-12T22:30:24.265ZINFOSledAgent (RSS): No ledger in /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/rss-sled-plan.json file = common/src/ledger.rs:188
1292024-08-12T22:30:24.265ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1118
1302024-08-12T22:30:24.265ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:156
1312024-08-12T22:30:24.265ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:129
1322024-08-12T22:30:24.265ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
1332024-08-12T22:30:24.265ZINFOSledAgent (RSS): Writing ledger to /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1342024-08-12T22:30:24.265ZINFOSledAgent (RSS): Writing ledger to /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1352024-08-12T22:30:24.265ZINFOSledAgent (RSS): Sled plan written to storage file = sled-agent/src/rack_setup/plan/sled.rs:176
1362024-08-12T22:30:24.265ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1151
1372024-08-12T22:30:24.265ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
1382024-08-12T22:30:24.265ZINFOSledAgent (bootstore): Writing ledger to /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1392024-08-12T22:30:24.266ZINFOSledAgent (bootstore): Writing ledger to /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1402024-08-12T22:30:24.266ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
1412024-08-12T22:30:24.266ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:217 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 6e18d78c-496f-48cb-b07e-b18dbc665f73, rack_id: 3430adde-27ec-4e0d-a703-b8e2cc54d5df, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1422024-08-12T22:30:24.266ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:66 remote_addr = [fdb0:18c0:4d0d:a02a::1]:53946
1432024-08-12T22:30:24.266ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 6e18d78c-496f-48cb-b07e-b18dbc665f73, rack_id: 3430adde-27ec-4e0d-a703-b8e2cc54d5df, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:359
1442024-08-12T22:30:24.266ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:374
1452024-08-12T22:30:24.266ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:405
1462024-08-12T22:30:24.266ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:573
1472024-08-12T22:30:24.266ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1482024-08-12T22:30:24.266ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1492024-08-12T22:30:24.266ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
1502024-08-12T22:30:24.266ZINFOSledAgent (StorageManager): No ledger in /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_ledger
1512024-08-12T22:30:24.267ZINFOSledAgent (StorageManager): No ledger in /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_ledger
1522024-08-12T22:30:24.267ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:562
1532024-08-12T22:30:24.267ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:590
1542024-08-12T22:30:24.267ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:394
1552024-08-12T22:30:24.267ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:374
1562024-08-12T22:30:24.267ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:480 result = DisksManagementResult { status: [] }
1572024-08-12T22:30:24.268ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:3960
1582024-08-12T22:30:24.268ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4342
1592024-08-12T22:30:24.299ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
1602024-08-12T22:30:24.319ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:316
1612024-08-12T22:30:24.337ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:354
1622024-08-12T22:30:24.345ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:381 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
1632024-08-12T22:30:24.345ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
1642024-08-12T22:30:24.345ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:393 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
1652024-08-12T22:30:24.345ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1662024-08-12T22:30:24.348ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 80.051428ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "ab58c92f-f3e5-402a-92be-1f7c1b886316", "content-length": "124", "date": "Mon, 12 Aug 2024 22:30:23 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "ab58c92f-f3e5-402a-92be-1f7c1b886316" } file = clients/ddm-admin-client/src/lib.rs:177
1672024-08-12T22:30:24.358ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_2acd7d8f-9546-4b2f-a46b-354a4078679d/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1682024-08-12T22:30:24.387ZINFOSledAgent: successfully created encrypted zvol: name="oxi_2acd7d8f-9546-4b2f-a46b-354a4078679d/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1692024-08-12T22:30:24.387ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_2acd7d8f-9546-4b2f-a46b-354a4078679d/swap" file = sled-agent/src/swap_device.rs:96
1702024-08-12T22:30:24.393ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:408 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
1712024-08-12T22:30:24.393ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:126
1722024-08-12T22:30:24.393ZINFOSledAgent (BackingFs): Ensuring dataset oxi_2acd7d8f-9546-4b2f-a46b-354a4078679d/backing/fmd file = sled-agent/src/backing_fs.rs:136
1732024-08-12T22:30:24.430ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1742024-08-12T22:30:24.430ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 85.778201ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "619834f5-85cb-4cbe-afae-9d75440ce5eb", "content-length": "124", "date": "Mon, 12 Aug 2024 22:30:24 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "619834f5-85cb-4cbe-afae-9d75440ce5eb" } file = clients/ddm-admin-client/src/lib.rs:177
1752024-08-12T22:30:24.474ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:170
1762024-08-12T22:30:24.518ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1772024-08-12T22:30:24.518ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 145.992979ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "41b25860-680a-46ee-b075-f49214b2292d", "content-length": "124", "date": "Mon, 12 Aug 2024 22:30:24 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "41b25860-680a-46ee-b075-f49214b2292d" } file = clients/ddm-admin-client/src/lib.rs:177
1782024-08-12T22:30:24.665ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1792024-08-12T22:30:24.666ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 113.60273ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "8371b309-e511-439b-b6e1-658e2c612df6", "content-length": "124", "date": "Mon, 12 Aug 2024 22:30:24 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "8371b309-e511-439b-b6e1-658e2c612df6" } file = clients/ddm-admin-client/src/lib.rs:177
1802024-08-12T22:30:24.694ZINFOSledAgent (BackingFs): Mounting oxi_2acd7d8f-9546-4b2f-a46b-354a4078679d/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:178
1812024-08-12T22:30:24.707ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:186
1822024-08-12T22:30:24.707ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:186
1832024-08-12T22:30:24.707ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:186
1842024-08-12T22:30:24.707ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:192
1852024-08-12T22:30:24.781ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1862024-08-12T22:30:24.781ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 192.143484ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "f5eaced9-fcb5-4f45-a394-d7ca39cc60d0", "content-length": "124", "date": "Mon, 12 Aug 2024 22:30:24 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "f5eaced9-fcb5-4f45-a394-d7ca39cc60d0" } file = clients/ddm-admin-client/src/lib.rs:177
1872024-08-12T22:30:24.974ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1882024-08-12T22:30:24.975ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 325.177214ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "685f2ed1-6bd6-45ec-a1c4-17718f36cad4", "content-length": "124", "date": "Mon, 12 Aug 2024 22:30:24 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "685f2ed1-6bd6-45ec-a1c4-17718f36cad4" } file = clients/ddm-admin-client/src/lib.rs:177
1892024-08-12T22:30:25.302ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1902024-08-12T22:30:25.302ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 355.393245ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "2c2f895e-5b4c-49a2-85ac-301e63f93564", "content-length": "124", "date": "Mon, 12 Aug 2024 22:30:24 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "2c2f895e-5b4c-49a2-85ac-301e63f93564" } file = clients/ddm-admin-client/src/lib.rs:177
1912024-08-12T22:30:25.440ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:90 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
1922024-08-12T22:30:25.446ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:53671 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
1932024-08-12T22:30:25.446ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
1942024-08-12T22:30:25.446ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:53671 file = oximeter/producer/src/lib.rs:283 interval = 30s producer_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
1952024-08-12T22:30:25.447ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 194.459721ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
1962024-08-12T22:30:25.448ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
1972024-08-12T22:30:25.448ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
1982024-08-12T22:30:25.448ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:913 underlay_address = fd00:1122:3344:101::1
1992024-08-12T22:30:25.448ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
2002024-08-12T22:30:25.643ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 436.382875ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
2012024-08-12T22:30:25.659ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2022024-08-12T22:30:26.081ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 717.338163ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
2032024-08-12T22:30:26.800ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 2.824541477s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
2042024-08-12T22:30:27.610ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2499
2052024-08-12T22:30:27.610ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2717
2062024-08-12T22:30:27.610ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2369
2072024-08-12T22:30:27.610ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2416
2082024-08-12T22:30:27.610ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2798
2092024-08-12T22:30:27.610ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:2840
2102024-08-12T22:30:27.612ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2642
2112024-08-12T22:30:27.613ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2410
2122024-08-12T22:30:27.614ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='d6d5e490-893d-4a52-9d04-c3d969b7a8eb'/> <propval type="astring" name="rack_id" value='3430adde-27ec-4e0d-a703-b8e2cc54d5df'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:a02a::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fe74:9cf2'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:a02a::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"sock","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_id" value='6e18d78c-496f-48cb-b07e-b18dbc665f73'/> <propval type="astring" name="rack_id" value='3430adde-27ec-4e0d-a703-b8e2cc54d5df'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='sock'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='6e18d78c-496f-48cb-b07e-b18dbc665f73'/> <propval type="astring" name="rack_uuid" value='3430adde-27ec-4e0d-a703-b8e2cc54d5df'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='6e18d78c-496f-48cb-b07e-b18dbc665f73'/> <propval type="astring" name="rack_uuid" value='3430adde-27ec-4e0d-a703-b8e2cc54d5df'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2132024-08-12T22:30:27.614ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2142024-08-12T22:30:27.872ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 53.556714ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2152024-08-12T22:30:28.106ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 53.877043ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2162024-08-12T22:30:28.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 220.52396ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2172024-08-12T22:30:28.633ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 461.274737ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182024-08-12T22:30:29.191ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.077472353s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192024-08-12T22:30:29.627ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 2.80877238s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
2202024-08-12T22:30:30.492ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.240737847s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2212024-08-12T22:30:31.843ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.124430666s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222024-08-12T22:30:32.437ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 11.804336671s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
2232024-08-12T22:30:32.983ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.091556098s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242024-08-12T22:30:34.476ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:12345
2252024-08-12T22:30:34.476ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:429
2262024-08-12T22:30:34.476ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
2272024-08-12T22:30:34.476ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2282024-08-12T22:30:34.477ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2292024-08-12T22:30:34.477ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2302024-08-12T22:30:34.477ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:73 remote_addr = [fdb0:18c0:4d0d:a02a::1]:53946
2312024-08-12T22:30:34.477ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.2.100, vlan_id: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false }], bgp: [], bfd: [] })))
2322024-08-12T22:30:34.477ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3726
2332024-08-12T22:30:34.477ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:108 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2342024-08-12T22:30:34.477ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:232 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2352024-08-12T22:30:34.703ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
2362024-08-12T22:30:34.703ZINFOSledAgent (RSS): No ledger in /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/rss-service-plan-v3.json file = common/src/ledger.rs:188
2372024-08-12T22:30:34.703ZINFOSledAgent (RSS): No ledger in /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/rss-service-plan-v3.json file = common/src/ledger.rs:188
2382024-08-12T22:30:34.703ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
2392024-08-12T22:30:34.704ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
2402024-08-12T22:30:34.709ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:3984 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2412024-08-12T22:30:34.709ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4000
2422024-08-12T22:30:34.709ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2432024-08-12T22:30:34.965ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50125
2442024-08-12T22:30:34.965ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
2452024-08-12T22:30:34.965ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 218 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50125 req_id = 3adbf880-74c0-415f-8017-90173776d357 response_code = 200 uri = /inventory
2462024-08-12T22:30:34.994ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50113
2472024-08-12T22:30:34.994ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50113 req_id = 2411e67b-0f53-4e5e-a45f-2514bfc5e114 response_code = 200 uri = /sled-role
2482024-08-12T22:30:34.995ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
2492024-08-12T22:30:34.995ZINFOSledAgent (RSS): Writing ledger to /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/.rss-service-plan-v3.json.tmp file = common/src/ledger.rs:199
2502024-08-12T22:30:34.995ZINFOSledAgent (RSS): Writing ledger to /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/.rss-service-plan-v3.json.tmp file = common/src/ledger.rs:199
2512024-08-12T22:30:34.996ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:828
2522024-08-12T22:30:35.025ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 62782c01-5a4f-4eef-beef-0f0f24c0e19e, pool_id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 47a168c3-1013-4122-8cbc-1155d47f004b, pool_id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: a33a84e2-3c4f-4ba5-a47b-a0410d19e4ad, pool_id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 5d1040bf-beed-425d-a983-188fdc297a8c, pool_id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: d4304f5b-3008-4f53-9788-b9ed0a61e8c7, pool_id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: ab724e72-ede4-48fd-b9f3-e455c0fd992f, pool_id: ca45bffe-f952-4bbd-bcc4-be5de210718a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 29ac352b-de26-446b-9d5b-f0fae49adc22, pool_id: 65eb96fa-5b90-4e6e-968e-0b7f2a4634d3 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: c002b79e-3fd8-4f94-bc19-8766c0ab760a, pool_id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: bf0118f9-df05-400c-9692-f08b44a791b9, pool_id: 09108500-899b-4c15-aa1f-0c8295b5001c (zpool) }] } file = sled-agent/src/rack_setup/service.rs:355 sled_address = [fd00:1122:3344:101::1]:12345
2532024-08-12T22:30:35.025ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36284
2542024-08-12T22:30:35.025ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:829 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
2552024-08-12T22:30:35.025ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 62782c01-5a4f-4eef-beef-0f0f24c0e19e, pool_id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 47a168c3-1013-4122-8cbc-1155d47f004b, pool_id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: a33a84e2-3c4f-4ba5-a47b-a0410d19e4ad, pool_id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 5d1040bf-beed-425d-a983-188fdc297a8c, pool_id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: d4304f5b-3008-4f53-9788-b9ed0a61e8c7, pool_id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: ab724e72-ede4-48fd-b9f3-e455c0fd992f, pool_id: ca45bffe-f952-4bbd-bcc4-be5de210718a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 29ac352b-de26-446b-9d5b-f0fae49adc22, pool_id: 65eb96fa-5b90-4e6e-968e-0b7f2a4634d3 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: c002b79e-3fd8-4f94-bc19-8766c0ab760a, pool_id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: bf0118f9-df05-400c-9692-f08b44a791b9, pool_id: 09108500-899b-4c15-aa1f-0c8295b5001c (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:405
2562024-08-12T22:30:35.025ZINFOSledAgent (StorageManager): No ledger in /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2572024-08-12T22:30:35.025ZINFOSledAgent (StorageManager): No ledger in /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2582024-08-12T22:30:35.025ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:644 request = omicron_physical_disks_ensure
2592024-08-12T22:30:35.026ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:374
2602024-08-12T22:30:35.026ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:414
2612024-08-12T22:30:35.026ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:491
2622024-08-12T22:30:35.026ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2632024-08-12T22:30:35.183ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:360
2642024-08-12T22:30:35.183ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool)
2652024-08-12T22:30:35.386ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
2662024-08-12T22:30:35.478ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/6e18d78c-496f-48cb-b07e-b18dbc665f73): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:229 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
2672024-08-12T22:30:35.503ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:304 zpool = ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }
2682024-08-12T22:30:35.511ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:348
2692024-08-12T22:30:35.511ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:350
2702024-08-12T22:30:35.511ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:354
2712024-08-12T22:30:35.511ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:356
2722024-08-12T22:30:35.511ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2732024-08-12T22:30:35.511ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2742024-08-12T22:30:35.511ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:368
2752024-08-12T22:30:35.552ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2762024-08-12T22:30:35.559ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_537dde5b-089b-4e20-9ef8-bf10127d7dce/crypt/zone file = sled-storage/src/dataset.rs:411
2772024-08-12T22:30:35.699ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_537dde5b-089b-4e20-9ef8-bf10127d7dce file = sled-storage/src/dataset.rs:496
2782024-08-12T22:30:35.706ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:512
2792024-08-12T22:30:35.706ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:429
2802024-08-12T22:30:35.706ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:414
2812024-08-12T22:30:35.706ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:491
2822024-08-12T22:30:35.706ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2832024-08-12T22:30:35.716ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:360
2842024-08-12T22:30:35.716ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 90f5d185-2468-460f-8d20-ecb50330e694 (zpool)
2852024-08-12T22:30:35.824ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:304 zpool = ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }
2862024-08-12T22:30:35.831ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:348
2872024-08-12T22:30:35.831ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:350
2882024-08-12T22:30:35.831ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:354
2892024-08-12T22:30:35.831ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:356
2902024-08-12T22:30:35.831ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2912024-08-12T22:30:35.831ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2922024-08-12T22:30:35.831ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:368
2932024-08-12T22:30:35.871ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2942024-08-12T22:30:35.878ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_90f5d185-2468-460f-8d20-ecb50330e694/crypt/zone file = sled-storage/src/dataset.rs:411
2952024-08-12T22:30:36.012ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_90f5d185-2468-460f-8d20-ecb50330e694 file = sled-storage/src/dataset.rs:496
2962024-08-12T22:30:36.022ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:512
2972024-08-12T22:30:36.022ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:429
2982024-08-12T22:30:36.022ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:414
2992024-08-12T22:30:36.022ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:491
3002024-08-12T22:30:36.022ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3012024-08-12T22:30:36.032ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:360
3022024-08-12T22:30:36.032ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool)
3032024-08-12T22:30:36.141ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:304 zpool = ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }
3042024-08-12T22:30:36.149ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:348
3052024-08-12T22:30:36.149ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:350
3062024-08-12T22:30:36.149ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:354
3072024-08-12T22:30:36.149ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:356
3082024-08-12T22:30:36.149ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3092024-08-12T22:30:36.149ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3102024-08-12T22:30:36.149ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:368
3112024-08-12T22:30:36.191ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3122024-08-12T22:30:36.198ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_9ac4fc8d-4a30-4017-bbdc-a392e988e5c6/crypt/zone file = sled-storage/src/dataset.rs:411
3132024-08-12T22:30:36.339ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 file = sled-storage/src/dataset.rs:496
3142024-08-12T22:30:36.347ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:512
3152024-08-12T22:30:36.347ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:429
3162024-08-12T22:30:36.347ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:414
3172024-08-12T22:30:36.347ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:491
3182024-08-12T22:30:36.347ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3192024-08-12T22:30:36.357ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:360
3202024-08-12T22:30:36.357ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool)
3212024-08-12T22:30:36.469ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:304 zpool = ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }
3222024-08-12T22:30:36.476ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:348
3232024-08-12T22:30:36.476ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:350
3242024-08-12T22:30:36.476ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:354
3252024-08-12T22:30:36.476ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:356
3262024-08-12T22:30:36.476ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3272024-08-12T22:30:36.476ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3282024-08-12T22:30:36.476ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:368
3292024-08-12T22:30:36.480ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/6e18d78c-496f-48cb-b07e-b18dbc665f73): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:229 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
3302024-08-12T22:30:36.519ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3312024-08-12T22:30:36.525ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_fab24dee-2d14-4ad6-8dd1-967cd0dd455c/crypt/zone file = sled-storage/src/dataset.rs:411
3322024-08-12T22:30:36.656ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_fab24dee-2d14-4ad6-8dd1-967cd0dd455c file = sled-storage/src/dataset.rs:496
3332024-08-12T22:30:36.663ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:512
3342024-08-12T22:30:36.663ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:429
3352024-08-12T22:30:36.663ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:414
3362024-08-12T22:30:36.663ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:491
3372024-08-12T22:30:36.663ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3382024-08-12T22:30:36.672ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:360
3392024-08-12T22:30:36.673ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool)
3402024-08-12T22:30:36.786ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:304 zpool = ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }
3412024-08-12T22:30:36.793ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:348
3422024-08-12T22:30:36.793ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:350
3432024-08-12T22:30:36.793ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:354
3442024-08-12T22:30:36.793ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:356
3452024-08-12T22:30:36.793ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3462024-08-12T22:30:36.793ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3472024-08-12T22:30:36.793ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:368
3482024-08-12T22:30:36.833ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3492024-08-12T22:30:36.840ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_37abb3ae-dbff-4acb-a865-55b4ed0374e9/crypt/zone file = sled-storage/src/dataset.rs:411
3502024-08-12T22:30:36.972ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_37abb3ae-dbff-4acb-a865-55b4ed0374e9 file = sled-storage/src/dataset.rs:496
3512024-08-12T22:30:36.979ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:512
3522024-08-12T22:30:36.979ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:429
3532024-08-12T22:30:36.979ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:414
3542024-08-12T22:30:36.979ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:491
3552024-08-12T22:30:36.979ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3562024-08-12T22:30:36.988ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:360
3572024-08-12T22:30:36.988ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = ca45bffe-f952-4bbd-bcc4-be5de210718a (zpool)
3582024-08-12T22:30:37.094ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:304 zpool = ZpoolName { id: ca45bffe-f952-4bbd-bcc4-be5de210718a (zpool), kind: External }
3592024-08-12T22:30:37.101ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:348
3602024-08-12T22:30:37.101ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:350
3612024-08-12T22:30:37.101ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:354
3622024-08-12T22:30:37.101ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:356
3632024-08-12T22:30:37.101ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3642024-08-12T22:30:37.101ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3652024-08-12T22:30:37.101ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:368
3662024-08-12T22:30:37.141ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3672024-08-12T22:30:37.149ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_ca45bffe-f952-4bbd-bcc4-be5de210718a/crypt/zone file = sled-storage/src/dataset.rs:411
3682024-08-12T22:30:37.283ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_ca45bffe-f952-4bbd-bcc4-be5de210718a file = sled-storage/src/dataset.rs:496
3692024-08-12T22:30:37.291ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:512
3702024-08-12T22:30:37.291ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:429
3712024-08-12T22:30:37.291ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:414
3722024-08-12T22:30:37.291ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:491
3732024-08-12T22:30:37.291ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3742024-08-12T22:30:37.300ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:360
3752024-08-12T22:30:37.300ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 65eb96fa-5b90-4e6e-968e-0b7f2a4634d3 (zpool)
3762024-08-12T22:30:37.410ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:304 zpool = ZpoolName { id: 65eb96fa-5b90-4e6e-968e-0b7f2a4634d3 (zpool), kind: External }
3772024-08-12T22:30:37.417ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:348
3782024-08-12T22:30:37.417ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:350
3792024-08-12T22:30:37.417ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:354
3802024-08-12T22:30:37.417ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:356
3812024-08-12T22:30:37.417ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3822024-08-12T22:30:37.417ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3832024-08-12T22:30:37.417ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:368
3842024-08-12T22:30:37.464ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3852024-08-12T22:30:37.473ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_65eb96fa-5b90-4e6e-968e-0b7f2a4634d3/crypt/zone file = sled-storage/src/dataset.rs:411
3862024-08-12T22:30:37.485ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/6e18d78c-496f-48cb-b07e-b18dbc665f73): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:229 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
3872024-08-12T22:30:37.619ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_65eb96fa-5b90-4e6e-968e-0b7f2a4634d3 file = sled-storage/src/dataset.rs:496
3882024-08-12T22:30:37.626ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:512
3892024-08-12T22:30:37.626ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:429
3902024-08-12T22:30:37.626ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:414
3912024-08-12T22:30:37.626ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:491
3922024-08-12T22:30:37.626ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
3932024-08-12T22:30:37.636ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:360
3942024-08-12T22:30:37.636ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool)
3952024-08-12T22:30:37.748ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:304 zpool = ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }
3962024-08-12T22:30:37.756ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:348
3972024-08-12T22:30:37.756ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:350
3982024-08-12T22:30:37.756ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:354
3992024-08-12T22:30:37.756ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:356
4002024-08-12T22:30:37.756ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4012024-08-12T22:30:37.756ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4022024-08-12T22:30:37.756ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:368
4032024-08-12T22:30:37.796ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4042024-08-12T22:30:37.803ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_ef93b2f7-5e79-4766-bedb-2c6d5ba93538/crypt/zone file = sled-storage/src/dataset.rs:411
4052024-08-12T22:30:37.949ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_ef93b2f7-5e79-4766-bedb-2c6d5ba93538 file = sled-storage/src/dataset.rs:496
4062024-08-12T22:30:37.957ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:512
4072024-08-12T22:30:37.957ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:429
4082024-08-12T22:30:37.957ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:414
4092024-08-12T22:30:37.957ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:491
4102024-08-12T22:30:37.957ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4112024-08-12T22:30:37.967ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:360
4122024-08-12T22:30:37.967ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 09108500-899b-4c15-aa1f-0c8295b5001c (zpool)
4132024-08-12T22:30:38.077ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:304 zpool = ZpoolName { id: 09108500-899b-4c15-aa1f-0c8295b5001c (zpool), kind: External }
4142024-08-12T22:30:38.085ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:348
4152024-08-12T22:30:38.085ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:350
4162024-08-12T22:30:38.085ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:354
4172024-08-12T22:30:38.085ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:356
4182024-08-12T22:30:38.085ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4192024-08-12T22:30:38.085ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4202024-08-12T22:30:38.085ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:368
4212024-08-12T22:30:38.130ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4222024-08-12T22:30:38.139ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_09108500-899b-4c15-aa1f-0c8295b5001c/crypt/zone file = sled-storage/src/dataset.rs:411
4232024-08-12T22:30:38.281ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_09108500-899b-4c15-aa1f-0c8295b5001c file = sled-storage/src/dataset.rs:496
4242024-08-12T22:30:38.289ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:512
4252024-08-12T22:30:38.289ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:429
4262024-08-12T22:30:38.289ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4272024-08-12T22:30:38.289ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2acd7d8f-9546-4b2f-a46b-354a4078679d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b9292fe7-3702-412e-ade5-90d5eadbfb98 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ca45bffe-f952-4bbd-bcc4-be5de210718a (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 65eb96fa-5b90-4e6e-968e-0b7f2a4634d3 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 09108500-899b-4c15-aa1f-0c8295b5001c (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4282024-08-12T22:30:38.289ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4292024-08-12T22:30:38.289ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4302024-08-12T22:30:38.289ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4312024-08-12T22:30:38.290ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:833 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
4322024-08-12T22:30:38.290ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
4332024-08-12T22:30:38.290ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:859 generation = Generation(1) sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
4342024-08-12T22:30:38.290ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:864 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
4352024-08-12T22:30:38.290ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:869 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
4362024-08-12T22:30:38.290ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:120 gen = Generation(1) sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
4372024-08-12T22:30:38.290ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:875 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
4382024-08-12T22:30:38.290ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:803 gen = Generation(1)
4392024-08-12T22:30:38.290ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:880 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
4402024-08-12T22:30:38.290ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 3264660 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:36284 req_id = bda83cd0-e05a-419d-81bd-a575f277f896 response_code = 200 uri = /omicron-physical-disks
4412024-08-12T22:30:38.321ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
4422024-08-12T22:30:38.321ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34635
4432024-08-12T22:30:38.321ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
4442024-08-12T22:30:38.321ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
4452024-08-12T22:30:38.380ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
4462024-08-12T22:30:38.380ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4472024-08-12T22:30:38.438ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
4482024-08-12T22:30:38.438ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4492024-08-12T22:30:38.487ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/6e18d78c-496f-48cb-b07e-b18dbc665f73): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:229 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
4502024-08-12T22:30:38.495ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
4512024-08-12T22:30:38.495ZINFOSledAgent (ServiceManager): No ledger in /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:188
4522024-08-12T22:30:38.495ZINFOSledAgent (ServiceManager): No ledger in /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:188
4532024-08-12T22:30:38.495ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
4542024-08-12T22:30:38.537ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
4552024-08-12T22:30:38.537ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
4562024-08-12T22:30:38.537ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
4572024-08-12T22:30:38.593ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f file = illumos-utils/src/zone.rs:316
4582024-08-12T22:30:38.597ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d file = illumos-utils/src/zone.rs:316
4592024-08-12T22:30:38.601ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504 file = illumos-utils/src/zone.rs:316
4602024-08-12T22:30:38.614ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f file = illumos-utils/src/zone.rs:354
4612024-08-12T22:30:38.618ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d file = illumos-utils/src/zone.rs:354
4622024-08-12T22:30:38.621ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504 file = illumos-utils/src/zone.rs:354
4632024-08-12T22:30:39.489ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/6e18d78c-496f-48cb-b07e-b18dbc665f73): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:229 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
4642024-08-12T22:30:40.132ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4652024-08-12T22:30:40.132ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
4662024-08-12T22:30:40.132ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d
4672024-08-12T22:30:40.615ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4682024-08-12T22:30:40.615ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
4692024-08-12T22:30:41.097ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4702024-08-12T22:30:41.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d") failed: Property not found. retry in 73.21034ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d
4712024-08-12T22:30:41.104ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504
4722024-08-12T22:30:41.108ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
4732024-08-12T22:30:41.108ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f
4742024-08-12T22:30:41.188ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d") failed: Property not found. retry in 133.687952ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d
4752024-08-12T22:30:41.231ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504") failed: Property not found. retry in 50.812242ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504
4762024-08-12T22:30:41.237ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f") failed: Property not found. retry in 67.658299ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f
4772024-08-12T22:30:41.289ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504") failed: Property not found. retry in 105.428419ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504
4782024-08-12T22:30:41.366ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f") failed: Property not found. retry in 124.13729ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f
4792024-08-12T22:30:41.384ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d") failed: Property not found. retry in 133.101046ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d
4802024-08-12T22:30:41.426ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504") failed: Property not found. retry in 167.911773ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504
4812024-08-12T22:30:41.537ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f") failed: Property not found. retry in 261.25728ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f
4822024-08-12T22:30:41.554ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d") failed: Property not found. retry in 462.908149ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d
4832024-08-12T22:30:41.612ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504") failed: Property not found. retry in 275.670042ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504
4842024-08-12T22:30:41.810ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f") failed: Property not found. retry in 574.255496ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f
4852024-08-12T22:30:41.896ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504") failed: Property not found. retry in 1.082955336s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504
4862024-08-12T22:30:42.024ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d") failed: Property not found. retry in 473.43503ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d
4872024-08-12T22:30:42.394ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f") failed: Property not found. retry in 1.193367889s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f
4882024-08-12T22:30:42.506ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d") failed: Property not found. retry in 1.037345404s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d
4892024-08-12T22:30:42.990ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504") failed: Property not found. retry in 1.144480301s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504
4902024-08-12T22:30:43.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d") failed: Property not found. retry in 964.49469ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d
4912024-08-12T22:30:43.595ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f") failed: Property not found. retry in 673.284984ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f
4922024-08-12T22:30:44.031ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4008
4932024-08-12T22:30:44.031ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4018 gateway = fd00:1122:3344:101::1
4942024-08-12T22:30:44.037ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4097
4952024-08-12T22:30:44.142ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504") failed: Property not found. retry in 642.805028ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504
4962024-08-12T22:30:44.163ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4145
4972024-08-12T22:30:44.163ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4172
4982024-08-12T22:30:44.203ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4185
4992024-08-12T22:30:44.203ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4049
5002024-08-12T22:30:44.257ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4091
5012024-08-12T22:30:44.257ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4152 rack_subnet = fd00:1122:3344::
5022024-08-12T22:30:44.277ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4163
5032024-08-12T22:30:44.277ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4205
5042024-08-12T22:30:44.279ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f") failed: Property not found. retry in 1.342194498s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f
5052024-08-12T22:30:44.366ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4237
5062024-08-12T22:30:44.366ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4243
5072024-08-12T22:30:44.490ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4284
5082024-08-12T22:30:44.490ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:366
5092024-08-12T22:30:44.526ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d") failed: Property not found. retry in 1.319423697s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d
5102024-08-12T22:30:44.526ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/6e18d78c-496f-48cb-b07e-b18dbc665f73): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/nexus.rs:571 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
5112024-08-12T22:30:44.527ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:414
5122024-08-12T22:30:44.528ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::2]:57789
5132024-08-12T22:30:44.528ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::2]:57789 req_id = 19341f07-36c4-4a0d-be0a-786fbea62f13 response_code = 200 uri = /sled-identifiers
5142024-08-12T22:30:44.558ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:675
5152024-08-12T22:30:44.559ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:713
5162024-08-12T22:30:44.559ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:716 uptime = 0
5172024-08-12T22:30:44.559ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: None,\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:436
5182024-08-12T22:30:44.594ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
5192024-08-12T22:30:44.614ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
5202024-08-12T22:30:44.632ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:405
5212024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:601 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5222024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:622 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5232024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:601 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5242024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:622 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5252024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:601 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5262024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:622 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5272024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:601 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5282024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:622 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5292024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:601 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5302024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:622 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5312024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:601 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5322024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:622 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5332024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:601 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5342024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:622 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5352024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:601 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5362024-08-12T22:30:44.632ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:622 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5372024-08-12T22:30:44.633ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:601 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5382024-08-12T22:30:44.633ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:622 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5392024-08-12T22:30:44.633ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:601 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5402024-08-12T22:30:44.633ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:622 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5412024-08-12T22:30:44.633ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:601 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5422024-08-12T22:30:44.633ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:622 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5432024-08-12T22:30:44.794ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504") failed: Property not found. retry in 826.009754ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504
5442024-08-12T22:30:45.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f") failed: Property not found. retry in 1.265709252s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f
5452024-08-12T22:30:45.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504") failed: Property not found. retry in 1.352951853s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504
5462024-08-12T22:30:45.855ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d") failed: Property not found. retry in 1.443852179s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d
5472024-08-12T22:30:47.315ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_internal_dns_f4c5d120-d2bb-4d71-a0b6-86945be45b9f
5482024-08-12T22:30:47.315ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_internal_dns_c0f640e3-627c-4a08-96bc-34d0a8af670d
5492024-08-12T22:30:47.315ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_internal_dns_3d896039-f4f3-4ec9-99e7-c74d19508504
5502024-08-12T22:30:47.315ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5512024-08-12T22:30:47.316ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5522024-08-12T22:30:47.316ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 8994601 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:34635 req_id = f3f5f166-cace-43d9-863f-47340492d817 response_code = 204 uri = /omicron-zones
5532024-08-12T22:30:47.318ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5542024-08-12T22:30:47.369ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5552024-08-12T22:30:47.417ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5562024-08-12T22:30:47.474ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:591
5572024-08-12T22:30:47.474ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5582024-08-12T22:30:47.504ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5592024-08-12T22:30:47.506ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5602024-08-12T22:30:47.506ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5612024-08-12T22:30:47.535ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
5622024-08-12T22:30:47.535ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43245
5632024-08-12T22:30:47.535ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
5642024-08-12T22:30:47.535ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
5652024-08-12T22:30:47.555ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
5662024-08-12T22:30:47.555ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5672024-08-12T22:30:47.573ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
5682024-08-12T22:30:47.573ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5692024-08-12T22:30:47.590ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
5702024-08-12T22:30:47.590ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
5712024-08-12T22:30:47.590ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
5722024-08-12T22:30:47.590ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
5732024-08-12T22:30:47.601ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5742024-08-12T22:30:47.632ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5752024-08-12T22:30:47.633ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5762024-08-12T22:30:47.633ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5772024-08-12T22:30:47.662ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 152, 188]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
5782024-08-12T22:30:47.662ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1233 zone_type = boundary_ntp
5792024-08-12T22:30:47.663ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:83
5802024-08-12T22:30:47.663ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:90
5812024-08-12T22:30:47.664ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:118
5822024-08-12T22:30:47.664ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
5832024-08-12T22:30:47.685ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265 file = illumos-utils/src/zone.rs:316
5842024-08-12T22:30:47.704ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265 file = illumos-utils/src/zone.rs:354
5852024-08-12T22:30:48.340ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5862024-08-12T22:30:48.340ZINFOSledAgent (ServiceManager): Booting oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265
5872024-08-12T22:30:48.470ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265") failed: Property not found. retry in 40.463835ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265
5882024-08-12T22:30:48.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265") failed: Property not found. retry in 148.356439ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265
5892024-08-12T22:30:48.720ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265") failed: Property not found. retry in 244.822028ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265
5902024-08-12T22:30:48.977ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265") failed: Property not found. retry in 284.845658ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265
5912024-08-12T22:30:49.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265") failed: Property not found. retry in 428.24442ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265
5922024-08-12T22:30:49.722ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265") failed: Property not found. retry in 1.266278871s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265
5932024-08-12T22:30:50.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265") failed: Property not found. retry in 703.782155ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265
5942024-08-12T22:30:51.711ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265") failed: Property not found. retry in 1.354829663s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265
5952024-08-12T22:30:53.074ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265") failed: Property not found. retry in 1.43567281s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265
5962024-08-12T22:30:54.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265
5972024-08-12T22:30:54.528ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5982024-08-12T22:30:54.529ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5992024-08-12T22:30:54.529ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 6994231 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:43245 req_id = c1b2e300-d36d-4c4e-88a7-63261e037445 response_code = 204 uri = /omicron-zones
6002024-08-12T22:30:54.531ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:632
6012024-08-12T22:30:54.560ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6022024-08-12T22:30:54.560ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58047
6032024-08-12T22:30:54.568ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3712
6042024-08-12T22:30:54.569ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 8376 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:58047 req_id = f0dc107f-4838-4573-bca1-cd40a63bd640 response_code = 500 uri = /timesync
6052024-08-12T22:30:54.572ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6062024-08-12T22:30:54.662ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6072024-08-12T22:30:54.662ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59538
6082024-08-12T22:30:58.671ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3712
6092024-08-12T22:30:58.671ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 4008839 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59538 req_id = 32746bee-1313-4d5f-b861-0ce9e21b265a response_code = 500 uri = /timesync
6102024-08-12T22:30:58.675ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6112024-08-12T22:30:58.884ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6122024-08-12T22:30:58.884ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:39206
6132024-08-12T22:30:58.892ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3712
6142024-08-12T22:30:58.893ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 8526 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:39206 req_id = fbcbc5e6-807d-4f4e-89db-b59f4a3cdd90 response_code = 500 uri = /timesync
6152024-08-12T22:30:58.895ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6162024-08-12T22:30:59.074ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6172024-08-12T22:30:59.074ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38191
6182024-08-12T22:31:02.336ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_14284e27-0c4a-4f5f-a1f7-0c6a1530f265': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3712
6192024-08-12T22:31:02.336ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 4008543 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:38191 req_id = a18e13a9-915f-470c-be16-5192b2d441da response_code = 500 uri = /timesync
6202024-08-12T22:31:02.338ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6212024-08-12T22:31:02.514ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6222024-08-12T22:31:02.514ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55421
6232024-08-12T22:31:02.523ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 8807 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55421 req_id = 731e8cc2-966b-46e9-a47e-1c2e2de4e378 response_code = 200 uri = /timesync
6242024-08-12T22:31:02.526ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6252024-08-12T22:31:02.526ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6262024-08-12T22:31:02.861ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6272024-08-12T22:31:02.861ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62572
6282024-08-12T22:31:02.871ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 9481 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62572 req_id = f2256448-3593-44a0-a398-f587dc98d061 response_code = 200 uri = /timesync
6292024-08-12T22:31:02.873ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6302024-08-12T22:31:02.873ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6312024-08-12T22:31:03.097ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6322024-08-12T22:31:03.097ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34606
6332024-08-12T22:31:03.117ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 20178 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:34606 req_id = c4fda32f-708c-4348-a0d3-66fc75a2d5d4 response_code = 200 uri = /timesync
6342024-08-12T22:31:03.120ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 396044326, ip_addr: 23.155.40.38, stratum: 2, ref_time: 1723501863.170463, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6352024-08-12T22:31:03.120ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1230
6362024-08-12T22:31:03.151ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
6372024-08-12T22:31:03.151ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:48429
6382024-08-12T22:31:03.151ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
6392024-08-12T22:31:03.151ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
6402024-08-12T22:31:03.172ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
6412024-08-12T22:31:03.172ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6422024-08-12T22:31:03.192ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
6432024-08-12T22:31:03.192ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6442024-08-12T22:31:03.210ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
6452024-08-12T22:31:03.210ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6462024-08-12T22:31:03.269ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
6472024-08-12T22:31:03.269ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6482024-08-12T22:31:03.331ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
6492024-08-12T22:31:03.331ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6502024-08-12T22:31:03.391ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
6512024-08-12T22:31:03.391ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6522024-08-12T22:31:03.450ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
6532024-08-12T22:31:03.450ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6542024-08-12T22:31:03.511ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
6552024-08-12T22:31:03.511ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
6562024-08-12T22:31:03.511ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
6572024-08-12T22:31:03.511ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
6582024-08-12T22:31:03.529ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 151.565208ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
6592024-08-12T22:31:03.544ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
6602024-08-12T22:31:03.544ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
6612024-08-12T22:31:03.544ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
6622024-08-12T22:31:03.544ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
6632024-08-12T22:31:03.544ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
6642024-08-12T22:31:03.544ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized), ("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized), ("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 392881 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = f1a3ba06-ff93-47c0-9ef2-48534876b9ec response_code = 500 uri = /omicron-zones
6652024-08-12T22:31:03.544ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "f1a3ba06-ff93-47c0-9ef2-48534876b9ec", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:03 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "f1a3ba06-ff93-47c0-9ef2-48534876b9ec" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 100.254058ms sled_address = [fd00:1122:3344:101::1]:12345
6662024-08-12T22:31:03.645ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
6672024-08-12T22:31:03.647ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
6682024-08-12T22:31:03.647ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
6692024-08-12T22:31:03.665ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
6702024-08-12T22:31:03.665ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6712024-08-12T22:31:03.683ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
6722024-08-12T22:31:03.683ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6732024-08-12T22:31:03.702ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
6742024-08-12T22:31:03.702ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6752024-08-12T22:31:03.720ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
6762024-08-12T22:31:03.720ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6772024-08-12T22:31:03.740ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
6782024-08-12T22:31:03.740ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6792024-08-12T22:31:03.758ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
6802024-08-12T22:31:03.758ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6812024-08-12T22:31:04.001ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
6822024-08-12T22:31:04.001ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6832024-08-12T22:31:04.019ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
6842024-08-12T22:31:04.019ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
6852024-08-12T22:31:04.019ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
6862024-08-12T22:31:04.019ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
6872024-08-12T22:31:04.050ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
6882024-08-12T22:31:04.050ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
6892024-08-12T22:31:04.050ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
6902024-08-12T22:31:04.050ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
6912024-08-12T22:31:04.050ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
6922024-08-12T22:31:04.050ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized), ("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized), ("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized), ("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 178415 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = 794cddf4-4eac-4f4b-a270-5e6482d44068 response_code = 500 uri = /omicron-zones
6932024-08-12T22:31:04.050ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "794cddf4-4eac-4f4b-a270-5e6482d44068", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:03 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "794cddf4-4eac-4f4b-a270-5e6482d44068" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 168.522772ms sled_address = [fd00:1122:3344:101::1]:12345
6942024-08-12T22:31:04.220ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
6952024-08-12T22:31:04.220ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
6962024-08-12T22:31:04.220ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
6972024-08-12T22:31:04.239ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
6982024-08-12T22:31:04.239ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6992024-08-12T22:31:04.257ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
7002024-08-12T22:31:04.257ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7012024-08-12T22:31:04.275ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7022024-08-12T22:31:04.275ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7032024-08-12T22:31:04.294ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7042024-08-12T22:31:04.294ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7052024-08-12T22:31:04.313ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7062024-08-12T22:31:04.313ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7072024-08-12T22:31:04.339ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7082024-08-12T22:31:04.339ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7092024-08-12T22:31:04.357ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7102024-08-12T22:31:04.357ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7112024-08-12T22:31:04.375ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
7122024-08-12T22:31:04.375ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
7132024-08-12T22:31:04.376ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
7142024-08-12T22:31:04.376ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
7152024-08-12T22:31:04.408ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
7162024-08-12T22:31:04.408ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
7172024-08-12T22:31:04.408ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
7182024-08-12T22:31:04.408ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
7192024-08-12T22:31:04.408ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
7202024-08-12T22:31:04.408ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized), ("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized), ("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 188150 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = 22fd538a-de2b-4970-866d-03b51bd4d02a response_code = 500 uri = /omicron-zones
7212024-08-12T22:31:04.408ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "22fd538a-de2b-4970-866d-03b51bd4d02a", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:03 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "22fd538a-de2b-4970-866d-03b51bd4d02a" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 177.087056ms sled_address = [fd00:1122:3344:101::1]:12345
7222024-08-12T22:31:04.587ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
7232024-08-12T22:31:04.587ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
7242024-08-12T22:31:04.587ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
7252024-08-12T22:31:04.607ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
7262024-08-12T22:31:04.607ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7272024-08-12T22:31:04.625ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
7282024-08-12T22:31:04.625ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7292024-08-12T22:31:04.642ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7302024-08-12T22:31:04.642ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7312024-08-12T22:31:04.661ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7322024-08-12T22:31:04.661ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7332024-08-12T22:31:04.679ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7342024-08-12T22:31:04.679ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7352024-08-12T22:31:04.697ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7362024-08-12T22:31:04.697ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7372024-08-12T22:31:04.714ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7382024-08-12T22:31:04.714ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7392024-08-12T22:31:04.731ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
7402024-08-12T22:31:04.731ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
7412024-08-12T22:31:04.731ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
7422024-08-12T22:31:04.731ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
7432024-08-12T22:31:04.763ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
7442024-08-12T22:31:04.763ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
7452024-08-12T22:31:04.763ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
7462024-08-12T22:31:04.763ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
7472024-08-12T22:31:04.763ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
7482024-08-12T22:31:04.763ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized), ("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized), ("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 176028 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = 9e42d520-bb7a-45f6-aa3b-f1f390ebd5e2 response_code = 500 uri = /omicron-zones
7492024-08-12T22:31:04.763ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9e42d520-bb7a-45f6-aa3b-f1f390ebd5e2", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9e42d520-bb7a-45f6-aa3b-f1f390ebd5e2" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 130.069128ms sled_address = [fd00:1122:3344:101::1]:12345
7502024-08-12T22:31:04.894ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
7512024-08-12T22:31:04.895ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
7522024-08-12T22:31:04.895ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
7532024-08-12T22:31:04.913ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
7542024-08-12T22:31:04.913ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7552024-08-12T22:31:04.930ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
7562024-08-12T22:31:04.930ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7572024-08-12T22:31:04.948ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7582024-08-12T22:31:04.948ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7592024-08-12T22:31:04.966ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7602024-08-12T22:31:04.967ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7612024-08-12T22:31:04.984ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7622024-08-12T22:31:04.984ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7632024-08-12T22:31:05.003ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7642024-08-12T22:31:05.003ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7652024-08-12T22:31:05.021ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7662024-08-12T22:31:05.021ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7672024-08-12T22:31:05.038ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
7682024-08-12T22:31:05.038ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
7692024-08-12T22:31:05.038ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
7702024-08-12T22:31:05.038ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
7712024-08-12T22:31:05.068ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
7722024-08-12T22:31:05.068ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
7732024-08-12T22:31:05.068ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
7742024-08-12T22:31:05.068ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
7752024-08-12T22:31:05.068ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
7762024-08-12T22:31:05.068ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized), ("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized), ("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 173632 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = e9cf5c8c-f6fd-43f2-afe2-846b54328884 response_code = 500 uri = /omicron-zones
7772024-08-12T22:31:05.068ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "e9cf5c8c-f6fd-43f2-afe2-846b54328884", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "e9cf5c8c-f6fd-43f2-afe2-846b54328884" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 298.146744ms sled_address = [fd00:1122:3344:101::1]:12345
7782024-08-12T22:31:05.368ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
7792024-08-12T22:31:05.368ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
7802024-08-12T22:31:05.368ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
7812024-08-12T22:31:05.388ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
7822024-08-12T22:31:05.388ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7832024-08-12T22:31:05.406ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
7842024-08-12T22:31:05.406ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7852024-08-12T22:31:05.423ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7862024-08-12T22:31:05.423ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7872024-08-12T22:31:05.441ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7882024-08-12T22:31:05.441ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7892024-08-12T22:31:05.459ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7902024-08-12T22:31:05.459ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7912024-08-12T22:31:05.477ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7922024-08-12T22:31:05.477ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7932024-08-12T22:31:05.495ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
7942024-08-12T22:31:05.495ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7952024-08-12T22:31:05.513ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
7962024-08-12T22:31:05.513ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
7972024-08-12T22:31:05.513ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
7982024-08-12T22:31:05.513ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
7992024-08-12T22:31:05.544ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
8002024-08-12T22:31:05.544ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
8012024-08-12T22:31:05.544ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
8022024-08-12T22:31:05.544ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
8032024-08-12T22:31:05.544ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
8042024-08-12T22:31:05.544ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized), ("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized), ("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 176615 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = 9692ad98-6c90-4228-95ec-5bbc61c82459 response_code = 500 uri = /omicron-zones
8052024-08-12T22:31:05.544ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9692ad98-6c90-4228-95ec-5bbc61c82459", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9692ad98-6c90-4228-95ec-5bbc61c82459" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 133.211555ms sled_address = [fd00:1122:3344:101::1]:12345
8062024-08-12T22:31:05.679ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
8072024-08-12T22:31:05.679ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
8082024-08-12T22:31:05.679ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
8092024-08-12T22:31:05.699ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
8102024-08-12T22:31:05.699ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8112024-08-12T22:31:05.717ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
8122024-08-12T22:31:05.717ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8132024-08-12T22:31:05.737ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8142024-08-12T22:31:05.737ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8152024-08-12T22:31:05.755ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8162024-08-12T22:31:05.755ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8172024-08-12T22:31:05.774ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8182024-08-12T22:31:05.774ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8192024-08-12T22:31:05.791ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8202024-08-12T22:31:05.791ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8212024-08-12T22:31:05.809ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8222024-08-12T22:31:05.810ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8232024-08-12T22:31:05.827ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
8242024-08-12T22:31:05.827ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
8252024-08-12T22:31:05.827ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
8262024-08-12T22:31:05.827ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
8272024-08-12T22:31:05.859ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
8282024-08-12T22:31:05.859ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
8292024-08-12T22:31:05.859ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
8302024-08-12T22:31:05.859ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
8312024-08-12T22:31:05.859ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
8322024-08-12T22:31:05.859ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized), ("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 179956 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = d3b2c74e-da15-4935-afa4-e8acb12aa5dd response_code = 500 uri = /omicron-zones
8332024-08-12T22:31:05.859ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "d3b2c74e-da15-4935-afa4-e8acb12aa5dd", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:05 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "d3b2c74e-da15-4935-afa4-e8acb12aa5dd" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 408.95967ms sled_address = [fd00:1122:3344:101::1]:12345
8342024-08-12T22:31:06.270ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
8352024-08-12T22:31:06.271ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
8362024-08-12T22:31:06.271ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
8372024-08-12T22:31:06.292ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
8382024-08-12T22:31:06.292ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8392024-08-12T22:31:06.310ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
8402024-08-12T22:31:06.310ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8412024-08-12T22:31:06.328ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8422024-08-12T22:31:06.328ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8432024-08-12T22:31:06.346ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8442024-08-12T22:31:06.346ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8452024-08-12T22:31:06.363ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8462024-08-12T22:31:06.363ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8472024-08-12T22:31:06.381ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8482024-08-12T22:31:06.381ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8492024-08-12T22:31:06.400ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8502024-08-12T22:31:06.400ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8512024-08-12T22:31:06.418ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
8522024-08-12T22:31:06.418ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
8532024-08-12T22:31:06.418ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
8542024-08-12T22:31:06.419ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
8552024-08-12T22:31:06.451ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
8562024-08-12T22:31:06.451ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
8572024-08-12T22:31:06.451ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
8582024-08-12T22:31:06.451ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
8592024-08-12T22:31:06.451ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
8602024-08-12T22:31:06.451ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized), ("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized), ("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 179971 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = b28beb19-b51a-455b-b7f2-d928b6eb82e9 response_code = 500 uri = /omicron-zones
8612024-08-12T22:31:06.451ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "b28beb19-b51a-455b-b7f2-d928b6eb82e9", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "b28beb19-b51a-455b-b7f2-d928b6eb82e9" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 421.761605ms sled_address = [fd00:1122:3344:101::1]:12345
8622024-08-12T22:31:06.875ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
8632024-08-12T22:31:06.875ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
8642024-08-12T22:31:06.875ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
8652024-08-12T22:31:06.896ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
8662024-08-12T22:31:06.896ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8672024-08-12T22:31:06.915ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
8682024-08-12T22:31:06.915ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8692024-08-12T22:31:06.933ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8702024-08-12T22:31:06.933ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8712024-08-12T22:31:06.952ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8722024-08-12T22:31:06.952ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8732024-08-12T22:31:06.970ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8742024-08-12T22:31:06.970ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8752024-08-12T22:31:06.989ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8762024-08-12T22:31:06.989ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8772024-08-12T22:31:07.008ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8782024-08-12T22:31:07.008ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8792024-08-12T22:31:07.026ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
8802024-08-12T22:31:07.026ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
8812024-08-12T22:31:07.026ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
8822024-08-12T22:31:07.026ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
8832024-08-12T22:31:07.058ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
8842024-08-12T22:31:07.058ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
8852024-08-12T22:31:07.058ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
8862024-08-12T22:31:07.058ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
8872024-08-12T22:31:07.058ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
8882024-08-12T22:31:07.058ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized), ("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized), ("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 182670 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = 4a6a1999-c1ea-49b8-9633-d8ef6c3653a0 response_code = 500 uri = /omicron-zones
8892024-08-12T22:31:07.058ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "4a6a1999-c1ea-49b8-9633-d8ef6c3653a0", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "4a6a1999-c1ea-49b8-9633-d8ef6c3653a0" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 266.162022ms sled_address = [fd00:1122:3344:101::1]:12345
8902024-08-12T22:31:07.326ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
8912024-08-12T22:31:07.327ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
8922024-08-12T22:31:07.327ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
8932024-08-12T22:31:07.345ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
8942024-08-12T22:31:07.346ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8952024-08-12T22:31:07.363ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
8962024-08-12T22:31:07.363ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8972024-08-12T22:31:07.379ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
8982024-08-12T22:31:07.379ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8992024-08-12T22:31:07.397ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9002024-08-12T22:31:07.397ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9012024-08-12T22:31:07.416ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9022024-08-12T22:31:07.416ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9032024-08-12T22:31:07.435ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9042024-08-12T22:31:07.435ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9052024-08-12T22:31:07.455ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9062024-08-12T22:31:07.455ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9072024-08-12T22:31:07.474ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
9082024-08-12T22:31:07.474ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
9092024-08-12T22:31:07.474ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
9102024-08-12T22:31:07.474ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
9112024-08-12T22:31:07.506ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
9122024-08-12T22:31:07.506ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
9132024-08-12T22:31:07.506ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
9142024-08-12T22:31:07.506ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
9152024-08-12T22:31:07.506ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
9162024-08-12T22:31:07.507ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized), ("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized), ("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 179402 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = 5784d6e2-8a90-4fb3-95a0-737c11b382f3 response_code = 500 uri = /omicron-zones
9172024-08-12T22:31:07.507ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "5784d6e2-8a90-4fb3-95a0-737c11b382f3", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:07 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "5784d6e2-8a90-4fb3-95a0-737c11b382f3" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 694.174232ms sled_address = [fd00:1122:3344:101::1]:12345
9182024-08-12T22:31:08.204ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
9192024-08-12T22:31:08.204ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
9202024-08-12T22:31:08.204ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
9212024-08-12T22:31:08.226ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
9222024-08-12T22:31:08.226ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9232024-08-12T22:31:08.244ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
9242024-08-12T22:31:08.244ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9252024-08-12T22:31:08.262ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9262024-08-12T22:31:08.262ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9272024-08-12T22:31:08.280ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9282024-08-12T22:31:08.280ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9292024-08-12T22:31:08.298ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9302024-08-12T22:31:08.298ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9312024-08-12T22:31:08.316ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9322024-08-12T22:31:08.316ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9332024-08-12T22:31:08.334ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9342024-08-12T22:31:08.335ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9352024-08-12T22:31:08.353ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
9362024-08-12T22:31:08.353ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
9372024-08-12T22:31:08.353ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
9382024-08-12T22:31:08.353ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
9392024-08-12T22:31:08.387ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
9402024-08-12T22:31:08.387ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
9412024-08-12T22:31:08.387ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
9422024-08-12T22:31:08.387ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
9432024-08-12T22:31:08.387ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
9442024-08-12T22:31:08.387ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized), ("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized), ("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 182288 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = 9cdcec14-b858-4168-8368-84394b4c3139 response_code = 500 uri = /omicron-zones
9452024-08-12T22:31:08.387ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9cdcec14-b858-4168-8368-84394b4c3139", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9cdcec14-b858-4168-8368-84394b4c3139" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 448.779301ms sled_address = [fd00:1122:3344:101::1]:12345
9462024-08-12T22:31:08.838ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
9472024-08-12T22:31:08.838ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
9482024-08-12T22:31:08.838ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
9492024-08-12T22:31:08.858ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
9502024-08-12T22:31:08.858ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9512024-08-12T22:31:08.879ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
9522024-08-12T22:31:08.879ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9532024-08-12T22:31:08.899ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9542024-08-12T22:31:08.899ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9552024-08-12T22:31:08.918ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9562024-08-12T22:31:08.918ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9572024-08-12T22:31:08.936ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9582024-08-12T22:31:08.936ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9592024-08-12T22:31:08.955ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9602024-08-12T22:31:08.955ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9612024-08-12T22:31:08.973ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9622024-08-12T22:31:08.973ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9632024-08-12T22:31:08.992ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
9642024-08-12T22:31:08.992ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
9652024-08-12T22:31:08.992ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
9662024-08-12T22:31:08.992ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
9672024-08-12T22:31:09.026ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
9682024-08-12T22:31:09.026ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
9692024-08-12T22:31:09.026ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
9702024-08-12T22:31:09.026ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
9712024-08-12T22:31:09.026ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
9722024-08-12T22:31:09.027ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized), ("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 187964 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = fcb27bb9-83ab-4ca8-927e-35c8b5508e21 response_code = 500 uri = /omicron-zones
9732024-08-12T22:31:09.027ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "fcb27bb9-83ab-4ca8-927e-35c8b5508e21", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "fcb27bb9-83ab-4ca8-927e-35c8b5508e21" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 750.21622ms sled_address = [fd00:1122:3344:101::1]:12345
9742024-08-12T22:31:09.779ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
9752024-08-12T22:31:09.780ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
9762024-08-12T22:31:09.780ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
9772024-08-12T22:31:09.801ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
9782024-08-12T22:31:09.801ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9792024-08-12T22:31:09.819ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
9802024-08-12T22:31:09.819ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9812024-08-12T22:31:09.837ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9822024-08-12T22:31:09.837ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9832024-08-12T22:31:09.855ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9842024-08-12T22:31:09.855ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9852024-08-12T22:31:09.875ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9862024-08-12T22:31:09.875ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9872024-08-12T22:31:09.894ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9882024-08-12T22:31:09.894ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9892024-08-12T22:31:09.913ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
9902024-08-12T22:31:09.913ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9912024-08-12T22:31:09.931ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
9922024-08-12T22:31:09.931ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
9932024-08-12T22:31:09.932ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
9942024-08-12T22:31:09.932ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
9952024-08-12T22:31:09.964ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
9962024-08-12T22:31:09.964ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
9972024-08-12T22:31:09.964ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
9982024-08-12T22:31:09.964ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
9992024-08-12T22:31:09.964ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
10002024-08-12T22:31:09.964ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized), ("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 183729 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = a17b6cfe-3c3f-457c-a73e-1c600e874a5b response_code = 500 uri = /omicron-zones
10012024-08-12T22:31:09.964ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "a17b6cfe-3c3f-457c-a73e-1c600e874a5b", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "a17b6cfe-3c3f-457c-a73e-1c600e874a5b" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 1.115756916s sled_address = [fd00:1122:3344:101::1]:12345
10022024-08-12T22:31:11.085ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
10032024-08-12T22:31:11.086ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
10042024-08-12T22:31:11.086ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
10052024-08-12T22:31:11.107ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
10062024-08-12T22:31:11.107ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10072024-08-12T22:31:11.125ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
10082024-08-12T22:31:11.125ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10092024-08-12T22:31:11.143ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
10102024-08-12T22:31:11.143ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
10112024-08-12T22:31:11.160ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
10122024-08-12T22:31:11.160ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
10132024-08-12T22:31:11.179ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
10142024-08-12T22:31:11.179ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
10152024-08-12T22:31:11.197ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
10162024-08-12T22:31:11.197ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
10172024-08-12T22:31:11.216ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
10182024-08-12T22:31:11.216ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
10192024-08-12T22:31:11.234ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
10202024-08-12T22:31:11.234ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
10212024-08-12T22:31:11.234ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
10222024-08-12T22:31:11.234ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
10232024-08-12T22:31:11.265ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
10242024-08-12T22:31:11.265ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
10252024-08-12T22:31:11.265ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
10262024-08-12T22:31:11.265ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
10272024-08-12T22:31:11.265ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3114 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
10282024-08-12T22:31:11.266ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30", TimeNotSynchronized), ("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018", TimeNotSynchronized), ("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983", TimeNotSynchronized), ("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1", TimeNotSynchronized), ("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a", TimeNotSynchronized)] file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:902 latency_us = 180323 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = a2e33e1e-8cf3-4c65-970b-0880345b7895 response_code = 500 uri = /omicron-zones
10292024-08-12T22:31:11.266ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "a2e33e1e-8cf3-4c65-970b-0880345b7895", "content-length": "124", "date": "Mon, 12 Aug 2024 22:31:11 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "a2e33e1e-8cf3-4c65-970b-0880345b7895" } file = sled-agent/src/rack_setup/service.rs:475 retry_after = 751.200918ms sled_address = [fd00:1122:3344:101::1]:12345
10302024-08-12T22:31:12.020ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
10312024-08-12T22:31:12.020ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
10322024-08-12T22:31:12.020ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
10332024-08-12T22:31:12.041ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
10342024-08-12T22:31:12.041ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10352024-08-12T22:31:12.061ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
10362024-08-12T22:31:12.061ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10372024-08-12T22:31:12.079ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
10382024-08-12T22:31:12.079ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
10392024-08-12T22:31:12.096ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
10402024-08-12T22:31:12.096ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
10412024-08-12T22:31:12.114ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
10422024-08-12T22:31:12.114ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
10432024-08-12T22:31:12.131ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
10442024-08-12T22:31:12.131ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
10452024-08-12T22:31:12.148ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
10462024-08-12T22:31:12.148ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
10472024-08-12T22:31:12.166ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
10482024-08-12T22:31:12.166ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
10492024-08-12T22:31:12.166ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
10502024-08-12T22:31:12.166ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
10512024-08-12T22:31:12.248ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
10522024-08-12T22:31:12.248ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
10532024-08-12T22:31:12.248ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
10542024-08-12T22:31:12.248ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
10552024-08-12T22:31:12.248ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
10562024-08-12T22:31:12.342ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1 file = illumos-utils/src/zone.rs:316
10572024-08-12T22:31:12.346ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a file = illumos-utils/src/zone.rs:316
10582024-08-12T22:31:12.350ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30 file = illumos-utils/src/zone.rs:316
10592024-08-12T22:31:12.354ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983 file = illumos-utils/src/zone.rs:316
10602024-08-12T22:31:12.359ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018 file = illumos-utils/src/zone.rs:316
10612024-08-12T22:31:12.364ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1 file = illumos-utils/src/zone.rs:354
10622024-08-12T22:31:12.368ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a file = illumos-utils/src/zone.rs:354
10632024-08-12T22:31:12.373ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30 file = illumos-utils/src/zone.rs:354
10642024-08-12T22:31:12.377ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983 file = illumos-utils/src/zone.rs:354
10652024-08-12T22:31:12.382ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018 file = illumos-utils/src/zone.rs:354
10662024-08-12T22:31:13.700ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:58911
10672024-08-12T22:31:13.701ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 78 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:58911 req_id = 659d03c1-fb5b-44eb-a240-69938399ba8f response_code = 200 uri = /baseboard
10682024-08-12T22:31:15.359ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='1d05f4a7-d996-4766-bd87-c722a5eff018'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10692024-08-12T22:31:15.360ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
10702024-08-12T22:31:15.365ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='ed89774a-25e8-48f6-b3ad-dcd6acb70983'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10712024-08-12T22:31:15.365ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
10722024-08-12T22:31:15.373ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='70027bfb-5a03-4f63-ae74-d647fff67eb1'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10732024-08-12T22:31:15.373ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
10742024-08-12T22:31:15.381ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10752024-08-12T22:31:15.382ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
10762024-08-12T22:31:15.387ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='6e18a319-e02f-4c83-a62c-8cd5eec5ac30'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10772024-08-12T22:31:15.387ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
10782024-08-12T22:31:15.508ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 65.499072ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
10792024-08-12T22:31:15.516ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1") failed: Property not found. retry in 33.789362ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
10802024-08-12T22:31:15.522ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 44.414066ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
10812024-08-12T22:31:15.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a") failed: Property not found. retry in 53.343484ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
10822024-08-12T22:31:15.536ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30") failed: Property not found. retry in 31.816003ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
10832024-08-12T22:31:15.558ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1") failed: Property not found. retry in 71.557289ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
10842024-08-12T22:31:15.660ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 112.93542ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
10852024-08-12T22:31:15.695ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1") failed: Property not found. retry in 273.875554ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
10862024-08-12T22:31:15.877ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a") failed: Property not found. retry in 99.561537ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
10872024-08-12T22:31:16.418ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30") failed: Property not found. retry in 105.732722ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
10882024-08-12T22:31:16.538ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 50.874925ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
10892024-08-12T22:31:16.737ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 172.62222ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
10902024-08-12T22:31:16.985ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1") failed: Property not found. retry in 524.914214ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
10912024-08-12T22:31:17.445ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a") failed: Property not found. retry in 208.815416ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
10922024-08-12T22:31:17.993ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30") failed: Property not found. retry in 265.71716ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
10932024-08-12T22:31:18.020ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 149.684597ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
10942024-08-12T22:31:18.029ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 252.851889ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
10952024-08-12T22:31:18.038ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1") failed: Property not found. retry in 782.880807ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
10962024-08-12T22:31:18.048ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a") failed: Property not found. retry in 223.662182ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
10972024-08-12T22:31:18.181ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 351.215566ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
10982024-08-12T22:31:18.284ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30") failed: Property not found. retry in 359.788089ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
10992024-08-12T22:31:18.301ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a") failed: Property not found. retry in 1.101045631s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
11002024-08-12T22:31:18.345ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 816.843912ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11012024-08-12T22:31:18.545ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 475.552464ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11022024-08-12T22:31:18.657ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30") failed: Property not found. retry in 603.833305ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
11032024-08-12T22:31:18.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1") failed: Property not found. retry in 1.140516915s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
11042024-08-12T22:31:19.002ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 462.01152ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
11052024-08-12T22:31:19.034ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 1.077251851s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11062024-08-12T22:31:19.175ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 688.504019ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11072024-08-12T22:31:19.274ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30") failed: Property not found. retry in 751.650506ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
11082024-08-12T22:31:19.416ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a") failed: Property not found. retry in 719.4673ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
11092024-08-12T22:31:19.875ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 929.97993ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11102024-08-12T22:31:19.991ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1") failed: Property not found. retry in 807.467045ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
11112024-08-12T22:31:20.037ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30") failed: Property not found. retry in 1.311277983s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
11122024-08-12T22:31:20.124ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 1.206387878s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11132024-08-12T22:31:20.148ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a") failed: Property not found. retry in 1.001335756s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
11142024-08-12T22:31:20.811ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1") failed: Property not found. retry in 1.234271237s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
11152024-08-12T22:31:20.819ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 950.340824ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11162024-08-12T22:31:21.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a") failed: Property not found. retry in 922.87464ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
11172024-08-12T22:31:21.343ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 702.705816ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11182024-08-12T22:31:21.360ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30") failed: Property not found. retry in 721.051255ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
11192024-08-12T22:31:21.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 625.095676ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11202024-08-12T22:31:22.056ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 1.485682066s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11212024-08-12T22:31:22.066ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1") failed: Property not found. retry in 996.452502ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
11222024-08-12T22:31:22.097ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30") failed: Property not found. retry in 817.612856ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
11232024-08-12T22:31:22.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a") failed: Property not found. retry in 1.023696738s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
11242024-08-12T22:31:22.419ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 1.463769501s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11252024-08-12T22:31:22.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30") failed: Property not found. retry in 1.456651274s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
11262024-08-12T22:31:23.075ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1") failed: Property not found. retry in 1.422614582s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
11272024-08-12T22:31:23.147ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a") failed: Property not found. retry in 1.328869706s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
11282024-08-12T22:31:23.554ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 1.003375053s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11292024-08-12T22:31:23.892ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 1.215923219s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11302024-08-12T22:31:24.393ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30") failed: Property not found. retry in 876.946475ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
11312024-08-12T22:31:24.487ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a") failed: Property not found. retry in 906.174157ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
11322024-08-12T22:31:24.507ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1") failed: Property not found. retry in 777.476963ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
11332024-08-12T22:31:24.567ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 1.119180749s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11342024-08-12T22:31:25.118ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 1.109086467s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11352024-08-12T22:31:25.280ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30") failed: Property not found. retry in 1.445091794s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
11362024-08-12T22:31:25.293ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1") failed: Property not found. retry in 1.415603321s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
11372024-08-12T22:31:25.403ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a") failed: Property not found. retry in 1.491444853s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
11382024-08-12T22:31:25.698ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 1.129573719s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11392024-08-12T22:31:26.237ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 1.427361932s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11402024-08-12T22:31:26.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 1.198779116s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11412024-08-12T22:31:27.676ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 756.637792ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11422024-08-12T22:31:28.465ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 518.240309ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11432024-08-12T22:31:28.480ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 883.58207ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11442024-08-12T22:31:28.996ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 1.363020649s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11452024-08-12T22:31:29.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 705.602452ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11462024-08-12T22:31:30.098ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 1.439631468s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11472024-08-12T22:31:30.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 1.306204357s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11482024-08-12T22:31:31.548ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 585.209709ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11492024-08-12T22:31:31.686ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 718.810263ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11502024-08-12T22:31:32.142ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 1.320391104s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11512024-08-12T22:31:32.414ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 1.075196591s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11522024-08-12T22:31:33.472ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 1.198715104s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11532024-08-12T22:31:33.497ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 1.193369387s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11542024-08-12T22:31:34.507ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.476365871s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
11552024-08-12T22:31:34.680ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 1.282108597s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11562024-08-12T22:31:34.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983") failed: Property not found. retry in 1.28612896s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11572024-08-12T22:31:35.973ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018") failed: Property not found. retry in 1.089517756s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11582024-08-12T22:31:37.084ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_cockroachdb_70027bfb-5a03-4f63-ae74-d647fff67eb1
11592024-08-12T22:31:37.084ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_cockroachdb_f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a
11602024-08-12T22:31:37.084ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_cockroachdb_6e18a319-e02f-4c83-a62c-8cd5eec5ac30
11612024-08-12T22:31:37.084ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_cockroachdb_ed89774a-25e8-48f6-b3ad-dcd6acb70983
11622024-08-12T22:31:37.084ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_cockroachdb_1d05f4a7-d996-4766-bd87-c722a5eff018
11632024-08-12T22:31:37.084ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
11642024-08-12T22:31:37.087ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
11652024-08-12T22:31:37.131ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 25085211 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48429 req_id = ee18a860-1b1f-4ca0-8f82-21c061ece0e9 response_code = 204 uri = /omicron-zones
11662024-08-12T22:31:37.163ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63447
11672024-08-12T22:31:37.192ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::5]:32221 file = sled-agent/src/services.rs:3546
11682024-08-12T22:31:43.724ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3564
11692024-08-12T22:31:48.565ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3587
11702024-08-12T22:31:48.565ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 11403125 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:63447 req_id = 0c71c54e-85f7-4d56-877e-3e7b72f7b35e response_code = 204 uri = /cockroachdb
11712024-08-12T22:31:48.566ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63382
11722024-08-12T22:31:48.566ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 129 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63382 req_id = cf312cfb-8c43-4949-8e68-8eec1d41aae6 response_code = 200 uri = /baseboard
11732024-08-12T22:31:48.599ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: 3d896039-f4f3-4ec9-99e7-c74d19508504, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: c0f640e3-627c-4a08-96bc-34d0a8af670d, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 8e08ee7f-bf7a-4fc7-a495-a07295059906, kind: Service { id: 14284e27-0c4a-4f5f-a1f7-0c6a1530f265 }, name: Name("ntp-14284e27-0c4a-4f5f-a1f7-0c6a1530f265"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1d05f4a7-d996-4766-bd87-c722a5eff018, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }, OmicronZoneConfig { id: d120a386-2706-4e01-bb8c-b5d65c0f637f, underlay_address: fd00:1122:3344:101::8, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: f9638164-af79-4853-9a7e-a593f5bab52c, kind: Service { id: d120a386-2706-4e01-bb8c-b5d65c0f637f }, name: Name("external-dns-d120a386-2706-4e01-bb8c-b5d65c0f637f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 182])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 0d8c5151-3b8f-4b66-b2ec-69426d844f51, underlay_address: fd00:1122:3344:101::9, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: c6463b16-74e6-4dec-9e5a-c72d26b9fe84, kind: Service { id: 0d8c5151-3b8f-4b66-b2ec-69426d844f51 }, name: Name("external-dns-0d8c5151-3b8f-4b66-b2ec-69426d844f51"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 254])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: d512bf60-e3a1-492a-aa6e-6bf28969f94c, underlay_address: fd00:1122:3344:101::a, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: 0bc77817-8af4-413e-be1d-d96e849efaa0, kind: Service { id: d512bf60-e3a1-492a-aa6e-6bf28969f94c }, name: Name("nexus-d512bf60-e3a1-492a-aa6e-6bf28969f94c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 8e5fb63d-011e-4f8e-b416-8713dac504ae, underlay_address: fd00:1122:3344:101::b, filesystem_pool: Some(ZpoolName { id: 09108500-899b-4c15-aa1f-0c8295b5001c (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: d2375ade-1e0b-4247-ba64-7f0a959e70ac, kind: Service { id: 8e5fb63d-011e-4f8e-b416-8713dac504ae }, name: Name("nexus-8e5fb63d-011e-4f8e-b416-8713dac504ae"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 224, 235])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 4c699543-0a26-420d-8b7a-4a0104f01339, underlay_address: fd00:1122:3344:101::c, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: e2f1b87a-6b81-4706-840f-a0af0417a6fb, kind: Service { id: 4c699543-0a26-420d-8b7a-4a0104f01339 }, name: Name("nexus-4c699543-0a26-420d-8b7a-4a0104f01339"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 240])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: a3cca9d9-445e-495c-bb4b-d68f7777b482, underlay_address: fd00:1122:3344:101::d, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: edbbb800-f4c8-4980-a5c9-af163b96b017, underlay_address: fd00:1122:3344:101::e, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: 36f128c9-08fe-4583-be21-fc48e81dffcb, underlay_address: fd00:1122:3344:101::f, filesystem_pool: Some(ZpoolName { id: 65eb96fa-5b90-4e6e-968e-0b7f2a4634d3 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: e41eb0e0-3766-4ece-a1f1-18626d27f03a, underlay_address: fd00:1122:3344:101::10, filesystem_pool: Some(ZpoolName { id: ca45bffe-f952-4bbd-bcc4-be5de210718a (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: bf8f3dc7-bf47-4284-a733-d0439d4503a7, underlay_address: fd00:1122:3344:101::11, filesystem_pool: Some(ZpoolName { id: 09108500-899b-4c15-aa1f-0c8295b5001c (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: b5dc5628-b92d-454b-bdaf-72d2635fbb46, underlay_address: fd00:1122:3344:101::12, filesystem_pool: Some(ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External } } } }, OmicronZoneConfig { id: 16576d37-bf1d-487a-91e1-85c50bec0856, underlay_address: fd00:1122:3344:101::13, filesystem_pool: Some(ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External } } } }, OmicronZoneConfig { id: 05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5, underlay_address: fd00:1122:3344:101::14, filesystem_pool: Some(ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1014862e-7c76-4226-a04a-ec71dd6d76fe, underlay_address: fd00:1122:3344:101::15, filesystem_pool: Some(ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External } } } }, OmicronZoneConfig { id: 0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e, underlay_address: fd00:1122:3344:101::16, filesystem_pool: Some(ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External } } } }, OmicronZoneConfig { id: 29f4c7b7-f5fa-4187-a8a7-9146b88f3181, underlay_address: fd00:1122:3344:101::17, filesystem_pool: Some(ZpoolName { id: ca45bffe-f952-4bbd-bcc4-be5de210718a (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ca45bffe-f952-4bbd-bcc4-be5de210718a (zpool), kind: External } } } }, OmicronZoneConfig { id: 9d505dd0-fa65-4b6d-a5d4-fcd351438ace, underlay_address: fd00:1122:3344:101::18, filesystem_pool: Some(ZpoolName { id: 65eb96fa-5b90-4e6e-968e-0b7f2a4634d3 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 65eb96fa-5b90-4e6e-968e-0b7f2a4634d3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 623adf4c-10ae-489c-9ee8-4a585f5237df, underlay_address: fd00:1122:3344:101::19, filesystem_pool: Some(ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External } } } }, OmicronZoneConfig { id: cbd6df64-6ddb-46ea-8711-9dcf16a956e3, underlay_address: fd00:1122:3344:101::1a, filesystem_pool: Some(ZpoolName { id: 09108500-899b-4c15-aa1f-0c8295b5001c (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 09108500-899b-4c15-aa1f-0c8295b5001c (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
11742024-08-12T22:31:48.599ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:33576
11752024-08-12T22:31:48.600ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
11762024-08-12T22:31:48.600ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 3d896039-f4f3-4ec9-99e7-c74d19508504, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: InternalDns }, responder: ... }
11772024-08-12T22:31:48.621ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
11782024-08-12T22:31:48.621ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f4c5d120-d2bb-4d71-a0b6-86945be45b9f, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: InternalDns }, responder: ... }
11792024-08-12T22:31:48.639ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
11802024-08-12T22:31:48.639ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: c0f640e3-627c-4a08-96bc-34d0a8af670d, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
11812024-08-12T22:31:48.658ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
11822024-08-12T22:31:48.658ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 6e18a319-e02f-4c83-a62c-8cd5eec5ac30, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: CockroachDb }, responder: ... }
11832024-08-12T22:31:48.677ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
11842024-08-12T22:31:48.677ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: ed89774a-25e8-48f6-b3ad-dcd6acb70983, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
11852024-08-12T22:31:48.695ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
11862024-08-12T22:31:48.695ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1d05f4a7-d996-4766-bd87-c722a5eff018, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
11872024-08-12T22:31:48.715ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
11882024-08-12T22:31:48.715ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 70027bfb-5a03-4f63-ae74-d647fff67eb1, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
11892024-08-12T22:31:48.735ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:405
11902024-08-12T22:31:48.735ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: f63ea97c-f4c7-446e-90a6-8dab7b9a2a8a, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
11912024-08-12T22:31:48.754ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d120a386-2706-4e01-bb8c-b5d65c0f637f, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
11922024-08-12T22:31:48.754ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: d120a386-2706-4e01-bb8c-b5d65c0f637f, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: ExternalDns }, responder: ... }
11932024-08-12T22:31:48.827ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0d8c5151-3b8f-4b66-b2ec-69426d844f51, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:405
11942024-08-12T22:31:48.827ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 0d8c5151-3b8f-4b66-b2ec-69426d844f51, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
11952024-08-12T22:31:48.901ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: edbbb800-f4c8-4980-a5c9-af163b96b017, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:405
11962024-08-12T22:31:48.901ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: edbbb800-f4c8-4980-a5c9-af163b96b017, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: Clickhouse }, responder: ... }
11972024-08-12T22:31:48.970ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b5dc5628-b92d-454b-bdaf-72d2635fbb46, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:405
11982024-08-12T22:31:48.970ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: b5dc5628-b92d-454b-bdaf-72d2635fbb46, dataset_name: DatasetName { pool_name: ZpoolName { id: 537dde5b-089b-4e20-9ef8-bf10127d7dce (zpool), kind: External }, kind: Crucible }, responder: ... }
11992024-08-12T22:31:49.032ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 16576d37-bf1d-487a-91e1-85c50bec0856, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:405
12002024-08-12T22:31:49.032ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 16576d37-bf1d-487a-91e1-85c50bec0856, dataset_name: DatasetName { pool_name: ZpoolName { id: 90f5d185-2468-460f-8d20-ecb50330e694 (zpool), kind: External }, kind: Crucible }, responder: ... }
12012024-08-12T22:31:49.102ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:405
12022024-08-12T22:31:49.102ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5, dataset_name: DatasetName { pool_name: ZpoolName { id: 9ac4fc8d-4a30-4017-bbdc-a392e988e5c6 (zpool), kind: External }, kind: Crucible }, responder: ... }
12032024-08-12T22:31:49.178ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1014862e-7c76-4226-a04a-ec71dd6d76fe, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:405
12042024-08-12T22:31:49.178ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 1014862e-7c76-4226-a04a-ec71dd6d76fe, dataset_name: DatasetName { pool_name: ZpoolName { id: fab24dee-2d14-4ad6-8dd1-967cd0dd455c (zpool), kind: External }, kind: Crucible }, responder: ... }
12052024-08-12T22:31:49.249ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:405
12062024-08-12T22:31:49.249ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e, dataset_name: DatasetName { pool_name: ZpoolName { id: 37abb3ae-dbff-4acb-a865-55b4ed0374e9 (zpool), kind: External }, kind: Crucible }, responder: ... }
12072024-08-12T22:31:49.321ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 29f4c7b7-f5fa-4187-a8a7-9146b88f3181, dataset_name: DatasetName { pool_name: ZpoolName { id: ca45bffe-f952-4bbd-bcc4-be5de210718a (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:405
12082024-08-12T22:31:49.321ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 29f4c7b7-f5fa-4187-a8a7-9146b88f3181, dataset_name: DatasetName { pool_name: ZpoolName { id: ca45bffe-f952-4bbd-bcc4-be5de210718a (zpool), kind: External }, kind: Crucible }, responder: ... }
12092024-08-12T22:31:49.385ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9d505dd0-fa65-4b6d-a5d4-fcd351438ace, dataset_name: DatasetName { pool_name: ZpoolName { id: 65eb96fa-5b90-4e6e-968e-0b7f2a4634d3 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:405
12102024-08-12T22:31:49.385ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 9d505dd0-fa65-4b6d-a5d4-fcd351438ace, dataset_name: DatasetName { pool_name: ZpoolName { id: 65eb96fa-5b90-4e6e-968e-0b7f2a4634d3 (zpool), kind: External }, kind: Crucible }, responder: ... }
12112024-08-12T22:31:49.454ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 623adf4c-10ae-489c-9ee8-4a585f5237df, dataset_name: DatasetName { pool_name: ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:405
12122024-08-12T22:31:49.454ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: 623adf4c-10ae-489c-9ee8-4a585f5237df, dataset_name: DatasetName { pool_name: ZpoolName { id: ef93b2f7-5e79-4766-bedb-2c6d5ba93538 (zpool), kind: External }, kind: Crucible }, responder: ... }
12132024-08-12T22:31:49.520ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: cbd6df64-6ddb-46ea-8711-9dcf16a956e3, dataset_name: DatasetName { pool_name: ZpoolName { id: 09108500-899b-4c15-aa1f-0c8295b5001c (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:405
12142024-08-12T22:31:49.520ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:772 request = NewFilesystemRequest { dataset_id: cbd6df64-6ddb-46ea-8711-9dcf16a956e3, dataset_name: DatasetName { pool_name: ZpoolName { id: 09108500-899b-4c15-aa1f-0c8295b5001c (zpool), kind: External }, kind: Crucible }, responder: ... }
12152024-08-12T22:31:49.587ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12162024-08-12T22:31:49.587ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
12172024-08-12T22:31:49.588ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
12182024-08-12T22:31:49.588ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12192024-08-12T22:31:49.705ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
12202024-08-12T22:31:49.705ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12212024-08-12T22:31:49.705ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12222024-08-12T22:31:49.706ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
12232024-08-12T22:31:49.734ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
12242024-08-12T22:31:49.734ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12252024-08-12T22:31:49.734ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12262024-08-12T22:31:49.734ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12272024-08-12T22:31:49.735ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
12282024-08-12T22:31:49.760ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12292024-08-12T22:31:49.760ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12302024-08-12T22:31:49.960ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
12312024-08-12T22:31:49.960ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12322024-08-12T22:31:49.960ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12332024-08-12T22:31:49.960ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12342024-08-12T22:31:49.960ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12352024-08-12T22:31:49.960ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12362024-08-12T22:31:49.984ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12372024-08-12T22:31:49.984ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
12382024-08-12T22:31:49.984ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482 file = illumos-utils/src/zone.rs:316
12392024-08-12T22:31:49.990ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a file = illumos-utils/src/zone.rs:316
12402024-08-12T22:31:49.995ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017 file = illumos-utils/src/zone.rs:316
12412024-08-12T22:31:49.999ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7 file = illumos-utils/src/zone.rs:316
12422024-08-12T22:31:50.004ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46 file = illumos-utils/src/zone.rs:316
12432024-08-12T22:31:50.009ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df file = illumos-utils/src/zone.rs:316
12442024-08-12T22:31:50.014ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3 file = illumos-utils/src/zone.rs:316
12452024-08-12T22:31:50.155ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482 file = illumos-utils/src/zone.rs:354
12462024-08-12T22:31:50.160ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a file = illumos-utils/src/zone.rs:354
12472024-08-12T22:31:50.165ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017 file = illumos-utils/src/zone.rs:354
12482024-08-12T22:31:50.169ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7 file = illumos-utils/src/zone.rs:354
12492024-08-12T22:31:50.174ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46 file = illumos-utils/src/zone.rs:354
12502024-08-12T22:31:50.179ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df file = illumos-utils/src/zone.rs:354
12512024-08-12T22:31:50.184ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3 file = illumos-utils/src/zone.rs:354
12522024-08-12T22:31:50.189ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856 file = illumos-utils/src/zone.rs:316
12532024-08-12T22:31:50.194ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5 file = illumos-utils/src/zone.rs:316
12542024-08-12T22:31:50.198ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb file = illumos-utils/src/zone.rs:316
12552024-08-12T22:31:50.203ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e file = illumos-utils/src/zone.rs:316
12562024-08-12T22:31:50.209ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe file = illumos-utils/src/zone.rs:316
12572024-08-12T22:31:50.215ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181 file = illumos-utils/src/zone.rs:316
12582024-08-12T22:31:50.221ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace file = illumos-utils/src/zone.rs:316
12592024-08-12T22:31:50.229ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856 file = illumos-utils/src/zone.rs:354
12602024-08-12T22:31:50.235ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5 file = illumos-utils/src/zone.rs:354
12612024-08-12T22:31:50.241ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb file = illumos-utils/src/zone.rs:354
12622024-08-12T22:31:50.247ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e file = illumos-utils/src/zone.rs:354
12632024-08-12T22:31:50.254ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe file = illumos-utils/src/zone.rs:354
12642024-08-12T22:31:50.261ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181 file = illumos-utils/src/zone.rs:354
12652024-08-12T22:31:50.268ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace file = illumos-utils/src/zone.rs:354
12662024-08-12T22:31:50.327ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
12672024-08-12T22:31:50.406ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
12682024-08-12T22:31:50.470ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
12692024-08-12T22:31:50.552ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
12702024-08-12T22:31:50.604ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
12712024-08-12T22:31:50.606ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
12722024-08-12T22:31:50.606ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
12732024-08-12T22:31:50.654ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 244, 182]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
12742024-08-12T22:31:50.654ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1233 zone_type = external_dns
12752024-08-12T22:31:50.656ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
12762024-08-12T22:31:50.656ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
12772024-08-12T22:31:50.704ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 224, 235]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
12782024-08-12T22:31:50.704ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1233 zone_type = nexus
12792024-08-12T22:31:50.705ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
12802024-08-12T22:31:50.706ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
12812024-08-12T22:31:50.757ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 201, 213]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
12822024-08-12T22:31:50.757ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1233 zone_type = nexus
12832024-08-12T22:31:50.759ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
12842024-08-12T22:31:50.759ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
12852024-08-12T22:31:50.808ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 200, 240]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
12862024-08-12T22:31:50.808ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1233 zone_type = nexus
12872024-08-12T22:31:50.812ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
12882024-08-12T22:31:50.812ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
12892024-08-12T22:31:50.861ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 229, 254]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
12902024-08-12T22:31:50.861ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1233 zone_type = external_dns
12912024-08-12T22:31:50.862ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:83
12922024-08-12T22:31:50.862ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:90
12932024-08-12T22:31:50.862ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:83
12942024-08-12T22:31:50.862ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:90
12952024-08-12T22:31:50.862ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:83
12962024-08-12T22:31:50.862ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:90
12972024-08-12T22:31:50.862ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:83
12982024-08-12T22:31:50.862ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:90
12992024-08-12T22:31:50.866ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:118
13002024-08-12T22:31:50.867ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
13012024-08-12T22:31:50.915ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:118
13022024-08-12T22:31:50.918ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
13032024-08-12T22:31:50.918ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:118
13042024-08-12T22:31:50.922ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
13052024-08-12T22:31:50.923ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:118
13062024-08-12T22:31:50.927ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:83
13072024-08-12T22:31:50.927ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
13082024-08-12T22:31:50.927ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:90
13092024-08-12T22:31:51.047ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.273872697s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
13102024-08-12T22:31:51.154ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:118
13112024-08-12T22:31:51.159ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
13122024-08-12T22:31:51.159ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f file = illumos-utils/src/zone.rs:316
13132024-08-12T22:31:51.169ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae file = illumos-utils/src/zone.rs:316
13142024-08-12T22:31:51.177ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c file = illumos-utils/src/zone.rs:316
13152024-08-12T22:31:51.185ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339 file = illumos-utils/src/zone.rs:316
13162024-08-12T22:31:51.231ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f file = illumos-utils/src/zone.rs:354
13172024-08-12T22:31:51.237ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae file = illumos-utils/src/zone.rs:354
13182024-08-12T22:31:51.244ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c file = illumos-utils/src/zone.rs:354
13192024-08-12T22:31:51.251ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339 file = illumos-utils/src/zone.rs:354
13202024-08-12T22:31:51.258ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51 file = illumos-utils/src/zone.rs:316
13212024-08-12T22:31:51.332ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51 file = illumos-utils/src/zone.rs:354
13222024-08-12T22:31:52.659ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13232024-08-12T22:31:52.660ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
13242024-08-12T22:31:52.669ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13252024-08-12T22:31:52.669ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
13262024-08-12T22:31:52.676ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13272024-08-12T22:31:52.676ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
13282024-08-12T22:31:53.244ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_ca45bffe-f952-4bbd-bcc4-be5de210718a/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='47f19933-1fb8-4b99-971d-6099ab35831b'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13292024-08-12T22:31:53.244ZINFOSledAgent (ServiceManager): Booting oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
13302024-08-12T22:31:53.251ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_ef93b2f7-5e79-4766-bedb-2c6d5ba93538/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='eccac96c-3ce2-4a56-825a-c04a60c7c6f9'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13312024-08-12T22:31:53.252ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_09108500-899b-4c15-aa1f-0c8295b5001c/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='c0007468-2eda-44ba-9d5a-19e1a214c711'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13322024-08-12T22:31:53.252ZINFOSledAgent (ServiceManager): Booting oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
13332024-08-12T22:31:53.258ZINFOSledAgent (ServiceManager): Booting oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
13342024-08-12T22:31:53.264ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_90f5d185-2468-460f-8d20-ecb50330e694/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='a515c748-ab58-4818-9028-5500be5ff937'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13352024-08-12T22:31:53.266ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_37abb3ae-dbff-4acb-a865-55b4ed0374e9/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='4167b6af-b7fd-4141-900b-9b4cf694c755'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13362024-08-12T22:31:53.266ZINFOSledAgent (ServiceManager): Booting oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
13372024-08-12T22:31:53.273ZINFOSledAgent (ServiceManager): Booting oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
13382024-08-12T22:31:53.281ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_fab24dee-2d14-4ad6-8dd1-967cd0dd455c/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='d1d89a7c-7b73-49df-89b8-4ab96b34f1dc'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13392024-08-12T22:31:53.281ZINFOSledAgent (ServiceManager): Booting oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
13402024-08-12T22:31:53.288ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='a3cca9d9-445e-495c-bb4b-d68f7777b482'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13412024-08-12T22:31:53.288ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
13422024-08-12T22:31:53.307ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_65eb96fa-5b90-4e6e-968e-0b7f2a4634d3/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='7461bf24-833c-42b0-8a21-3b3616a23919'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13432024-08-12T22:31:53.307ZINFOSledAgent (ServiceManager): Booting oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
13442024-08-12T22:31:53.658ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_537dde5b-089b-4e20-9ef8-bf10127d7dce/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='11f54d17-c205-4631-a99f-224ac4513d90'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13452024-08-12T22:31:53.661ZINFOSledAgent (ServiceManager): Booting oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
13462024-08-12T22:31:53.727ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 52.564222ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
13472024-08-12T22:31:53.791ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 34.107278ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
13482024-08-12T22:31:53.871ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 39.382189ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
13492024-08-12T22:31:53.885ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_9ac4fc8d-4a30-4017-bbdc-a392e988e5c6/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='d8c272f0-49d0-455d-b605-5b2b07cbb276'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13502024-08-12T22:31:53.897ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 62.001995ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
13512024-08-12T22:31:53.924ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 85.56772ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
13522024-08-12T22:31:53.959ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 122.024405ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
13532024-08-12T22:31:53.988ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 73.031732ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
13542024-08-12T22:31:54.012ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 63.718406ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
13552024-08-12T22:31:54.012ZINFOSledAgent (ServiceManager): Booting oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
13562024-08-12T22:31:54.064ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 238.742926ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
13572024-08-12T22:31:54.108ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 66.005233ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
13582024-08-12T22:31:54.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 49.432773ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
13592024-08-12T22:31:54.501ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 63.377706ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
13602024-08-12T22:31:54.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 37.518617ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
13612024-08-12T22:31:54.822ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 27.484404ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
13622024-08-12T22:31:55.014ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 142.622618ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
13632024-08-12T22:31:55.069ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 116.887266ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
13642024-08-12T22:31:55.127ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 54.216524ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
13652024-08-12T22:31:55.188ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 27.946194ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
13662024-08-12T22:31:55.457ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 65.502769ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
13672024-08-12T22:31:55.522ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 140.765169ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
13682024-08-12T22:31:55.959ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 381.999507ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
13692024-08-12T22:31:56.011ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 57.063343ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
13702024-08-12T22:31:56.090ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 72.533718ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
13712024-08-12T22:31:56.144ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 65.904336ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
13722024-08-12T22:31:56.215ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 125.535927ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
13732024-08-12T22:31:56.252ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 58.915165ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
13742024-08-12T22:31:56.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 597.295379ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
13752024-08-12T22:31:56.288ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 438.147202ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
13762024-08-12T22:31:56.307ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 100.753091ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
13772024-08-12T22:31:56.331ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 50.046303ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
13782024-08-12T22:31:56.343ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 184.088334ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
13792024-08-12T22:31:56.367ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 200.701876ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
13802024-08-12T22:31:56.399ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 36.644727ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
13812024-08-12T22:31:56.414ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 443.728522ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
13822024-08-12T22:31:56.427ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 233.205626ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
13832024-08-12T22:31:56.439ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 139.955862ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
13842024-08-12T22:31:56.453ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 288.417314ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
13852024-08-12T22:31:56.464ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 119.98951ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
13862024-08-12T22:31:56.477ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 260.845253ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
13872024-08-12T22:31:56.488ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 267.381874ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
13882024-08-12T22:31:56.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 220.097518ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
13892024-08-12T22:31:56.511ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 98.739528ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
13902024-08-12T22:31:56.539ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 248.192328ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
13912024-08-12T22:31:56.583ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 569.087092ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
13922024-08-12T22:31:56.595ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 210.061406ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
13932024-08-12T22:31:56.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 268.073096ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
13942024-08-12T22:31:56.621ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 279.206496ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
13952024-08-12T22:31:56.675ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 493.683493ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
13962024-08-12T22:31:56.733ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 573.014606ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
13972024-08-12T22:31:56.746ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 593.705177ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
13982024-08-12T22:31:56.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 407.183569ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
13992024-08-12T22:31:56.769ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 523.11678ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
14002024-08-12T22:31:56.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 450.111249ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
14012024-08-12T22:31:56.802ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 530.229654ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
14022024-08-12T22:31:56.819ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 841.546172ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
14032024-08-12T22:31:56.874ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 502.289709ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
14042024-08-12T22:31:56.886ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 923.148509ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
14052024-08-12T22:31:56.900ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 669.968315ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
14062024-08-12T22:31:56.914ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 425.039399ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
14072024-08-12T22:31:57.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 412.855243ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
14082024-08-12T22:31:57.183ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 582.621046ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
14092024-08-12T22:31:57.198ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 445.754337ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
14102024-08-12T22:31:57.250ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 593.125973ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
14112024-08-12T22:31:57.310ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.074445472s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
14122024-08-12T22:31:57.324ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 777.070419ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
14132024-08-12T22:31:57.349ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 1.373430755s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
14142024-08-12T22:31:57.363ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.488549487s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
14152024-08-12T22:31:57.375ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 524.483553ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
14162024-08-12T22:31:57.390ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.401872629s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
14172024-08-12T22:31:57.500ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14182024-08-12T22:31:57.500ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
14192024-08-12T22:31:57.526ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14202024-08-12T22:31:57.526ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
14212024-08-12T22:31:57.588ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 580.532046ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
14222024-08-12T22:31:57.605ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 640.212526ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
14232024-08-12T22:31:57.662ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.007384133s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
14242024-08-12T22:31:57.679ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 577.667583ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
14252024-08-12T22:31:57.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 942.928996ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
14262024-08-12T22:31:57.825ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 846.159558ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
14272024-08-12T22:31:57.855ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.388350064s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
14282024-08-12T22:31:57.914ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.450232138s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
14292024-08-12T22:31:57.983ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 51.703404ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
14302024-08-12T22:31:57.999ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 50.187186ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
14312024-08-12T22:31:58.047ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 122.781358ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
14322024-08-12T22:31:58.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 149.719981ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
14332024-08-12T22:31:58.113ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 639.788338ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
14342024-08-12T22:31:58.181ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 875.577746ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
14352024-08-12T22:31:58.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 292.145445ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
14362024-08-12T22:31:58.227ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 225.526444ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
14372024-08-12T22:31:58.257ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 706.605961ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
14382024-08-12T22:31:58.267ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 763.233687ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
14392024-08-12T22:31:58.398ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.054067456s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
14402024-08-12T22:31:58.465ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 388.668555ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
14412024-08-12T22:31:58.499ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 326.305344ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
14422024-08-12T22:31:58.621ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14432024-08-12T22:31:58.621ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14442024-08-12T22:31:58.621ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14452024-08-12T22:31:58.622ZINFOSledAgent (ServiceManager): Booting oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
14462024-08-12T22:31:58.630ZINFOSledAgent (ServiceManager): Booting oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
14472024-08-12T22:31:58.636ZINFOSledAgent (ServiceManager): Booting oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
14482024-08-12T22:31:58.682ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.275092512s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
14492024-08-12T22:31:58.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 975.41789ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
14502024-08-12T22:31:58.735ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 666.330726ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
14512024-08-12T22:31:58.747ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 677.86662ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
14522024-08-12T22:31:58.765ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 1.248556212s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
14532024-08-12T22:31:58.804ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 987.615045ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
14542024-08-12T22:31:58.830ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 44.205639ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
14552024-08-12T22:31:58.841ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 1.031527965s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
14562024-08-12T22:31:58.848ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 35.591444ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
14572024-08-12T22:31:58.863ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.348464147s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
14582024-08-12T22:31:58.873ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 539.056365ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
14592024-08-12T22:31:58.884ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 43.532509ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
14602024-08-12T22:31:58.892ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 147.321778ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
14612024-08-12T22:31:58.902ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 117.595924ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
14622024-08-12T22:31:58.939ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 115.038552ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
14632024-08-12T22:31:58.976ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 1.126802647s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
14642024-08-12T22:31:59.031ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 160.05003ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
14652024-08-12T22:31:59.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 744.986294ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
14662024-08-12T22:31:59.056ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 217.247525ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
14672024-08-12T22:31:59.067ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 238.751689ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
14682024-08-12T22:31:59.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 804.646869ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
14692024-08-12T22:31:59.206ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 399.269336ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
14702024-08-12T22:31:59.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 552.608858ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
14712024-08-12T22:31:59.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 578.829369ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
14722024-08-12T22:31:59.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 450.532163ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
14732024-08-12T22:31:59.377ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 985.03281ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
14742024-08-12T22:31:59.413ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 829.824628ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
14752024-08-12T22:31:59.425ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 637.569034ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
14762024-08-12T22:31:59.436ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 1.167191747s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
14772024-08-12T22:31:59.469ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 936.275185ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
14782024-08-12T22:31:59.617ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 507.250656ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
14792024-08-12T22:31:59.681ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 1.348694489s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
14802024-08-12T22:31:59.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.095840497s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
14812024-08-12T22:31:59.803ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.188396393s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
14822024-08-12T22:31:59.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.402324169s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
14832024-08-12T22:31:59.826ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 576.892111ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
14842024-08-12T22:31:59.878ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 583.958744ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
14852024-08-12T22:31:59.890ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 1.038098207s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
14862024-08-12T22:31:59.902ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.423207154s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
14872024-08-12T22:31:59.970ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 533.379585ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
14882024-08-12T22:32:00.026ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 989.645081ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
14892024-08-12T22:32:00.076ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 943.811876ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
14902024-08-12T22:32:00.116ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 800.743804ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
14912024-08-12T22:32:00.139ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 975.577078ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
14922024-08-12T22:32:00.223ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.435823774s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
14932024-08-12T22:32:00.253ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 1.306700269s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
14942024-08-12T22:32:00.376ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 831.25213ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
14952024-08-12T22:32:00.418ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.27131397s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
14962024-08-12T22:32:00.431ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.06299805s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
14972024-08-12T22:32:00.476ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 1.236839592s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
14982024-08-12T22:32:00.517ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 829.964345ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
14992024-08-12T22:32:00.616ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 538.539604ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
15002024-08-12T22:32:00.890ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.370790907s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
15012024-08-12T22:32:00.930ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 567.466228ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
15022024-08-12T22:32:00.943ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 511.773275ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
15032024-08-12T22:32:01.006ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 937.441705ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
15042024-08-12T22:32:01.029ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 572.440069ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
15052024-08-12T22:32:01.040ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 959.328607ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
15062024-08-12T22:32:01.051ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 717.00421ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
15072024-08-12T22:32:01.128ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 649.23593ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
15082024-08-12T22:32:01.167ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 1.099607201s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
15092024-08-12T22:32:01.219ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 815.358708ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
15102024-08-12T22:32:01.231ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.461285602s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
15112024-08-12T22:32:01.339ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 547.516979ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
15122024-08-12T22:32:01.366ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.227583103s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
15132024-08-12T22:32:01.470ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 1.078028578s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
15142024-08-12T22:32:01.508ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.020613538s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
15152024-08-12T22:32:01.520ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 909.518876ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
15162024-08-12T22:32:01.572ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 671.906029ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
15172024-08-12T22:32:01.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 1.377906538s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
15182024-08-12T22:32:02.254ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.394253695s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
15192024-08-12T22:32:02.278ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.055610753s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
15202024-08-12T22:32:02.328ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.180833127s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
15212024-08-12T22:32:02.362ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 1.444442237s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
15222024-08-12T22:32:02.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 1.364037663s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
15232024-08-12T22:32:02.409ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 662.860798ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
15242024-08-12T22:32:02.428ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 642.8894ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
15252024-08-12T22:32:02.445ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 1.297077682s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
15262024-08-12T22:32:02.462ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 538.182391ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
15272024-08-12T22:32:02.480ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 583.685429ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
15282024-08-12T22:32:02.496ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 1.367164315s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
15292024-08-12T22:32:02.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 1.478647263s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
15302024-08-12T22:32:02.523ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 1.436593622s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
15312024-08-12T22:32:02.544ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 721.013471ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
15322024-08-12T22:32:02.561ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 679.747239ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
15332024-08-12T22:32:02.611ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 638.744551ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
15342024-08-12T22:32:02.709ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.43544091s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
15352024-08-12T22:32:03.013ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.196030334s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
15362024-08-12T22:32:03.077ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 637.227226ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
15372024-08-12T22:32:03.089ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 921.736182ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
15382024-08-12T22:32:03.100ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.063711135s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
15392024-08-12T22:32:03.129ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 711.284913ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
15402024-08-12T22:32:03.254ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 757.305059ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
15412024-08-12T22:32:03.267ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 884.568366ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
15422024-08-12T22:32:03.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 768.906079ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
15432024-08-12T22:32:03.346ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.260743972s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
15442024-08-12T22:32:03.523ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.481052421s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
15452024-08-12T22:32:03.663ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.121455048s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
15462024-08-12T22:32:03.728ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 1.432592753s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
15472024-08-12T22:32:03.758ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 1.481206006s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
15482024-08-12T22:32:03.780ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 500.88139ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
15492024-08-12T22:32:03.820ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 981.923019ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
15502024-08-12T22:32:03.853ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 818.120202ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
15512024-08-12T22:32:03.877ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 1.448512182s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
15522024-08-12T22:32:03.973ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 877.689747ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
15532024-08-12T22:32:04.000ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 1.315022073s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
15542024-08-12T22:32:04.023ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 720.651721ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
15552024-08-12T22:32:04.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 864.625194ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
15562024-08-12T22:32:04.065ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 617.861522ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
15572024-08-12T22:32:04.161ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 577.690707ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
15582024-08-12T22:32:04.174ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 928.737919ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
15592024-08-12T22:32:04.187ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 951.892515ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
15602024-08-12T22:32:04.226ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.415086665s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
15612024-08-12T22:32:04.294ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 709.622896ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
15622024-08-12T22:32:04.622ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.247043517s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
15632024-08-12T22:32:04.686ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 1.19398923s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
15642024-08-12T22:32:04.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.091827894s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
15652024-08-12T22:32:04.752ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 771.718065ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
15662024-08-12T22:32:04.766ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 817.158449ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
15672024-08-12T22:32:04.799ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.296626867s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
15682024-08-12T22:32:04.815ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 837.183832ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
15692024-08-12T22:32:04.865ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 720.753965ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
15702024-08-12T22:32:04.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.087790089s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
15712024-08-12T22:32:05.015ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.445071086s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
15722024-08-12T22:32:05.026ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 636.44711ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
15732024-08-12T22:32:05.114ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.346587136s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
15742024-08-12T22:32:05.149ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.21087749s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
15752024-08-12T22:32:05.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 567.278581ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
15762024-08-12T22:32:05.250ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 953.000482ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
15772024-08-12T22:32:05.326ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 1.008176712s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
15782024-08-12T22:32:05.337ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 1.297307967s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
15792024-08-12T22:32:05.536ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.135278742s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
15802024-08-12T22:32:05.592ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 928.580492ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
15812024-08-12T22:32:05.603ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 876.360422ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
15822024-08-12T22:32:05.655ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.064209035s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
15832024-08-12T22:32:05.666ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 1.06032964s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
15842024-08-12T22:32:05.677ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 1.391248918s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
15852024-08-12T22:32:05.750ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 975.221641ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
15862024-08-12T22:32:05.804ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 824.391093ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
15872024-08-12T22:32:05.882ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.105634786s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
15882024-08-12T22:32:05.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 889.963367ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
15892024-08-12T22:32:06.015ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.425355529s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
15902024-08-12T22:32:06.106ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 619.191428ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
15912024-08-12T22:32:06.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 1.113157103s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
15922024-08-12T22:32:06.344ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 667.022907ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
15932024-08-12T22:32:06.375ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.348495945s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
15942024-08-12T22:32:06.472ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.436154997s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
15952024-08-12T22:32:06.485ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.349817704s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
15962024-08-12T22:32:06.497ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 1.200734847s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
15972024-08-12T22:32:06.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 588.358133ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
15982024-08-12T22:32:06.637ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.215011319s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
15992024-08-12T22:32:06.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 1.26932727s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
16002024-08-12T22:32:06.681ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.238249177s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
16012024-08-12T22:32:06.728ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.044333288s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
16022024-08-12T22:32:06.735ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 1.317342056s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
16032024-08-12T22:32:06.743ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 846.383919ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
16042024-08-12T22:32:06.751ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 686.233779ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
16052024-08-12T22:32:06.792ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 958.785271ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
16062024-08-12T22:32:06.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.09584093s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
16072024-08-12T22:32:07.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 1.059225877s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
16082024-08-12T22:32:07.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 1.2780015s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
16092024-08-12T22:32:07.126ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 1.424037005s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
16102024-08-12T22:32:07.335ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 1.221420748s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
16112024-08-12T22:32:07.446ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 1.46455822s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
16122024-08-12T22:32:07.454ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 637.391055ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
16132024-08-12T22:32:07.599ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 873.238505ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
16142024-08-12T22:32:07.707ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 1.114914067s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
16152024-08-12T22:32:07.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 679.134618ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
16162024-08-12T22:32:07.759ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 1.056343562s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
16172024-08-12T22:32:07.780ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 767.514559ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
16182024-08-12T22:32:07.843ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.026846751s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
16192024-08-12T22:32:07.860ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 977.51605ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
16202024-08-12T22:32:07.915ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.481243687s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
16212024-08-12T22:32:07.922ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 1.448689989s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
16222024-08-12T22:32:07.930ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 725.536161ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
16232024-08-12T22:32:08.061ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 808.013783ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
16242024-08-12T22:32:08.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 615.610051ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
16252024-08-12T22:32:08.099ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.082799559s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
16262024-08-12T22:32:08.107ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 681.235065ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
16272024-08-12T22:32:08.365ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 880.190442ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
16282024-08-12T22:32:08.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 510.35971ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
16292024-08-12T22:32:08.482ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.345444059s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
16302024-08-12T22:32:08.557ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.168644063s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
16312024-08-12T22:32:08.564ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 689.50975ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
16322024-08-12T22:32:08.572ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 1.410325578s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
16332024-08-12T22:32:08.665ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 663.879676ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
16342024-08-12T22:32:08.712ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 518.194131ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
16352024-08-12T22:32:08.799ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.015832596s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
16362024-08-12T22:32:08.807ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.308763131s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
16372024-08-12T22:32:08.824ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 819.206986ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
16382024-08-12T22:32:08.832ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 674.917051ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
16392024-08-12T22:32:08.846ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.484051908s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
16402024-08-12T22:32:08.879ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 1.323054716s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
16412024-08-12T22:32:08.886ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.084986878s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
16422024-08-12T22:32:08.919ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 629.096429ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
16432024-08-12T22:32:08.939ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 529.88928ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
16442024-08-12T22:32:09.197ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.473581947s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
16452024-08-12T22:32:09.241ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 637.932331ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
16462024-08-12T22:32:09.256ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 786.298721ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
16472024-08-12T22:32:09.264ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 754.502183ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
16482024-08-12T22:32:09.340ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 612.693929ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
16492024-08-12T22:32:09.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 966.255677ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
16502024-08-12T22:32:09.408ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 882.398988ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
16512024-08-12T22:32:09.479ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.16846828s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
16522024-08-12T22:32:09.516ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 785.745617ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
16532024-08-12T22:32:09.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 537.713825ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
16542024-08-12T22:32:09.653ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 1.024496906s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
16552024-08-12T22:32:09.736ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.387232923s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
16562024-08-12T22:32:09.838ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 900.215667ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
16572024-08-12T22:32:09.890ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 928.627182ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
16582024-08-12T22:32:09.892ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
16592024-08-12T22:32:09.893ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
16602024-08-12T22:32:09.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.045682649s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
16612024-08-12T22:32:09.981ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.087580809s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
16622024-08-12T22:32:09.991ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 1.282498756s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
16632024-08-12T22:32:10.029ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 600.52754ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
16642024-08-12T22:32:10.052ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 1.436639473s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
16652024-08-12T22:32:10.108ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 977.972345ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
16662024-08-12T22:32:10.125ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.308073055s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
16672024-08-12T22:32:10.131ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 74.30603ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
16682024-08-12T22:32:10.211ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 578.322193ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
16692024-08-12T22:32:10.219ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 142.916469ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
16702024-08-12T22:32:10.300ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.352708206s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
16712024-08-12T22:32:10.311ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 595.705244ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
16722024-08-12T22:32:10.343ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.367089062s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
16732024-08-12T22:32:10.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 1.28034869s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
16742024-08-12T22:32:10.369ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 177.649435ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
16752024-08-12T22:32:10.557ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 407.837166ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
16762024-08-12T22:32:10.639ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 1.334450386s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
16772024-08-12T22:32:10.655ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.183381337s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
16782024-08-12T22:32:10.679ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.168936912s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
16792024-08-12T22:32:10.687ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 713.134836ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
16802024-08-12T22:32:10.748ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.499402835s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
16812024-08-12T22:32:10.799ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 847.174374ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
16822024-08-12T22:32:10.863ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 843.68523ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
16832024-08-12T22:32:10.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 1.029592163s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
16842024-08-12T22:32:10.974ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 949.032093ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
16852024-08-12T22:32:11.020ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 875.404178ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
16862024-08-12T22:32:11.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 528.800065ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
16872024-08-12T22:32:11.096ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 1.039093016s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
16882024-08-12T22:32:11.131ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.262579357s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
16892024-08-12T22:32:11.284ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 597.854791ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
16902024-08-12T22:32:11.411ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 860.647474ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
16912024-08-12T22:32:11.443ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 663.278649ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
16922024-08-12T22:32:11.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 1.452550113s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
16932024-08-12T22:32:11.617ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 966.340393ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
16942024-08-12T22:32:11.648ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 1.141543135s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
16952024-08-12T22:32:11.657ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 1.343745214s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
16962024-08-12T22:32:11.667ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.092106555s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
16972024-08-12T22:32:11.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 577.61738ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
16982024-08-12T22:32:11.728ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.034913936s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
16992024-08-12T22:32:11.849ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.032642585s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
17002024-08-12T22:32:11.859ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.447468399s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
17012024-08-12T22:32:11.891ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 905.060867ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
17022024-08-12T22:32:11.904ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.018695036s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
17032024-08-12T22:32:11.933ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 660.67651ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
17042024-08-12T22:32:11.956ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 996.475585ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
17052024-08-12T22:32:11.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 1.178559271s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
17062024-08-12T22:32:12.118ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.354309147s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
17072024-08-12T22:32:12.146ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 722.567951ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
17082024-08-12T22:32:12.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.072118778s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
17092024-08-12T22:32:12.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 1.083836843s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
17102024-08-12T22:32:12.305ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 757.219152ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
17112024-08-12T22:32:12.405ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.220612107s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
17122024-08-12T22:32:12.602ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.355337832s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
17132024-08-12T22:32:12.620ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 1.089393493s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
17142024-08-12T22:32:12.774ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.24042362s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
17152024-08-12T22:32:12.785ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 550.521858ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
17162024-08-12T22:32:12.802ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 614.989676ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
17172024-08-12T22:32:12.813ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 1.20175314s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
17182024-08-12T22:32:12.882ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 1.492624702s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
17192024-08-12T22:32:12.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.18893899s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
17202024-08-12T22:32:12.937ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.233215732s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
17212024-08-12T22:32:12.966ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 540.525223ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
17222024-08-12T22:32:12.981ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 1.444476049s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
17232024-08-12T22:32:13.016ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 958.089257ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
17242024-08-12T22:32:13.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 810.214824ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
17252024-08-12T22:32:13.186ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 1.36660612s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
17262024-08-12T22:32:13.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.213830576s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
17272024-08-12T22:32:13.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.327620146s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
17282024-08-12T22:32:13.379ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 700.104877ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
17292024-08-12T22:32:13.398ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 607.067497ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
17302024-08-12T22:32:13.434ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 1.252404724s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
17312024-08-12T22:32:13.488ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.096065912s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
17322024-08-12T22:32:13.521ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 1.06885967s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
17332024-08-12T22:32:13.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.008616594s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
17342024-08-12T22:32:13.730ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 1.200763813s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
17352024-08-12T22:32:13.902ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 1.205717719s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
17362024-08-12T22:32:13.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 760.46813ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
17372024-08-12T22:32:13.986ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 532.329233ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
17382024-08-12T22:32:14.015ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 522.867312ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
17392024-08-12T22:32:14.026ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.368812898s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
17402024-08-12T22:32:14.036ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 1.105861767s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
17412024-08-12T22:32:14.092ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.314512726s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
17422024-08-12T22:32:14.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.037146702s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
17432024-08-12T22:32:14.186ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.236690013s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
17442024-08-12T22:32:14.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 680.298408ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
17452024-08-12T22:32:14.443ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 1.157801866s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
17462024-08-12T22:32:14.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 984.174713ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
17472024-08-12T22:32:14.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 1.227789855s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
17482024-08-12T22:32:14.567ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 1.026846712s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
17492024-08-12T22:32:14.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.327117095s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
17502024-08-12T22:32:14.602ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 747.538418ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
17512024-08-12T22:32:14.620ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 993.116318ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
17522024-08-12T22:32:14.665ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.23217591s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
17532024-08-12T22:32:14.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 561.075042ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
17542024-08-12T22:32:14.711ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 861.586462ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
17552024-08-12T22:32:14.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.321833601s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
17562024-08-12T22:32:14.942ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 844.357861ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
17572024-08-12T22:32:15.079ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 778.184233ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
17582024-08-12T22:32:15.118ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 506.754238ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
17592024-08-12T22:32:15.151ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.22411203s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
17602024-08-12T22:32:15.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 1.429133197s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
17612024-08-12T22:32:15.272ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.239077483s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
17622024-08-12T22:32:15.363ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 967.335901ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
17632024-08-12T22:32:15.407ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.302726833s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
17642024-08-12T22:32:15.419ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 538.466047ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
17652024-08-12T22:32:15.432ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.228531699s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
17662024-08-12T22:32:15.528ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 1.172298028s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
17672024-08-12T22:32:15.588ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 894.831107ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
17682024-08-12T22:32:15.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 794.596395ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
17692024-08-12T22:32:15.619ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 778.517356ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
17702024-08-12T22:32:15.628ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 905.628931ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
17712024-08-12T22:32:15.639ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 1.343780884s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
17722024-08-12T22:32:15.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 1.046360228s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
17732024-08-12T22:32:15.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 888.228241ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
17742024-08-12T22:32:15.866ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 552.804141ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
17752024-08-12T22:32:15.908ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.427334226s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
17762024-08-12T22:32:15.918ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.246282979s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
17772024-08-12T22:32:15.972ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 722.295548ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
17782024-08-12T22:32:16.076ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 697.097023ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
17792024-08-12T22:32:16.347ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.037072044s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
17802024-08-12T22:32:16.391ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.143435659s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
17812024-08-12T22:32:16.410ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 1.006577383s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
17822024-08-12T22:32:16.422ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 655.413848ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
17832024-08-12T22:32:16.432ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 1.448392391s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
17842024-08-12T22:32:16.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 687.615591ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
17852024-08-12T22:32:16.523ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 831.620446ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
17862024-08-12T22:32:16.551ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 850.02264ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
17872024-08-12T22:32:16.604ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 502.394983ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
17882024-08-12T22:32:16.673ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.165505234s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
17892024-08-12T22:32:16.708ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 1.186069831s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
17902024-08-12T22:32:16.719ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 824.354268ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
17912024-08-12T22:32:16.733ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 1.102284319s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
17922024-08-12T22:32:16.744ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.007132327s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
17932024-08-12T22:32:16.789ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 980.479471ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
17942024-08-12T22:32:16.858ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 1.070989202s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
17952024-08-12T22:32:16.996ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 650.770657ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
17962024-08-12T22:32:17.092ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 1.327521615s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
17972024-08-12T22:32:17.121ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 1.270731442s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
17982024-08-12T22:32:17.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.16065254s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
17992024-08-12T22:32:17.196ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 1.056963633s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
18002024-08-12T22:32:17.348ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.331666553s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
18012024-08-12T22:32:17.367ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 624.294557ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
18022024-08-12T22:32:17.393ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.452311085s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
18032024-08-12T22:32:17.412ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 1.24830521s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
18042024-08-12T22:32:17.430ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 1.391721274s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
18052024-08-12T22:32:17.545ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.474573079s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
18062024-08-12T22:32:17.556ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.050483753s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
18072024-08-12T22:32:17.657ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 610.179504ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
18082024-08-12T22:32:17.760ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 587.84395ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
18092024-08-12T22:32:17.779ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.433349445s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
18102024-08-12T22:32:17.845ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 981.824416ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
18112024-08-12T22:32:17.853ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 815.682826ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
18122024-08-12T22:32:17.890ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 1.160521315s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
18132024-08-12T22:32:17.902ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 670.046125ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
18142024-08-12T22:32:17.937ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 1.061281754s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
18152024-08-12T22:32:18.000ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 970.205967ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
18162024-08-12T22:32:18.263ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 637.082386ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
18172024-08-12T22:32:18.277ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 665.137302ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
18182024-08-12T22:32:18.346ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.365114622s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
18192024-08-12T22:32:18.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.436927092s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
18202024-08-12T22:32:18.400ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 583.463379ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
18212024-08-12T22:32:18.428ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 669.795532ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
18222024-08-12T22:32:18.581ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 1.184247656s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
18232024-08-12T22:32:18.618ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 906.421654ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
18242024-08-12T22:32:18.619ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61399
18252024-08-12T22:32:18.619ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 341 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61399 req_id = 98c42d33-4fa7-486a-92e5-b1d20cbadcd6 response_code = 200 uri = /baseboard
18262024-08-12T22:32:18.671ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 945.134174ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
18272024-08-12T22:32:18.681ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.284370966s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
18282024-08-12T22:32:18.693ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.181231486s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
18292024-08-12T22:32:18.831ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 1.131620658s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
18302024-08-12T22:32:18.839ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 888.235435ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
18312024-08-12T22:32:18.855ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 690.21475ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
18322024-08-12T22:32:18.910ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 1.25046155s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
18332024-08-12T22:32:18.951ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 623.181223ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
18342024-08-12T22:32:18.979ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.027547119s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
18352024-08-12T22:32:18.993ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 925.283558ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
18362024-08-12T22:32:19.006ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 1.216940701s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
18372024-08-12T22:32:19.028ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.11214835s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
18382024-08-12T22:32:19.060ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 889.782855ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
18392024-08-12T22:32:19.107ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 1.215280614s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
18402024-08-12T22:32:19.224ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.436561766s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
18412024-08-12T22:32:19.535ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.49825774s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
18422024-08-12T22:32:19.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.364330605s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
18432024-08-12T22:32:19.583ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 1.268578094s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
18442024-08-12T22:32:19.625ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3") failed: Property not found. retry in 1.052811867s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
18452024-08-12T22:32:19.720ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 1.050111528s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
18462024-08-12T22:32:19.737ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 1.404533432s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
18472024-08-12T22:32:19.774ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 786.620733ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
18482024-08-12T22:32:19.803ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.465180617s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
18492024-08-12T22:32:19.885ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 559.916008ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
18502024-08-12T22:32:19.928ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 738.368165ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
18512024-08-12T22:32:19.959ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 699.403225ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
18522024-08-12T22:32:19.971ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 730.880098ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
18532024-08-12T22:32:19.979ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb") failed: Property not found. retry in 1.342521724s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
18542024-08-12T22:32:20.015ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.411314228s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
18552024-08-12T22:32:20.150ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.196260195s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
18562024-08-12T22:32:20.172ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 1.339553587s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
18572024-08-12T22:32:20.233ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 696.864196ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
18582024-08-12T22:32:20.331ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 1.046738208s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
18592024-08-12T22:32:20.454ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.375797196s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
18602024-08-12T22:32:20.569ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017") failed: Property not found. retry in 1.096283937s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
18612024-08-12T22:32:20.668ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 1.051280724s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
18622024-08-12T22:32:20.677ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.111088113s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
18632024-08-12T22:32:20.685ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51") failed: Property not found. retry in 1.308342637s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
18642024-08-12T22:32:20.710ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 578.018785ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
18652024-08-12T22:32:20.780ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 854.405711ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
18662024-08-12T22:32:20.861ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 597.443892ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
18672024-08-12T22:32:20.927ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46") failed: Property not found. retry in 1.009685447s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
18682024-08-12T22:32:20.938ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df") failed: Property not found. retry in 1.267179337s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
18692024-08-12T22:32:21.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181") failed: Property not found. retry in 1.30684334s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
18702024-08-12T22:32:21.151ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482") failed: Property not found. retry in 1.445932201s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
18712024-08-12T22:32:21.279ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c") failed: Property not found. retry in 1.472759199s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
18722024-08-12T22:32:21.297ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856") failed: Property not found. retry in 1.319270669s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
18732024-08-12T22:32:21.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe") failed: Property not found. retry in 1.430254138s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
18742024-08-12T22:32:21.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f") failed: Property not found. retry in 1.048624447s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
18752024-08-12T22:32:21.436ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7") failed: Property not found. retry in 1.374628615s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
18762024-08-12T22:32:21.473ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae") failed: Property not found. retry in 1.155310128s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
18772024-08-12T22:32:21.523ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a") failed: Property not found. retry in 938.298684ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
18782024-08-12T22:32:21.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5") failed: Property not found. retry in 999.176926ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
18792024-08-12T22:32:21.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339") failed: Property not found. retry in 1.138085122s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
18802024-08-12T22:32:21.798ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e") failed: Property not found. retry in 1.010986289s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
18812024-08-12T22:32:21.845ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace") failed: Property not found. retry in 1.201800037s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
18822024-08-12T22:32:23.082ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_oximeter_a3cca9d9-445e-495c-bb4b-d68f7777b482
18832024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_crucible_pantry_e41eb0e0-3766-4ece-a1f1-18626d27f03a
18842024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_external_dns_d120a386-2706-4e01-bb8c-b5d65c0f637f
18852024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_nexus_8e5fb63d-011e-4f8e-b416-8713dac504ae
18862024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_clickhouse_edbbb800-f4c8-4980-a5c9-af163b96b017
18872024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_crucible_pantry_bf8f3dc7-bf47-4284-a733-d0439d4503a7
18882024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_crucible_b5dc5628-b92d-454b-bdaf-72d2635fbb46
18892024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_nexus_d512bf60-e3a1-492a-aa6e-6bf28969f94c
18902024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_nexus_4c699543-0a26-420d-8b7a-4a0104f01339
18912024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_crucible_623adf4c-10ae-489c-9ee8-4a585f5237df
18922024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_crucible_cbd6df64-6ddb-46ea-8711-9dcf16a956e3
18932024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_crucible_16576d37-bf1d-487a-91e1-85c50bec0856
18942024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_crucible_05cbfbfe-6fa6-4f42-8318-0cc5d14fc1c5
18952024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_crucible_pantry_36f128c9-08fe-4583-be21-fc48e81dffcb
18962024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_crucible_0e3126d1-2dfd-4ff4-a12b-e331eecb3b8e
18972024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_crucible_1014862e-7c76-4226-a04a-ec71dd6d76fe
18982024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_external_dns_0d8c5151-3b8f-4b66-b2ec-69426d844f51
18992024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_crucible_29f4c7b7-f5fa-4187-a8a7-9146b88f3181
19002024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3110 zone = oxz_crucible_9d505dd0-fa65-4b6d-a5d4-fcd351438ace
19012024-08-12T22:32:23.083ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
19022024-08-12T22:32:23.085ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
19032024-08-12T22:32:23.089ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 34489765 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:33576 req_id = e7549cd4-eb69-4ed7-9678-0fa695d89812 response_code = 204 uri = /omicron-zones
19042024-08-12T22:32:23.104ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1250
19052024-08-12T22:32:23.104ZINFOSledAgent (RSS): Writing ledger to /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
19062024-08-12T22:32:23.105ZINFOSledAgent (RSS): Writing ledger to /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
19072024-08-12T22:32:23.109ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:687
19082024-08-12T22:32:23.113ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::c]:12221 file = sled-agent/src/rack_setup/service.rs:700
19092024-08-12T22:32:23.144ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, }, ], switch: Switch0, uplink_port_fec: None, uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:831
19102024-08-12T22:32:23.145ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19112024-08-12T22:32:23.210ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19122024-08-12T22:32:23.290ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19132024-08-12T22:32:23.501ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19142024-08-12T22:32:23.740ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19152024-08-12T22:32:24.035ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19162024-08-12T22:32:24.354ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19172024-08-12T22:32:24.756ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19182024-08-12T22:32:24.991ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19192024-08-12T22:32:25.355ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19202024-08-12T22:32:25.856ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 4.941366033s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
19212024-08-12T22:32:26.061ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19222024-08-12T22:32:26.747ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19232024-08-12T22:32:27.257ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19242024-08-12T22:32:28.218ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19252024-08-12T22:32:29.700ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19262024-08-12T22:32:30.829ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 8.973100397s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/producer/src/lib.rs:424 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
19272024-08-12T22:32:31.612ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19282024-08-12T22:32:33.217ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19292024-08-12T22:32:34.343ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19302024-08-12T22:32:35.891ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/3430adde-27ec-4e0d-a703-b8e2cc54d5df/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:893
19312024-08-12T22:32:38.826ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "f9035fc5-8d4d-4544-8d35-a4af360e63a9", "content-length": "173", "date": "Mon, 12 Aug 2024 22:32:38 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"6e18d78c-496f-48cb-b07e-b18dbc665f73\"", request_id: "f9035fc5-8d4d-4544-8d35-a4af360e63a9" } file = sled-agent/src/rack_setup/service.rs:893
19322024-08-12T22:32:40.938ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "067034bf-dd75-456e-bead-73e2b648792f", "content-length": "173", "date": "Mon, 12 Aug 2024 22:32:40 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"6e18d78c-496f-48cb-b07e-b18dbc665f73\"", request_id: "067034bf-dd75-456e-bead-73e2b648792f" } file = sled-agent/src/rack_setup/service.rs:893
19332024-08-12T22:32:44.072ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "62c3128b-fa3b-453c-918f-8c449481498c", "content-length": "173", "date": "Mon, 12 Aug 2024 22:32:44 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"6e18d78c-496f-48cb-b07e-b18dbc665f73\"", request_id: "62c3128b-fa3b-453c-918f-8c449481498c" } file = sled-agent/src/rack_setup/service.rs:893
19342024-08-12T22:32:47.592ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "3f9a8dd2-a7e7-49ac-bcdc-d0d374fdb2cb", "content-length": "173", "date": "Mon, 12 Aug 2024 22:32:47 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"6e18d78c-496f-48cb-b07e-b18dbc665f73\"", request_id: "3f9a8dd2-a7e7-49ac-bcdc-d0d374fdb2cb" } file = sled-agent/src/rack_setup/service.rs:893
19352024-08-12T22:32:48.654ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:42694
19362024-08-12T22:32:48.655ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 34 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:42694 req_id = c593a795-ecc2-43ea-8d96-631f661d4e66 response_code = 200 uri = /baseboard
19372024-08-12T22:32:52.192ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "219a0ea5-59c4-459d-8415-3378e6aeb21d", "content-length": "173", "date": "Mon, 12 Aug 2024 22:32:51 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"6e18d78c-496f-48cb-b07e-b18dbc665f73\"", request_id: "219a0ea5-59c4-459d-8415-3378e6aeb21d" } file = sled-agent/src/rack_setup/service.rs:893
19382024-08-12T22:32:59.152ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "cc75fecc-514e-4eb1-8e7e-4852bbec40e8", "content-length": "173", "date": "Mon, 12 Aug 2024 22:32:59 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"6e18d78c-496f-48cb-b07e-b18dbc665f73\"", request_id: "cc75fecc-514e-4eb1-8e7e-4852bbec40e8" } file = sled-agent/src/rack_setup/service.rs:893
19392024-08-12T22:33:04.685ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:544 new_generation = 1 old_generation = 1 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
19402024-08-12T22:33:04.766ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41989
19412024-08-12T22:33:04.768ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
19422024-08-12T22:33:04.857ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 90480 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41989 req_id = ffa642d6-ca1d-4d34-bf9b-299b48e62698 response_code = 200 uri = /inventory
19432024-08-12T22:33:04.858ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
19442024-08-12T22:33:04.858ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
19452024-08-12T22:33:04.859ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
19462024-08-12T22:33:04.859ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1789 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41989 req_id = 827f37ce-20ee-4f46-ab93-4367af20bdcf response_code = 200 uri = /omicron-zones
19472024-08-12T22:33:07.126ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64019
19482024-08-12T22:33:07.127ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 340 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64019 req_id = 07c4513f-37e5-45ab-9d0a-66ca0f43fd68 response_code = 200 uri = /vpc-routes
19492024-08-12T22:33:07.150ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47220
19502024-08-12T22:33:07.151ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47220 req_id = 61af3faa-ffe2-4dc7-9901-3f2067ec4a38 response_code = 200 uri = /vpc-routes
19512024-08-12T22:33:07.203ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 2444 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64019 req_id = f1534ee1-580d-4d19-8b44-8a2781796f4d response_code = 204 uri = /vpc-routes
19522024-08-12T22:33:07.217ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 647 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47220 req_id = 7cf546c1-823a-4c6e-99f7-41a6666434f1 response_code = 204 uri = /vpc-routes
19532024-08-12T22:33:07.245ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48296
19542024-08-12T22:33:07.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 281 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48296 req_id = c065164c-336a-40ac-b0d2-7d4d48b8fdd8 response_code = 200 uri = /v2p
19552024-08-12T22:33:07.305ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62274
19562024-08-12T22:33:07.305ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62274 req_id = 9ac8f076-2299-48e8-aa4f-8277b0336df9 response_code = 200 uri = /v2p
19572024-08-12T22:33:07.812ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35092
19582024-08-12T22:33:07.812ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35092 req_id = d1572294-77fe-462b-92b3-788f62deaa6b response_code = 200 uri = /vpc-routes
19592024-08-12T22:33:07.820ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35092 req_id = 8efd1bc6-2108-4125-af78-438fa50ec44c response_code = 204 uri = /vpc-routes
19602024-08-12T22:33:07.848ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61776
19612024-08-12T22:33:07.848ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61776 req_id = ec9663dc-d521-4e5a-a672-bda50f2968a4 response_code = 200 uri = /v2p
19622024-08-12T22:33:08.745ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43783
19632024-08-12T22:33:08.747ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c6463b16-74e6-4dec-9e5a-c72d26b9fe84, kind: Service { id: 0d8c5151-3b8f-4b66-b2ec-69426d844f51 }, name: Name("external-dns-0d8c5151-3b8f-4b66-b2ec-69426d844f51"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 254])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f9638164-af79-4853-9a7e-a593f5bab52c, kind: Service { id: d120a386-2706-4e01-bb8c-b5d65c0f637f }, name: Name("external-dns-d120a386-2706-4e01-bb8c-b5d65c0f637f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 182])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0bc77817-8af4-413e-be1d-d96e849efaa0, kind: Service { id: d512bf60-e3a1-492a-aa6e-6bf28969f94c }, name: Name("nexus-d512bf60-e3a1-492a-aa6e-6bf28969f94c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d2375ade-1e0b-4247-ba64-7f0a959e70ac, kind: Service { id: 8e5fb63d-011e-4f8e-b416-8713dac504ae }, name: Name("nexus-8e5fb63d-011e-4f8e-b416-8713dac504ae"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 224, 235])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e2f1b87a-6b81-4706-840f-a0af0417a6fb, kind: Service { id: 4c699543-0a26-420d-8b7a-4a0104f01339 }, name: Name("nexus-4c699543-0a26-420d-8b7a-4a0104f01339"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 240])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19642024-08-12T22:33:08.747ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19652024-08-12T22:33:08.747ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
19662024-08-12T22:33:08.747ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19672024-08-12T22:33:08.747ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19682024-08-12T22:33:08.747ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19692024-08-12T22:33:08.747ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19702024-08-12T22:33:08.747ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1679 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43783 req_id = dc2f3de3-1290-4c8b-95b6-f9642387e93c response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19712024-08-12T22:33:08.749ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:903
19722024-08-12T22:33:08.784ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45763
19732024-08-12T22:33:08.784ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c6463b16-74e6-4dec-9e5a-c72d26b9fe84, kind: Service { id: 0d8c5151-3b8f-4b66-b2ec-69426d844f51 }, name: Name("external-dns-0d8c5151-3b8f-4b66-b2ec-69426d844f51"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 254])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f9638164-af79-4853-9a7e-a593f5bab52c, kind: Service { id: d120a386-2706-4e01-bb8c-b5d65c0f637f }, name: Name("external-dns-d120a386-2706-4e01-bb8c-b5d65c0f637f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 182])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0bc77817-8af4-413e-be1d-d96e849efaa0, kind: Service { id: d512bf60-e3a1-492a-aa6e-6bf28969f94c }, name: Name("nexus-d512bf60-e3a1-492a-aa6e-6bf28969f94c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d2375ade-1e0b-4247-ba64-7f0a959e70ac, kind: Service { id: 8e5fb63d-011e-4f8e-b416-8713dac504ae }, name: Name("nexus-8e5fb63d-011e-4f8e-b416-8713dac504ae"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 224, 235])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e2f1b87a-6b81-4706-840f-a0af0417a6fb, kind: Service { id: 4c699543-0a26-420d-8b7a-4a0104f01339 }, name: Name("nexus-4c699543-0a26-420d-8b7a-4a0104f01339"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 240])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19742024-08-12T22:33:08.784ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19752024-08-12T22:33:08.784ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
19762024-08-12T22:33:08.784ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19772024-08-12T22:33:08.784ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19782024-08-12T22:33:08.784ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19792024-08-12T22:33:08.784ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19802024-08-12T22:33:08.784ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 416 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45763 req_id = 22f406d3-f6ea-4ae7-85fe-5d5e18fb110d response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19812024-08-12T22:33:08.851ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43846
19822024-08-12T22:33:08.852ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c6463b16-74e6-4dec-9e5a-c72d26b9fe84, kind: Service { id: 0d8c5151-3b8f-4b66-b2ec-69426d844f51 }, name: Name("external-dns-0d8c5151-3b8f-4b66-b2ec-69426d844f51"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 254])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f9638164-af79-4853-9a7e-a593f5bab52c, kind: Service { id: d120a386-2706-4e01-bb8c-b5d65c0f637f }, name: Name("external-dns-d120a386-2706-4e01-bb8c-b5d65c0f637f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 182])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0bc77817-8af4-413e-be1d-d96e849efaa0, kind: Service { id: d512bf60-e3a1-492a-aa6e-6bf28969f94c }, name: Name("nexus-d512bf60-e3a1-492a-aa6e-6bf28969f94c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d2375ade-1e0b-4247-ba64-7f0a959e70ac, kind: Service { id: 8e5fb63d-011e-4f8e-b416-8713dac504ae }, name: Name("nexus-8e5fb63d-011e-4f8e-b416-8713dac504ae"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 224, 235])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e2f1b87a-6b81-4706-840f-a0af0417a6fb, kind: Service { id: 4c699543-0a26-420d-8b7a-4a0104f01339 }, name: Name("nexus-4c699543-0a26-420d-8b7a-4a0104f01339"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 240])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19832024-08-12T22:33:08.852ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19842024-08-12T22:33:08.852ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
19852024-08-12T22:33:08.852ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19862024-08-12T22:33:08.852ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19872024-08-12T22:33:08.852ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19882024-08-12T22:33:08.852ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19892024-08-12T22:33:08.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 285 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43846 req_id = 46e4b883-837f-4bf3-b3a7-09f2eb2dd8de response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19902024-08-12T22:33:08.939ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42781
19912024-08-12T22:33:08.940ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c6463b16-74e6-4dec-9e5a-c72d26b9fe84, kind: Service { id: 0d8c5151-3b8f-4b66-b2ec-69426d844f51 }, name: Name("external-dns-0d8c5151-3b8f-4b66-b2ec-69426d844f51"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 254])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f9638164-af79-4853-9a7e-a593f5bab52c, kind: Service { id: d120a386-2706-4e01-bb8c-b5d65c0f637f }, name: Name("external-dns-d120a386-2706-4e01-bb8c-b5d65c0f637f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 182])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0bc77817-8af4-413e-be1d-d96e849efaa0, kind: Service { id: d512bf60-e3a1-492a-aa6e-6bf28969f94c }, name: Name("nexus-d512bf60-e3a1-492a-aa6e-6bf28969f94c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d2375ade-1e0b-4247-ba64-7f0a959e70ac, kind: Service { id: 8e5fb63d-011e-4f8e-b416-8713dac504ae }, name: Name("nexus-8e5fb63d-011e-4f8e-b416-8713dac504ae"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 224, 235])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e2f1b87a-6b81-4706-840f-a0af0417a6fb, kind: Service { id: 4c699543-0a26-420d-8b7a-4a0104f01339 }, name: Name("nexus-4c699543-0a26-420d-8b7a-4a0104f01339"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 240])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19922024-08-12T22:33:08.940ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19932024-08-12T22:33:08.940ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
19942024-08-12T22:33:08.940ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19952024-08-12T22:33:08.940ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19962024-08-12T22:33:08.940ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19972024-08-12T22:33:08.940ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19982024-08-12T22:33:08.940ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 840 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42781 req_id = 1d630035-d2ce-4732-97e6-9639017b18cf response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19992024-08-12T22:33:09.001ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65525
20002024-08-12T22:33:09.001ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
20012024-08-12T22:33:09.086ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 84809 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65525 req_id = fdb1c9a0-f7d5-4243-a800-e368e256cd55 response_code = 200 uri = /inventory
20022024-08-12T22:33:09.087ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
20032024-08-12T22:33:09.087ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/config/omicron-zones.json file = common/src/ledger.rs:177
20042024-08-12T22:33:09.087ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/config/omicron-zones.json file = common/src/ledger.rs:177
20052024-08-12T22:33:09.087ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 986 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65525 req_id = 0ba2dcd5-1661-4c0e-a7e8-781632ecf10c response_code = 200 uri = /omicron-zones
20062024-08-12T22:33:09.935ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:53671 remote_addr = [fd00:1122:3344:101::d]:50227 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
20072024-08-12T22:33:09.954ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 18688 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = a46126b5-fb50-40b7-a246-edd16bbf4774 response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
20082024-08-12T22:33:12.947ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40667
20092024-08-12T22:33:12.948ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 737 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40667 req_id = 025a45b6-9b01-47bd-bd25-f65f04a1f2a5 response_code = 200 uri = /vpc-routes
20102024-08-12T22:33:12.954ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 319 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40667 req_id = 6d38071b-61d6-45db-8845-2556eec1b8d2 response_code = 204 uri = /vpc-routes
20112024-08-12T22:33:18.693ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62980
20122024-08-12T22:33:18.693ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 40 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62980 req_id = c144a998-341c-440b-bc5d-9152181ffa09 response_code = 200 uri = /baseboard
20132024-08-12T22:33:37.164ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64884
20142024-08-12T22:33:37.164ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64884 req_id = 1096e6b0-0bc1-42bf-a36f-f50145a2793b response_code = 200 uri = /vpc-routes
20152024-08-12T22:33:37.171ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64884 req_id = abd24262-1308-448f-8f7e-27e4b8a1c94e response_code = 204 uri = /vpc-routes
20162024-08-12T22:33:37.230ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42846
20172024-08-12T22:33:37.230ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42846 req_id = 8419f974-9343-4bb2-9e49-623149ba546e response_code = 200 uri = /vpc-routes
20182024-08-12T22:33:37.236ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42846 req_id = 439084d7-126e-4786-a944-6e89291834bc response_code = 204 uri = /vpc-routes
20192024-08-12T22:33:37.302ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35195
20202024-08-12T22:33:37.302ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35195 req_id = 2ce7c8f7-d979-401b-b6e8-0a8b23a2e3d7 response_code = 200 uri = /v2p
20212024-08-12T22:33:37.313ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51405
20222024-08-12T22:33:37.313ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51405 req_id = 470fdf96-f1c6-4669-bbad-287077b20bee response_code = 200 uri = /v2p
20232024-08-12T22:33:37.493ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47023
20242024-08-12T22:33:37.493ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
20252024-08-12T22:33:37.544ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
20262024-08-12T22:33:37.573ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 79417 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47023 req_id = b411c5b2-2cbe-4c4a-806b-45de5f35d9f5 response_code = 204 uri = /switch-ports
20272024-08-12T22:33:37.574ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1077 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47023 req_id = ebc2f65e-c39e-4b35-9224-327bb4327cec response_code = 200 uri = /network-bootstore-config
20282024-08-12T22:33:37.595ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
20292024-08-12T22:33:37.595ZINFOSledAgent (bootstore): Writing ledger to /pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
20302024-08-12T22:33:37.596ZINFOSledAgent (bootstore): Writing ledger to /pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
20312024-08-12T22:33:37.597ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
20322024-08-12T22:33:37.597ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 2210 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47023 req_id = 38fa62c4-8c98-4cb7-a3d1-d335072a0d8d response_code = 204 uri = /network-bootstore-config
20332024-08-12T22:33:37.608ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59321
20342024-08-12T22:33:37.608ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
20352024-08-12T22:33:37.634ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
20362024-08-12T22:33:37.656ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 47973 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59321 req_id = c6c7e9ab-6299-4ecf-a5d1-47984b9afa01 response_code = 204 uri = /switch-ports
20372024-08-12T22:33:37.657ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 771 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59321 req_id = b895f0ff-c9e4-4ce4-b3e2-e03908adf4d0 response_code = 200 uri = /network-bootstore-config
20382024-08-12T22:33:37.950ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37043
20392024-08-12T22:33:37.950ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37043 req_id = 48ee1f4f-4e5a-4504-9608-4c8938bf0486 response_code = 200 uri = /v2p
20402024-08-12T22:33:38.141ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61437
20412024-08-12T22:33:38.141ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61437 req_id = 4c0acb0d-9f76-46a4-ac1d-9570000e3fb7 response_code = 200 uri = /vpc-routes
20422024-08-12T22:33:38.148ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61437 req_id = 2b7f40c3-6bed-4ab3-9f9f-a695ccefeeaf response_code = 204 uri = /vpc-routes
20432024-08-12T22:33:38.394ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45159
20442024-08-12T22:33:38.394ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
20452024-08-12T22:33:38.418ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
20462024-08-12T22:33:38.440ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 45444 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:45159 req_id = 73fda6f8-6794-45f9-82b6-0fd2747137ff response_code = 204 uri = /switch-ports
20472024-08-12T22:33:38.440ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45159 req_id = 6a89d2c1-dcf9-4467-aacd-4076f64157b4 response_code = 200 uri = /network-bootstore-config
20482024-08-12T22:33:39.939ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 5958 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = 5cb806ee-373d-4ed0-96e0-4326e4150e77 response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
20492024-08-12T22:33:48.733ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:45502
20502024-08-12T22:33:48.734ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 572 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:45502 req_id = 5e852004-56fc-49c4-b639-4e9f2d52da0a response_code = 200 uri = /baseboard
20512024-08-12T22:34:07.161ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49315
20522024-08-12T22:34:07.161ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49315 req_id = a5606159-859d-4664-b950-9e54bb60c257 response_code = 200 uri = /vpc-routes
20532024-08-12T22:34:07.190ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60877
20542024-08-12T22:34:07.190ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60877 req_id = cc33c323-40c3-4d73-a4a3-200b4b21b5a5 response_code = 200 uri = /vpc-routes
20552024-08-12T22:34:07.199ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60877 req_id = d8e138c1-2f07-44e3-96b5-d42c44de7bdb response_code = 204 uri = /vpc-routes
20562024-08-12T22:34:07.224ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49315 req_id = ff3a5ae5-1d81-4ecd-84c8-ae2981017710 response_code = 204 uri = /vpc-routes
20572024-08-12T22:34:07.228ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63186
20582024-08-12T22:34:07.228ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63186 req_id = 0d0d047d-0828-4df5-8924-c6e3f0ea815e response_code = 200 uri = /v2p
20592024-08-12T22:34:07.274ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59235
20602024-08-12T22:34:07.274ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59235 req_id = 8b387a11-4e5b-4803-97f8-bc0e7699c13e response_code = 200 uri = /v2p
20612024-08-12T22:34:07.310ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51483
20622024-08-12T22:34:07.311ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
20632024-08-12T22:34:07.337ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
20642024-08-12T22:34:07.359ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 48456 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:51483 req_id = dbe0a673-1918-41de-9d02-747420dc0cd9 response_code = 204 uri = /switch-ports
20652024-08-12T22:34:07.359ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60207
20662024-08-12T22:34:07.359ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51483 req_id = 89c47169-021c-429a-81f4-1ec0b5667f58 response_code = 200 uri = /network-bootstore-config
20672024-08-12T22:34:07.360ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
20682024-08-12T22:34:07.385ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
20692024-08-12T22:34:07.407ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 47158 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:60207 req_id = 267d7022-b560-461e-a8b3-1c28d4ad0c71 response_code = 204 uri = /switch-ports
20702024-08-12T22:34:07.407ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 165 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60207 req_id = 75c85ae4-5200-4785-89e6-ab16fbf8c4a8 response_code = 200 uri = /network-bootstore-config
20712024-08-12T22:34:07.999ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62220
20722024-08-12T22:34:07.999ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62220 req_id = 4d20a6cc-8de7-4020-8db8-0fc74c48f5e8 response_code = 200 uri = /vpc-routes
20732024-08-12T22:34:08.008ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62220 req_id = 03d02e92-a2a8-4fd5-9ff9-9abdf99bce61 response_code = 204 uri = /vpc-routes
20742024-08-12T22:34:08.016ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48700
20752024-08-12T22:34:08.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 695 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48700 req_id = 63588bfe-42a9-4062-8cde-4bdc35d2dc66 response_code = 200 uri = /v2p
20762024-08-12T22:34:08.111ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47428
20772024-08-12T22:34:08.111ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
20782024-08-12T22:34:08.135ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
20792024-08-12T22:34:08.157ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 46738 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:47428 req_id = df411a3a-7ec5-41b3-b7a0-3cbbf17a23e3 response_code = 204 uri = /switch-ports
20802024-08-12T22:34:08.158ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47428 req_id = fcf90777-d3b7-466c-95b2-6bfbea216fc5 response_code = 200 uri = /network-bootstore-config
20812024-08-12T22:34:09.940ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 6661 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = 080a62f4-9e44-4e0e-875c-ddcaa1c2ada0 response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
20822024-08-12T22:34:18.765ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:52654
20832024-08-12T22:34:18.765ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 63 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:52654 req_id = b141387f-dec6-41eb-aff7-8c249a4508d4 response_code = 200 uri = /baseboard
20842024-08-12T22:34:37.281ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43224
20852024-08-12T22:34:37.281ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 204 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43224 req_id = 84f47691-6690-4af3-af59-0d79372fb266 response_code = 200 uri = /v2p
20862024-08-12T22:34:37.321ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37403
20872024-08-12T22:34:37.322ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37403 req_id = 472e1c1e-e3ac-46d6-be71-591ff9da865a response_code = 200 uri = /vpc-routes
20882024-08-12T22:34:37.331ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:37403 req_id = bd1f595c-2d95-4072-b642-49c9f7c86524 response_code = 204 uri = /vpc-routes
20892024-08-12T22:34:37.357ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61510
20902024-08-12T22:34:37.358ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61510 req_id = 40e66f92-85b8-4512-b22c-51dd012d17c1 response_code = 200 uri = /v2p
20912024-08-12T22:34:37.410ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36245
20922024-08-12T22:34:37.411ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 960 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36245 req_id = af9f3503-bcd3-4e08-9c4b-1ad26d900e14 response_code = 200 uri = /vpc-routes
20932024-08-12T22:34:37.419ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36245 req_id = 4edaa624-d6c8-40ec-ad6c-15b7057bff6b response_code = 204 uri = /vpc-routes
20942024-08-12T22:34:37.537ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40296
20952024-08-12T22:34:37.537ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
20962024-08-12T22:34:37.559ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
20972024-08-12T22:34:37.579ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 42053 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:40296 req_id = 594750b2-ee9f-473d-8d00-591936eea9e7 response_code = 204 uri = /switch-ports
20982024-08-12T22:34:37.579ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43335
20992024-08-12T22:34:37.580ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
21002024-08-12T22:34:37.583ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 3750 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40296 req_id = c0bdf18f-df25-4ec7-89e3-bceb0ffa2bf9 response_code = 200 uri = /network-bootstore-config
21012024-08-12T22:34:37.603ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
21022024-08-12T22:34:37.624ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 45210 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:43335 req_id = b405765b-66ea-4aac-9fe2-705234eb4d6d response_code = 204 uri = /switch-ports
21032024-08-12T22:34:37.625ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 186 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43335 req_id = 40223f88-8b64-455c-b5ea-04c302acacb3 response_code = 200 uri = /network-bootstore-config
21042024-08-12T22:34:38.048ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40107
21052024-08-12T22:34:38.048ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40107 req_id = 99f95bd8-2390-4850-ae93-5547133ae2f7 response_code = 200 uri = /vpc-routes
21062024-08-12T22:34:38.066ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1079 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40107 req_id = 35a57349-a5e8-4ab9-9307-8bdd7acee185 response_code = 204 uri = /vpc-routes
21072024-08-12T22:34:38.084ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48994
21082024-08-12T22:34:38.084ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48994 req_id = 51ff0eb1-c076-4cfb-b914-d96a42dcef17 response_code = 200 uri = /v2p
21092024-08-12T22:34:38.377ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39337
21102024-08-12T22:34:38.377ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
21112024-08-12T22:34:38.402ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
21122024-08-12T22:34:38.424ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 47109 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:39337 req_id = 3bbd16a0-919f-4021-a8dc-566ef4a294ee response_code = 204 uri = /switch-ports
21132024-08-12T22:34:38.425ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39337 req_id = af67287f-4bf9-4b92-9a77-550923b9e5bf response_code = 200 uri = /network-bootstore-config
21142024-08-12T22:34:39.940ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 7310 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = 642890ca-9f16-4690-b3af-082c824485e0 response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
21152024-08-12T22:34:48.799ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62274
21162024-08-12T22:34:48.799ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 85 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62274 req_id = 72f52619-fec6-4951-a414-3c14a2c80f96 response_code = 200 uri = /baseboard
21172024-08-12T22:35:07.209ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54577
21182024-08-12T22:35:07.209ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54577 req_id = 81cbc6c4-1ebc-4f40-b1ba-feb76787fee5 response_code = 200 uri = /vpc-routes
21192024-08-12T22:35:07.218ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54577 req_id = d2b2c526-4237-4955-93be-f31924399496 response_code = 204 uri = /vpc-routes
21202024-08-12T22:35:07.218ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46649
21212024-08-12T22:35:07.218ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46649 req_id = 8fd3bd94-b614-49bc-afee-d0f3c4473169 response_code = 200 uri = /v2p
21222024-08-12T22:35:07.229ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50618
21232024-08-12T22:35:07.229ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50618 req_id = 59ce005d-91d4-4ce1-82c6-c27c090fff06 response_code = 200 uri = /v2p
21242024-08-12T22:35:07.254ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40091
21252024-08-12T22:35:07.254ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 34 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40091 req_id = ba834255-db04-4818-9692-6566012e249d response_code = 200 uri = /vpc-routes
21262024-08-12T22:35:07.264ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40091 req_id = f38b618c-a450-4865-a53d-fdc721ef66da response_code = 204 uri = /vpc-routes
21272024-08-12T22:35:07.392ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44151
21282024-08-12T22:35:07.392ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
21292024-08-12T22:35:07.417ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
21302024-08-12T22:35:07.439ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 47223 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:44151 req_id = 3aeecc73-9b6c-4196-9fb4-cac84f0a802d response_code = 204 uri = /switch-ports
21312024-08-12T22:35:07.439ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45826
21322024-08-12T22:35:07.439ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
21332024-08-12T22:35:07.443ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 3509 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44151 req_id = 7cb48229-7336-4768-a431-9d581c51692b response_code = 200 uri = /network-bootstore-config
21342024-08-12T22:35:07.463ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
21352024-08-12T22:35:07.485ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 45622 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:45826 req_id = e26e1ca2-487f-4ba5-ba54-b2976a87a9b2 response_code = 204 uri = /switch-ports
21362024-08-12T22:35:07.485ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45826 req_id = 3adaca99-f15e-4a4a-886c-060e6d88703d response_code = 200 uri = /network-bootstore-config
21372024-08-12T22:35:07.936ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52722
21382024-08-12T22:35:07.936ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52722 req_id = 4b33faf9-ccce-423e-b2ea-c305add6a158 response_code = 200 uri = /vpc-routes
21392024-08-12T22:35:07.945ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 784 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52722 req_id = 55347cf7-95c7-4deb-bc77-08bd3e2b0175 response_code = 204 uri = /vpc-routes
21402024-08-12T22:35:07.967ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58119
21412024-08-12T22:35:07.967ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58119 req_id = de6180dd-7f84-4d2b-aeb1-117e4a98e7b9 response_code = 200 uri = /v2p
21422024-08-12T22:35:08.038ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43125
21432024-08-12T22:35:08.038ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
21442024-08-12T22:35:08.062ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
21452024-08-12T22:35:08.085ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 46946 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:43125 req_id = 181f3c42-c726-4be4-9137-f5e260361711 response_code = 204 uri = /switch-ports
21462024-08-12T22:35:08.085ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43125 req_id = 12bd7014-953c-4bf1-a349-1d1d20f006fe response_code = 200 uri = /network-bootstore-config
21472024-08-12T22:35:09.945ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 5541 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = 85ceb0d4-a8da-48b6-b3af-1c9b6f2b80d3 response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
21482024-08-12T22:35:18.831ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:56271
21492024-08-12T22:35:18.831ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 39 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:56271 req_id = e8abf41a-b459-48bf-b363-d62f0fd5ee34 response_code = 200 uri = /baseboard
21502024-08-12T22:35:37.313ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62252
21512024-08-12T22:35:37.313ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62252 req_id = ef4ce2b7-8256-4519-a855-9e8cc6edb331 response_code = 200 uri = /v2p
21522024-08-12T22:35:37.324ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45870
21532024-08-12T22:35:37.325ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45870 req_id = 8ce443ed-91c0-4557-ad0c-b4d522376e17 response_code = 200 uri = /vpc-routes
21542024-08-12T22:35:37.334ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45870 req_id = 3e4015f0-8807-4f7c-a2ba-a4ce80f44fcb response_code = 204 uri = /vpc-routes
21552024-08-12T22:35:37.356ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60054
21562024-08-12T22:35:37.357ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60054 req_id = 9d1fbd61-fae4-4ed3-9aba-9ba805c9703b response_code = 200 uri = /v2p
21572024-08-12T22:35:37.421ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39538
21582024-08-12T22:35:37.422ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 993 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39538 req_id = 79d92cbd-0314-458c-845a-e9c5b42c7d3c response_code = 200 uri = /vpc-routes
21592024-08-12T22:35:37.430ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39538 req_id = c64c87d7-4875-4543-ade3-01ce519a43d4 response_code = 204 uri = /vpc-routes
21602024-08-12T22:35:37.709ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36385
21612024-08-12T22:35:37.709ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
21622024-08-12T22:35:37.736ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
21632024-08-12T22:35:37.762ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 52825 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:36385 req_id = 19789268-e8c1-4948-ae37-75b03253166a response_code = 204 uri = /switch-ports
21642024-08-12T22:35:37.762ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37909
21652024-08-12T22:35:37.762ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
21662024-08-12T22:35:37.762ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 244 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36385 req_id = a397b8dc-00b4-4f1e-bc5a-2bf2a32cfa3b response_code = 200 uri = /network-bootstore-config
21672024-08-12T22:35:37.786ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
21682024-08-12T22:35:37.796ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
21692024-08-12T22:35:37.808ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 45651 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37909 req_id = 55071bdf-3679-49e7-9b93-da16a3973731 response_code = 204 uri = /switch-ports
21702024-08-12T22:35:37.808ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37909 req_id = 16e1e750-c682-4ba7-8906-c4eaa50468c2 response_code = 200 uri = /network-bootstore-config
21712024-08-12T22:35:37.893ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55048
21722024-08-12T22:35:37.893ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55048 req_id = 8b97ceb8-d152-4f5b-858b-3a550b4129a0 response_code = 200 uri = /vpc-routes
21732024-08-12T22:35:37.903ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1117 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55048 req_id = 4cde5715-5e47-4306-8d5f-47ed2f14f426 response_code = 204 uri = /vpc-routes
21742024-08-12T22:35:38.074ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50843
21752024-08-12T22:35:38.074ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50843 req_id = cb64022f-c33c-4a67-9eb2-3eebb1d0521e response_code = 200 uri = /v2p
21762024-08-12T22:35:38.291ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52305
21772024-08-12T22:35:38.291ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
21782024-08-12T22:35:38.314ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
21792024-08-12T22:35:38.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 43560 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52305 req_id = e711de51-4354-43b9-9fb8-04e73d783489 response_code = 204 uri = /switch-ports
21802024-08-12T22:35:38.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52305 req_id = 86dbfdd2-71c0-416b-b9f3-bf73526c5d42 response_code = 200 uri = /network-bootstore-config
21812024-08-12T22:35:39.982ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 6626 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = 6ed1a100-306a-46f4-aea0-5f305b5461e9 response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
21822024-08-12T22:35:48.862ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:55336
21832024-08-12T22:35:48.862ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 51 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:55336 req_id = 812af907-ba1e-4c6d-a82b-ec778d939a56 response_code = 200 uri = /baseboard
21842024-08-12T22:36:07.184ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39472
21852024-08-12T22:36:07.184ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39472 req_id = d83d0f96-c779-4aab-816a-b555cb1e7aa3 response_code = 200 uri = /vpc-routes
21862024-08-12T22:36:07.194ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39472 req_id = fdffd2c5-3aea-4567-956c-ad3238180ee0 response_code = 204 uri = /vpc-routes
21872024-08-12T22:36:07.235ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45735
21882024-08-12T22:36:07.235ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45735 req_id = 5267b030-44d1-4025-ac09-911adb9913fc response_code = 200 uri = /vpc-routes
21892024-08-12T22:36:07.244ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 909 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45735 req_id = 7c6baebf-3a71-4516-a0f5-4780d455284d response_code = 204 uri = /vpc-routes
21902024-08-12T22:36:07.291ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60308
21912024-08-12T22:36:07.292ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60308 req_id = a542c3ae-0acd-4748-83b0-27d5d1c41139 response_code = 200 uri = /v2p
21922024-08-12T22:36:07.340ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57339
21932024-08-12T22:36:07.340ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57339 req_id = a9870bfa-da21-44a6-8dad-7488039ab24d response_code = 200 uri = /v2p
21942024-08-12T22:36:07.417ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59067
21952024-08-12T22:36:07.417ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
21962024-08-12T22:36:07.441ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
21972024-08-12T22:36:07.463ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 45758 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:59067 req_id = 6da6a281-aad8-4221-8a3a-02e2e33020d1 response_code = 204 uri = /switch-ports
21982024-08-12T22:36:07.463ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51184
21992024-08-12T22:36:07.463ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
22002024-08-12T22:36:07.467ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 3592 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59067 req_id = 2ad2e9e2-2163-4942-a2f7-59c9699994df response_code = 200 uri = /network-bootstore-config
22012024-08-12T22:36:07.487ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
22022024-08-12T22:36:07.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 45626 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:51184 req_id = 1c444119-4e9c-4ba6-b658-6f98690009b1 response_code = 204 uri = /switch-ports
22032024-08-12T22:36:07.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51184 req_id = a4d9e1ba-12d0-46b7-9467-5493c454f5a5 response_code = 200 uri = /network-bootstore-config
22042024-08-12T22:36:07.888ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46811
22052024-08-12T22:36:07.888ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46811 req_id = e605e171-5397-499c-a46c-5f3a19305393 response_code = 200 uri = /vpc-routes
22062024-08-12T22:36:07.897ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46811 req_id = 87394f0a-87a6-4057-b5ba-cc64eba467b4 response_code = 204 uri = /vpc-routes
22072024-08-12T22:36:07.942ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38604
22082024-08-12T22:36:07.942ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 151 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38604 req_id = c703e428-184b-417d-92dc-72c062a2e14e response_code = 200 uri = /v2p
22092024-08-12T22:36:08.107ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50446
22102024-08-12T22:36:08.107ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
22112024-08-12T22:36:08.130ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
22122024-08-12T22:36:08.151ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 44165 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:50446 req_id = 414a7cdf-89bb-446a-8368-4c6cae098d58 response_code = 204 uri = /switch-ports
22132024-08-12T22:36:08.152ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50446 req_id = b42e843b-6b7e-4e7d-9395-01c04e8b5f78 response_code = 200 uri = /network-bootstore-config
22142024-08-12T22:36:09.983ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 7234 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = 1b4befa0-5a9a-4214-b6b4-abfc13476ea1 response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
22152024-08-12T22:36:18.896ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:45463
22162024-08-12T22:36:18.896ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 40 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:45463 req_id = 524a9a21-36b4-4b7d-bef0-7b0028056f5c response_code = 200 uri = /baseboard
22172024-08-12T22:36:37.260ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50996
22182024-08-12T22:36:37.260ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 339 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50996 req_id = 43fa522b-037d-4928-8fa4-b17a04d5bac7 response_code = 200 uri = /vpc-routes
22192024-08-12T22:36:37.270ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50996 req_id = 2c6b6d32-6275-477e-b9f2-353e84635d36 response_code = 204 uri = /vpc-routes
22202024-08-12T22:36:37.333ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41771
22212024-08-12T22:36:37.333ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41771 req_id = b1d0189b-bd00-4e57-a88f-7f88c0f5a402 response_code = 200 uri = /v2p
22222024-08-12T22:36:37.442ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55934
22232024-08-12T22:36:37.443ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 972 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55934 req_id = eec7528a-8534-401e-b710-0abfe3bcbdee response_code = 200 uri = /vpc-routes
22242024-08-12T22:36:37.449ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59879
22252024-08-12T22:36:37.449ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59879 req_id = 24f2499b-2567-4bcd-8dc2-ec9a58dbb34e response_code = 200 uri = /v2p
22262024-08-12T22:36:37.452ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55934 req_id = bfe723ba-c0a7-4c8f-a1f4-70606df40088 response_code = 204 uri = /vpc-routes
22272024-08-12T22:36:37.643ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47924
22282024-08-12T22:36:37.643ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
22292024-08-12T22:36:37.666ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
22302024-08-12T22:36:37.691ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 47804 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47924 req_id = 2b3a8f0e-240d-4ba7-b03e-b39d0e7b0456 response_code = 204 uri = /switch-ports
22312024-08-12T22:36:37.692ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47924 req_id = e921142a-0260-42ca-8b7d-ec8f3efa94df response_code = 200 uri = /network-bootstore-config
22322024-08-12T22:36:37.714ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37368
22332024-08-12T22:36:37.714ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
22342024-08-12T22:36:37.739ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
22352024-08-12T22:36:37.763ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 48618 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37368 req_id = 886331f5-fa8b-4769-a349-7088ef3b02a2 response_code = 204 uri = /switch-ports
22362024-08-12T22:36:37.763ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37368 req_id = 560f195c-1143-4088-af81-6e68e96363c6 response_code = 200 uri = /network-bootstore-config
22372024-08-12T22:36:37.930ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34356
22382024-08-12T22:36:37.930ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 202 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34356 req_id = 8e12af50-79b3-4b35-9f04-44aedff70529 response_code = 200 uri = /v2p
22392024-08-12T22:36:38.046ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34698
22402024-08-12T22:36:38.046ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34698 req_id = 23dc2ae8-b3b3-44c0-9bf2-1e8701535983 response_code = 200 uri = /vpc-routes
22412024-08-12T22:36:38.055ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34698 req_id = c7d7e444-5f10-4360-87a7-5b76cd982333 response_code = 204 uri = /vpc-routes
22422024-08-12T22:36:38.256ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54500
22432024-08-12T22:36:38.256ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
22442024-08-12T22:36:38.281ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
22452024-08-12T22:36:38.304ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 48200 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:54500 req_id = 1172d54f-acc4-49e5-afed-8779781af45b response_code = 204 uri = /switch-ports
22462024-08-12T22:36:38.304ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54500 req_id = 2e3d59b0-8cdc-4636-94e5-f1a0fcebc9b2 response_code = 200 uri = /network-bootstore-config
22472024-08-12T22:36:39.981ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 6635 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = 5e981706-56f4-4857-a376-4d120203b4fe response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
22482024-08-12T22:36:48.927ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:45032
22492024-08-12T22:36:48.927ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 47 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:45032 req_id = 03bed53a-1e5c-4307-a159-29c19377a0da response_code = 200 uri = /baseboard
22502024-08-12T22:37:07.158ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49973
22512024-08-12T22:37:07.158ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49973 req_id = 529301f9-4238-46e4-a671-003ad680d756 response_code = 200 uri = /vpc-routes
22522024-08-12T22:37:07.167ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49973 req_id = a7992d9f-89f2-4c72-b72a-5dd9e811b0b5 response_code = 204 uri = /vpc-routes
22532024-08-12T22:37:07.191ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52488
22542024-08-12T22:37:07.192ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52488 req_id = 528e9e6c-66c9-421e-853d-6853d0a32717 response_code = 200 uri = /v2p
22552024-08-12T22:37:07.250ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53205
22562024-08-12T22:37:07.251ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 728 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53205 req_id = 0106caf0-ade4-452c-b3bf-34db564e67b7 response_code = 200 uri = /v2p
22572024-08-12T22:37:07.298ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49405
22582024-08-12T22:37:07.299ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 41 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49405 req_id = faaf75e7-42ec-4c38-badb-fa83d92b167b response_code = 200 uri = /vpc-routes
22592024-08-12T22:37:07.301ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43165
22602024-08-12T22:37:07.301ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
22612024-08-12T22:37:07.327ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
22622024-08-12T22:37:07.350ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 49175 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:43165 req_id = 2733e584-562e-4e1b-aa67-c1c787430811 response_code = 204 uri = /switch-ports
22632024-08-12T22:37:07.351ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 613 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49405 req_id = ffbae701-e0d5-4bbb-8683-b344206607a4 response_code = 204 uri = /vpc-routes
22642024-08-12T22:37:07.351ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43165 req_id = f689b1aa-99c3-4dc6-b3aa-a9f81f93bb47 response_code = 200 uri = /network-bootstore-config
22652024-08-12T22:37:07.405ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61275
22662024-08-12T22:37:07.405ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
22672024-08-12T22:37:07.428ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
22682024-08-12T22:37:07.449ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 43933 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:61275 req_id = 4bcbef72-353b-4b8e-9a48-acca94d8bea7 response_code = 204 uri = /switch-ports
22692024-08-12T22:37:07.449ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61275 req_id = 36831924-556f-41a9-b0b9-c27764e43832 response_code = 200 uri = /network-bootstore-config
22702024-08-12T22:37:07.817ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36876
22712024-08-12T22:37:07.817ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36876 req_id = f00745e1-2098-4fcf-bc58-1c82d241b662 response_code = 200 uri = /vpc-routes
22722024-08-12T22:37:07.825ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36876 req_id = c4d2b85c-a664-4d57-a22d-dcc0b578c24e response_code = 204 uri = /vpc-routes
22732024-08-12T22:37:07.867ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64272
22742024-08-12T22:37:07.868ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64272 req_id = 30744efe-94fb-4013-8807-057182dce9ab response_code = 200 uri = /v2p
22752024-08-12T22:37:08.052ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42199
22762024-08-12T22:37:08.052ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
22772024-08-12T22:37:08.077ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
22782024-08-12T22:37:08.098ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 45698 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:42199 req_id = dca9ad7e-6b09-4ffe-b11b-00cf4be5fde4 response_code = 204 uri = /switch-ports
22792024-08-12T22:37:08.098ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42199 req_id = 6b0543d6-50f6-4a57-8222-5e5cee061a1e response_code = 200 uri = /network-bootstore-config
22802024-08-12T22:37:09.981ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 6234 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = 1d8ec724-8b53-4951-8b93-10cbb811db66 response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
22812024-08-12T22:37:18.962ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39706
22822024-08-12T22:37:18.962ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 74 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39706 req_id = b614ed53-da76-477a-9d4c-ee166f775320 response_code = 200 uri = /baseboard
22832024-08-12T22:37:37.303ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51953
22842024-08-12T22:37:37.305ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1212 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51953 req_id = 10edbc6b-c64e-4100-ad18-eb20bbb43652 response_code = 200 uri = /vpc-routes
22852024-08-12T22:37:37.314ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51953 req_id = 966a47ee-321b-413e-9cff-84bbaab7c9b6 response_code = 204 uri = /vpc-routes
22862024-08-12T22:37:37.381ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59445
22872024-08-12T22:37:37.383ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1457 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59445 req_id = ca7ab877-93ad-4506-814b-52e4865e587c response_code = 200 uri = /v2p
22882024-08-12T22:37:37.481ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51424
22892024-08-12T22:37:37.482ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c6463b16-74e6-4dec-9e5a-c72d26b9fe84, kind: Service { id: 0d8c5151-3b8f-4b66-b2ec-69426d844f51 }, name: Name("external-dns-0d8c5151-3b8f-4b66-b2ec-69426d844f51"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 254])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f9638164-af79-4853-9a7e-a593f5bab52c, kind: Service { id: d120a386-2706-4e01-bb8c-b5d65c0f637f }, name: Name("external-dns-d120a386-2706-4e01-bb8c-b5d65c0f637f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 182])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0bc77817-8af4-413e-be1d-d96e849efaa0, kind: Service { id: d512bf60-e3a1-492a-aa6e-6bf28969f94c }, name: Name("nexus-d512bf60-e3a1-492a-aa6e-6bf28969f94c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d2375ade-1e0b-4247-ba64-7f0a959e70ac, kind: Service { id: 8e5fb63d-011e-4f8e-b416-8713dac504ae }, name: Name("nexus-8e5fb63d-011e-4f8e-b416-8713dac504ae"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 224, 235])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e2f1b87a-6b81-4706-840f-a0af0417a6fb, kind: Service { id: 4c699543-0a26-420d-8b7a-4a0104f01339 }, name: Name("nexus-4c699543-0a26-420d-8b7a-4a0104f01339"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 240])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
22902024-08-12T22:37:37.482ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22912024-08-12T22:37:37.482ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
22922024-08-12T22:37:37.482ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22932024-08-12T22:37:37.482ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22942024-08-12T22:37:37.482ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22952024-08-12T22:37:37.482ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22962024-08-12T22:37:37.482ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1220 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51424 req_id = 3975f6ab-3373-4728-b068-d88c4b6ea8d1 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
22972024-08-12T22:37:37.504ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44441
22982024-08-12T22:37:37.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 24 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44441 req_id = 2d76621e-72b4-40fd-9b56-54a0e1607719 response_code = 200 uri = /vpc-routes
22992024-08-12T22:37:37.512ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44441 req_id = 6bcfdb0a-0f4c-4dea-8ba5-a0ee8e8c5f1d response_code = 204 uri = /vpc-routes
23002024-08-12T22:37:37.557ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49471
23012024-08-12T22:37:37.557ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49471 req_id = 2bc950e6-02c5-48a5-9dfb-2ad7eda78762 response_code = 200 uri = /v2p
23022024-08-12T22:37:37.579ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61161
23032024-08-12T22:37:37.579ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c6463b16-74e6-4dec-9e5a-c72d26b9fe84, kind: Service { id: 0d8c5151-3b8f-4b66-b2ec-69426d844f51 }, name: Name("external-dns-0d8c5151-3b8f-4b66-b2ec-69426d844f51"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 254])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f9638164-af79-4853-9a7e-a593f5bab52c, kind: Service { id: d120a386-2706-4e01-bb8c-b5d65c0f637f }, name: Name("external-dns-d120a386-2706-4e01-bb8c-b5d65c0f637f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 182])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0bc77817-8af4-413e-be1d-d96e849efaa0, kind: Service { id: d512bf60-e3a1-492a-aa6e-6bf28969f94c }, name: Name("nexus-d512bf60-e3a1-492a-aa6e-6bf28969f94c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d2375ade-1e0b-4247-ba64-7f0a959e70ac, kind: Service { id: 8e5fb63d-011e-4f8e-b416-8713dac504ae }, name: Name("nexus-8e5fb63d-011e-4f8e-b416-8713dac504ae"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 224, 235])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e2f1b87a-6b81-4706-840f-a0af0417a6fb, kind: Service { id: 4c699543-0a26-420d-8b7a-4a0104f01339 }, name: Name("nexus-4c699543-0a26-420d-8b7a-4a0104f01339"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 240])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
23042024-08-12T22:37:37.579ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23052024-08-12T22:37:37.579ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
23062024-08-12T22:37:37.579ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23072024-08-12T22:37:37.579ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23082024-08-12T22:37:37.580ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23092024-08-12T22:37:37.580ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23102024-08-12T22:37:37.580ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 938 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61161 req_id = 83cb90cc-596b-41f8-995b-0ee07c9080ff response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23112024-08-12T22:37:37.681ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35551
23122024-08-12T22:37:37.681ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
23132024-08-12T22:37:37.711ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
23142024-08-12T22:37:37.739ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 58107 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:35551 req_id = d986d4b7-defa-4115-8931-976973dfa74f response_code = 204 uri = /switch-ports
23152024-08-12T22:37:37.739ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43256
23162024-08-12T22:37:37.740ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
23172024-08-12T22:37:37.740ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35551 req_id = 0c2d4786-f1bc-4cec-ae14-0b2e68a53784 response_code = 200 uri = /network-bootstore-config
23182024-08-12T22:37:37.765ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
23192024-08-12T22:37:37.789ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 50082 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:43256 req_id = d925aa42-f39c-42b9-aa6c-4ceda19a46ee response_code = 204 uri = /switch-ports
23202024-08-12T22:37:37.790ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43256 req_id = 65cf616d-681f-411a-acd1-52301ccbfdef response_code = 200 uri = /network-bootstore-config
23212024-08-12T22:37:38.008ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54155
23222024-08-12T22:37:38.008ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54155 req_id = 5ab1ec23-7f86-4313-8986-04bcf5c9566b response_code = 200 uri = /v2p
23232024-08-12T22:37:38.060ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48859
23242024-08-12T22:37:38.061ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c6463b16-74e6-4dec-9e5a-c72d26b9fe84, kind: Service { id: 0d8c5151-3b8f-4b66-b2ec-69426d844f51 }, name: Name("external-dns-0d8c5151-3b8f-4b66-b2ec-69426d844f51"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 254])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f9638164-af79-4853-9a7e-a593f5bab52c, kind: Service { id: d120a386-2706-4e01-bb8c-b5d65c0f637f }, name: Name("external-dns-d120a386-2706-4e01-bb8c-b5d65c0f637f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 182])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0bc77817-8af4-413e-be1d-d96e849efaa0, kind: Service { id: d512bf60-e3a1-492a-aa6e-6bf28969f94c }, name: Name("nexus-d512bf60-e3a1-492a-aa6e-6bf28969f94c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 213])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d2375ade-1e0b-4247-ba64-7f0a959e70ac, kind: Service { id: 8e5fb63d-011e-4f8e-b416-8713dac504ae }, name: Name("nexus-8e5fb63d-011e-4f8e-b416-8713dac504ae"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 224, 235])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e2f1b87a-6b81-4706-840f-a0af0417a6fb, kind: Service { id: 4c699543-0a26-420d-8b7a-4a0104f01339 }, name: Name("nexus-4c699543-0a26-420d-8b7a-4a0104f01339"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 240])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
23252024-08-12T22:37:38.061ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23262024-08-12T22:37:38.061ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
23272024-08-12T22:37:38.061ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23282024-08-12T22:37:38.061ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23292024-08-12T22:37:38.061ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23302024-08-12T22:37:38.061ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23312024-08-12T22:37:38.061ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1116 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48859 req_id = 4e517d7c-604c-4516-a77a-4187a9949e00 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23322024-08-12T22:37:38.115ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58886
23332024-08-12T22:37:38.115ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58886 req_id = 8e7142b1-1cd5-4de0-8bc2-b9f0dc8838e3 response_code = 200 uri = /vpc-routes
23342024-08-12T22:37:38.125ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 776 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58886 req_id = 84398d03-f21f-4522-ae0f-e10dd8ad9365 response_code = 204 uri = /vpc-routes
23352024-08-12T22:37:38.398ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38974
23362024-08-12T22:37:38.399ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
23372024-08-12T22:37:38.422ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
23382024-08-12T22:37:38.445ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 46346 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:38974 req_id = c8921c59-6f9d-42d2-949a-42ad44abdf0f response_code = 204 uri = /switch-ports
23392024-08-12T22:37:38.445ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38974 req_id = d09ad8a8-285a-44c7-b371-3073cf7ca868 response_code = 200 uri = /network-bootstore-config
23402024-08-12T22:37:39.981ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 6999 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = 8c8451ae-5d1c-41b4-83bf-83b161858458 response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
23412024-08-12T22:37:48.993ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39408
23422024-08-12T22:37:48.993ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 80 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39408 req_id = cd848c0e-bcd4-4ee1-95da-c4a2710224d6 response_code = 200 uri = /baseboard
23432024-08-12T22:38:07.175ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45567
23442024-08-12T22:38:07.176ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 870 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45567 req_id = 799b9b2b-36fc-4cab-b196-934cc4dbd99e response_code = 200 uri = /vpc-routes
23452024-08-12T22:38:07.178ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47029
23462024-08-12T22:38:07.178ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 27 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47029 req_id = 857f5d19-3045-4183-82e2-ff6752e078ed response_code = 200 uri = /vpc-routes
23472024-08-12T22:38:07.185ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45567 req_id = 07501ca0-9d62-461e-9232-8a831575ca0d response_code = 204 uri = /vpc-routes
23482024-08-12T22:38:07.185ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47029 req_id = 61bb38f1-76b6-4e32-a036-d693efdd97f1 response_code = 204 uri = /vpc-routes
23492024-08-12T22:38:07.187ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59569
23502024-08-12T22:38:07.187ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59569 req_id = 7b64f475-13c2-4b96-9489-cf50bfda2c45 response_code = 200 uri = /v2p
23512024-08-12T22:38:07.325ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35770
23522024-08-12T22:38:07.325ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35770 req_id = d0161176-7855-4c87-a31b-fd36f1f76a72 response_code = 200 uri = /v2p
23532024-08-12T22:38:07.384ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48834
23542024-08-12T22:38:07.384ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
23552024-08-12T22:38:07.408ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
23562024-08-12T22:38:07.429ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 45187 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:48834 req_id = aa3e312e-a6e3-4b05-92bc-77b074f2c3a2 response_code = 204 uri = /switch-ports
23572024-08-12T22:38:07.429ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55064
23582024-08-12T22:38:07.430ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
23592024-08-12T22:38:07.452ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
23602024-08-12T22:38:07.472ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 42949 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:55064 req_id = b2c347ba-44ae-4b14-b357-6d8880a22087 response_code = 204 uri = /switch-ports
23612024-08-12T22:38:07.472ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48834 req_id = 10edf73e-eb32-49a9-ad8a-bb04e52d6512 response_code = 200 uri = /network-bootstore-config
23622024-08-12T22:38:07.472ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55064 req_id = 31bd8d62-546c-45db-8d4f-170dc136cb6d response_code = 200 uri = /network-bootstore-config
23632024-08-12T22:38:07.832ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56414
23642024-08-12T22:38:07.833ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 724 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56414 req_id = 18d6ce16-5fdf-46fc-b68c-d7643b67008b response_code = 200 uri = /vpc-routes
23652024-08-12T22:38:07.841ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56414 req_id = 85cab39f-693b-440c-a0c7-db69bfc71cec response_code = 204 uri = /vpc-routes
23662024-08-12T22:38:07.932ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45014
23672024-08-12T22:38:07.932ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45014 req_id = 3a23c654-187f-47d1-9887-d28ce2965a58 response_code = 200 uri = /v2p
23682024-08-12T22:38:08.029ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61666
23692024-08-12T22:38:08.029ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
23702024-08-12T22:38:08.053ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
23712024-08-12T22:38:08.074ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 44593 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:61666 req_id = 087f33af-4c0a-4c2e-8e4e-8c6ab34c3115 response_code = 204 uri = /switch-ports
23722024-08-12T22:38:08.074ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61666 req_id = cf960bcf-9eb4-4357-aad3-5e14007c82d8 response_code = 200 uri = /network-bootstore-config
23732024-08-12T22:38:10.100ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 6487 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = 475c89b0-ffa8-4f6a-aaab-22dc84c70061 response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
23742024-08-12T22:38:19.027ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:58825
23752024-08-12T22:38:19.027ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 33 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:58825 req_id = 8f78f92d-07c3-43c7-b6a1-06f520a403e7 response_code = 200 uri = /baseboard
23762024-08-12T22:38:37.241ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61050
23772024-08-12T22:38:37.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61050 req_id = b999638c-fe3f-4e29-ab38-0bdabc31cc99 response_code = 200 uri = /vpc-routes
23782024-08-12T22:38:37.246ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52000
23792024-08-12T22:38:37.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52000 req_id = a9dadb0e-089f-4f01-afcc-1b7815251495 response_code = 200 uri = /v2p
23802024-08-12T22:38:37.254ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1199 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61050 req_id = 7855dfd3-1ed9-4ff7-a96f-1022c75dd01e response_code = 204 uri = /vpc-routes
23812024-08-12T22:38:37.369ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65411
23822024-08-12T22:38:37.369ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65411 req_id = 37a4d973-9bac-44f8-919f-46f3b6c993a5 response_code = 200 uri = /vpc-routes
23832024-08-12T22:38:37.379ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1414 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65411 req_id = c2729203-a6e0-4c3c-ac8c-561002f2151d response_code = 204 uri = /vpc-routes
23842024-08-12T22:38:37.445ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46119
23852024-08-12T22:38:37.445ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 193 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46119 req_id = f7670e63-2aa1-413e-9449-55584d383c54 response_code = 200 uri = /v2p
23862024-08-12T22:38:37.534ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43464
23872024-08-12T22:38:37.534ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
23882024-08-12T22:38:37.557ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
23892024-08-12T22:38:37.580ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 46037 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:43464 req_id = d4285bf5-3536-4f6c-92da-b62fcd0100c6 response_code = 204 uri = /switch-ports
23902024-08-12T22:38:37.580ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43464 req_id = 65bce1cc-63bc-4040-8a2e-556a9f3f3b6f response_code = 200 uri = /network-bootstore-config
23912024-08-12T22:38:37.668ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54424
23922024-08-12T22:38:37.668ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
23932024-08-12T22:38:37.693ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
23942024-08-12T22:38:37.721ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 52144 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54424 req_id = 25002683-eb17-4c1b-8dd1-7c43d911875b response_code = 204 uri = /switch-ports
23952024-08-12T22:38:37.721ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54424 req_id = 62725218-a925-4cf9-98b7-cf9ee9e6a00d response_code = 200 uri = /network-bootstore-config
23962024-08-12T22:38:38.075ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38640
23972024-08-12T22:38:38.076ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 196 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38640 req_id = 7c0d81b9-f386-4d71-a8bf-f4cecb8b1cef response_code = 200 uri = /v2p
23982024-08-12T22:38:38.139ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40432
23992024-08-12T22:38:38.139ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40432 req_id = 51ec5db2-4faa-4dcd-9454-2043dbc8124f response_code = 200 uri = /vpc-routes
24002024-08-12T22:38:38.148ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 745 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40432 req_id = 4051b9c4-c43c-499e-af76-44db1a5dea09 response_code = 204 uri = /vpc-routes
24012024-08-12T22:38:38.586ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49957
24022024-08-12T22:38:38.587ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
24032024-08-12T22:38:38.611ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
24042024-08-12T22:38:38.631ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 44547 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:49957 req_id = 4bf64a13-334e-4497-8faa-a841041d976a response_code = 204 uri = /switch-ports
24052024-08-12T22:38:38.631ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49957 req_id = 0ab69d7f-ed2c-47fb-8e0a-f899e521358f response_code = 200 uri = /network-bootstore-config
24062024-08-12T22:38:40.101ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 7225 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = baf87359-636e-45f0-8929-2aacbef615ac response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
24072024-08-12T22:38:49.060ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62111
24082024-08-12T22:38:49.061ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 694 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62111 req_id = 0dbc2f27-3cc4-45a1-9422-58d6cb3e5680 response_code = 200 uri = /baseboard
24092024-08-12T22:39:07.162ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58657
24102024-08-12T22:39:07.162ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58657 req_id = 9f35bebb-ee10-4dd7-85d9-2e3e4c325b9c response_code = 200 uri = /vpc-routes
24112024-08-12T22:39:07.170ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58657 req_id = 8f72d918-7467-472b-a3da-c73eb18e2cfe response_code = 204 uri = /vpc-routes
24122024-08-12T22:39:07.185ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54222
24132024-08-12T22:39:07.186ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 995 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54222 req_id = 64254f34-f9cd-4728-9042-9d33234709a4 response_code = 200 uri = /v2p
24142024-08-12T22:39:07.262ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36495
24152024-08-12T22:39:07.262ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36495 req_id = bc437c45-e7cf-4e86-a93b-a1a1f7ba6dda response_code = 200 uri = /vpc-routes
24162024-08-12T22:39:07.270ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36495 req_id = 82b5a930-8eda-4746-8c5f-e84d2b3af07a response_code = 204 uri = /vpc-routes
24172024-08-12T22:39:07.283ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61549
24182024-08-12T22:39:07.284ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 700 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61549 req_id = a5500a7e-2641-48bf-a569-807bb90dc1de response_code = 200 uri = /v2p
24192024-08-12T22:39:07.307ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49125
24202024-08-12T22:39:07.307ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
24212024-08-12T22:39:07.332ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
24222024-08-12T22:39:07.354ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 46902 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:49125 req_id = 51a8eb9e-18eb-4f74-a2a0-55f3ffe8309c response_code = 204 uri = /switch-ports
24232024-08-12T22:39:07.354ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49125 req_id = 8547aef5-c72b-41a2-8f39-0d820cf2df98 response_code = 200 uri = /network-bootstore-config
24242024-08-12T22:39:07.419ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63116
24252024-08-12T22:39:07.420ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
24262024-08-12T22:39:07.443ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
24272024-08-12T22:39:07.462ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 42665 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63116 req_id = f6af91ed-383e-4129-897b-4c1581ee80b4 response_code = 204 uri = /switch-ports
24282024-08-12T22:39:07.462ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63116 req_id = 1ad6ef36-c2b8-485f-8e00-cc6aa1a3267f response_code = 200 uri = /network-bootstore-config
24292024-08-12T22:39:07.827ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35339
24302024-08-12T22:39:07.827ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35339 req_id = 729294ef-9ba2-44d0-857a-8fb1a34252e9 response_code = 200 uri = /vpc-routes
24312024-08-12T22:39:07.835ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35339 req_id = f783b046-2afd-4014-811c-33d772bc925b response_code = 204 uri = /vpc-routes
24322024-08-12T22:39:07.865ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:32967
24332024-08-12T22:39:07.865ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:32967 req_id = 3813ba1e-7168-44d1-90df-182a90b0323f response_code = 200 uri = /v2p
24342024-08-12T22:39:07.978ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40174
24352024-08-12T22:39:07.978ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
24362024-08-12T22:39:08.004ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
24372024-08-12T22:39:08.027ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 48890 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40174 req_id = 411b6051-33bd-45ab-ac37-cd58f2bcefdd response_code = 204 uri = /switch-ports
24382024-08-12T22:39:08.028ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40174 req_id = 5ad1413e-4331-437b-acfb-0da28f309a28 response_code = 200 uri = /network-bootstore-config
24392024-08-12T22:39:10.100ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 7112 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = 37b11df3-0c2c-40f3-b7e1-1d6dce53ef30 response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
24402024-08-12T22:39:19.093ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:60788
24412024-08-12T22:39:19.093ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 51 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:60788 req_id = 76923813-8f78-4b91-b380-ecb064874535 response_code = 200 uri = /baseboard
24422024-08-12T22:39:22.985ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51057
24432024-08-12T22:39:22.986ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51057 req_id = d9c46ef6-3e25-4ceb-9d6e-9aa3e43c4c1f response_code = 200 uri = /v2p
24442024-08-12T22:39:22.986ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:778 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 247, 125, 106])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(1237126) }
24452024-08-12T22:39:22.986ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51057 req_id = 4f2e963b-3847-47a0-bcd4-25308a22da3c response_code = 204 uri = /v2p
24462024-08-12T22:39:23.128ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34083
24472024-08-12T22:39:23.134ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:574 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: fbe24bec-ccf8-4e6f-a407-f4c01f58602b, kind: Instance { id: 6751d36e-e2f3-4c4e-9403-88826a35df91 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 125, 106])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(1237126), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: fbe24bec-ccf8-4e6f-a407-f4c01f58602b, kind: Instance { id: 6751d36e-e2f3-4c4e-9403-88826a35df91 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 125, 106])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(1237126), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: fbe24bec-ccf8-4e6f-a407-f4c01f58602b, kind: Instance { id: 6751d36e-e2f3-4c4e-9403-88826a35df91 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 125, 106])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(1237126), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(1237126))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: fbe24bec-ccf8-4e6f-a407-f4c01f58602b, kind: Instance { id: 6751d36e-e2f3-4c4e-9403-88826a35df91 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 125, 106])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(1237126), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-30e3dd3f278", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 6364a4bb-4411-48dc-b68e-c9426e8b27f5, read_only_parent: Some(Volume { block_size: 512, id: 34c7e218-1a16-4d12-af8a-65e92675126f, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 0e4ac7e1-9472-4f66-954e-6ec94ea1b92a, key: Some("syMSLYwARdVwIcfCgl7f/Gz+sViPuhy4XcurX9XWsZs="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19001", "[fd00:1122:3344:101::12]:19001", "[fd00:1122:3344:101::13]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 6364a4bb-4411-48dc-b68e-c9426e8b27f5, key: Some("WkrMz60P67sO+4kKRVzLC22lhF9EFIdkKIrYhLUKN/8="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19002", "[fd00:1122:3344:101::14]:19000", "[fd00:1122:3344:101::16]:19000"] } }] } }], cloud_init_bytes: Some("6zyQTVNXSU40LjEAAgEBAAIAAioA+AEAKgBAAAAAAAAAAAAAAAApeFY0EmNpZGF0YSAgICAgRkFUMTIgICAOH75bfKwiwHQLVrQOuwcAzRBe6/Ay5M0WzRnr/lRoaXMgaXMgbm90IGEgYm9vdGFibGUgZGlzay4gIFBsZWFzZSBpbnNlcnQgYSBib290YWJsZSBmbG9wcHkgYW5kDQpwcmVzcyBhbnkga2V5IHRvIHRyeSBhZ2FpbiAuLi4gDQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVar4////DwAAAAAAAAAA8P//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////D/j///8PAAAAAAAAAADw//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////8PY2lkYXRhICAgICAIAAAAAAAAAAAAAAAAAAAAAAAAAABBbQBlAHQAYQAtAA8A1GQAYQB0AGEAAAD//wAA/////01FVEEtRH4xICAgAABt67QMWQxZAADrtAxZAgC3AAAAQXUAcwBlAHIALQAPALBkAGEAdABhAAAA//8AAP////9VU0VSLUR+MSAgIAAAbeu0DFkMWQAA67QMWQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHsiaW5zdGFuY2UtaWQiOiI2NzUxZDM2ZS1lMmYzLTRjNGUtOTQwMy04ODgyNmEzNWRmOTEiLCJsb2NhbC1ob3N0bmFtZSI6ImxvY2Fsc2hhcmsiLCJwdWJsaWMta2V5cyI6WyJzc2gtZWQyNTUxOSBBQUFBQzNOemFDMWxaREkxTlRFNUFBQUFJS0tCam5lY3hsaFlMbW1xT2IrYkJ1WkhiZ1F4Z3JDWU5uN2gvTy96MWhqRiJdfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA") } instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 instance_runtime = InstanceRuntimeState { propolis_id: Some(da3cbda3-0737-4959-962d-a7999be1623b (propolis)), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-08-12T22:39:22.789496622Z } metadata = InstanceMetadata { silo_id: 20403355-95b9-42f5-9d9e-91bac4880358, project_id: d30a2c80-385c-41cf-8ae0-f7bb2786d469 } propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = da3cbda3-0737-4959-962d-a7999be1623b vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2024-08-12T22:39:22.741212Z }
24482024-08-12T22:39:23.134ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:608 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 (instance)
24492024-08-12T22:39:23.135ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:999 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 migration_id = None propolis_id = da3cbda3-0737-4959-962d-a7999be1623b state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: fbe24bec-ccf8-4e6f-a407-f4c01f58602b, kind: Instance { id: 6751d36e-e2f3-4c4e-9403-88826a35df91 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 125, 106])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(1237126), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: fbe24bec-ccf8-4e6f-a407-f4c01f58602b, kind: Instance { id: 6751d36e-e2f3-4c4e-9403-88826a35df91 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 125, 106])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(1237126), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: fbe24bec-ccf8-4e6f-a407-f4c01f58602b, kind: Instance { id: 6751d36e-e2f3-4c4e-9403-88826a35df91 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 125, 106])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(1237126), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(1237126))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, VpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: fbe24bec-ccf8-4e6f-a407-f4c01f58602b, kind: Instance { id: 6751d36e-e2f3-4c4e-9403-88826a35df91 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 125, 106])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(1237126), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-30e3dd3f278", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 6364a4bb-4411-48dc-b68e-c9426e8b27f5, read_only_parent: Some(Volume { block_size: 512, id: 34c7e218-1a16-4d12-af8a-65e92675126f, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 0e4ac7e1-9472-4f66-954e-6ec94ea1b92a, key: Some("syMSLYwARdVwIcfCgl7f/Gz+sViPuhy4XcurX9XWsZs="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19001", "[fd00:1122:3344:101::12]:19001", "[fd00:1122:3344:101::13]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 6364a4bb-4411-48dc-b68e-c9426e8b27f5, key: Some("WkrMz60P67sO+4kKRVzLC22lhF9EFIdkKIrYhLUKN/8="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19002", "[fd00:1122:3344:101::14]:19000", "[fd00:1122:3344:101::16]:19000"] } }] } }], cloud_init_bytes: Some("6zyQTVNXSU40LjEAAgEBAAIAAioA+AEAKgBAAAAAAAAAAAAAAAApeFY0EmNpZGF0YSAgICAgRkFUMTIgICAOH75bfKwiwHQLVrQOuwcAzRBe6/Ay5M0WzRnr/lRoaXMgaXMgbm90IGEgYm9vdGFibGUgZGlzay4gIFBsZWFzZSBpbnNlcnQgYSBib290YWJsZSBmbG9wcHkgYW5kDQpwcmVzcyBhbnkga2V5IHRvIHRyeSBhZ2FpbiAuLi4gDQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVar4////DwAAAAAAAAAA8P//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////D/j///8PAAAAAAAAAADw//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////8PY2lkYXRhICAgICAIAAAAAAAAAAAAAAAAAAAAAAAAAABBbQBlAHQAYQAtAA8A1GQAYQB0AGEAAAD//wAA/////01FVEEtRH4xICAgAABt67QMWQxZAADrtAxZAgC3AAAAQXUAcwBlAHIALQAPALBkAGEAdABhAAAA//8AAP////9VU0VSLUR+MSAgIAAAbeu0DFkMWQAA67QMWQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHsiaW5zdGFuY2UtaWQiOiI2NzUxZDM2ZS1lMmYzLTRjNGUtOTQwMy04ODgyNmEzNWRmOTEiLCJsb2NhbC1ob3N0bmFtZSI6ImxvY2Fsc2hhcmsiLCJwdWJsaWMta2V5cyI6WyJzc2gtZWQyNTUxOSBBQUFBQzNOemFDMWxaREkxTlRFNUFBQUFJS0tCam5lY3hsaFlMbW1xT2IrYkJ1WkhiZ1F4Z3JDWU5uN2gvTy96MWhqRiJdfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA") }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2024-08-12T22:39:22.741212Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
24502024-08-12T22:39:23.136ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 7841 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34083 req_id = 14f8f69c-1e44-4bfe-8072-16796f607588 response_code = 200 uri = /instances/6751d36e-e2f3-4c4e-9403-88826a35df91
24512024-08-12T22:39:23.233ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35724
24522024-08-12T22:39:23.234ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 568 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35724 req_id = 4faff882-38d0-49b2-bce9-a4fa9d1cb980 response_code = 200 uri = /instances/6751d36e-e2f3-4c4e-9403-88826a35df91/state
24532024-08-12T22:39:23.235ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 247, 125, 106]), slot: 0, vni: Vni { inner: 1237126 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
24542024-08-12T22:39:23.235ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
24552024-08-12T22:39:23.271ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b file = illumos-utils/src/zone.rs:316 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24562024-08-12T22:39:23.293ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b file = illumos-utils/src/zone.rs:354 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24572024-08-12T22:39:24.675ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24582024-08-12T22:39:24.675ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b zone file = illumos-utils/src/running_zone.rs:488 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 zone = oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b
24592024-08-12T22:39:24.872ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b") failed: Property not found. retry in 64.542059ms file = illumos-utils/src/svc.rs:36 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 zone = oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b
24602024-08-12T22:39:24.958ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b") failed: Property not found. retry in 115.197065ms file = illumos-utils/src/svc.rs:36 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 zone = oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b
24612024-08-12T22:39:25.088ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b") failed: Property not found. retry in 218.936935ms file = illumos-utils/src/svc.rs:36 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 zone = oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b
24622024-08-12T22:39:25.319ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b") failed: Property not found. retry in 277.810268ms file = illumos-utils/src/svc.rs:36 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 zone = oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b
24632024-08-12T22:39:25.607ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b") failed: Property not found. retry in 992.579503ms file = illumos-utils/src/svc.rs:36 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 zone = oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b
24642024-08-12T22:39:26.611ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b") failed: Property not found. retry in 706.396877ms file = illumos-utils/src/svc.rs:36 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 zone = oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b
24652024-08-12T22:39:27.327ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b") failed: Property not found. retry in 1.289612821s file = illumos-utils/src/svc.rs:36 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 zone = oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b
24662024-08-12T22:39:28.627ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b") failed: Property not found. retry in 1.205644792s file = illumos-utils/src/svc.rs:36 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 zone = oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b
24672024-08-12T22:39:29.860ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b file = sled-agent/src/instance.rs:1453 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24682024-08-12T22:39:29.867ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b") failed: Property not found. retry in 43.972918ms file = illumos-utils/src/svc.rs:36 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24692024-08-12T22:39:29.920ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b") failed: Property not found. retry in 112.385265ms file = illumos-utils/src/svc.rs:36 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24702024-08-12T22:39:30.045ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b") failed: Property not found. retry in 117.213476ms file = illumos-utils/src/svc.rs:36 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24712024-08-12T22:39:30.172ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b") failed: Property not found. retry in 283.804159ms file = illumos-utils/src/svc.rs:36 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24722024-08-12T22:39:30.468ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1462 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24732024-08-12T22:39:30.506ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1486 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24742024-08-12T22:39:30.507ZINFOSledAgent (InstanceManager): Sending ensure request to propolis: InstanceEnsureRequest { cloud_init_bytes: Some("6zyQTVNXSU40LjEAAgEBAAIAAioA+AEAKgBAAAAAAAAAAAAAAAApeFY0EmNpZGF0YSAgICAgRkFUMTIgICAOH75bfKwiwHQLVrQOuwcAzRBe6/Ay5M0WzRnr/lRoaXMgaXMgbm90IGEgYm9vdGFibGUgZGlzay4gIFBsZWFzZSBpbnNlcnQgYSBib290YWJsZSBmbG9wcHkgYW5kDQpwcmVzcyBhbnkga2V5IHRvIHRyeSBhZ2FpbiAuLi4gDQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVar4////DwAAAAAAAAAA8P//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////D/j///8PAAAAAAAAAADw//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////8PY2lkYXRhICAgICAIAAAAAAAAAAAAAAAAAAAAAAAAAABBbQBlAHQAYQAtAA8A1GQAYQB0AGEAAAD//wAA/////01FVEEtRH4xICAgAABt67QMWQxZAADrtAxZAgC3AAAAQXUAcwBlAHIALQAPALBkAGEAdABhAAAA//8AAP////9VU0VSLUR+MSAgIAAAbeu0DFkMWQAA67QMWQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHsiaW5zdGFuY2UtaWQiOiI2NzUxZDM2ZS1lMmYzLTRjNGUtOTQwMy04ODgyNmEzNWRmOTEiLCJsb2NhbC1ob3N0bmFtZSI6ImxvY2Fsc2hhcmsiLCJwdWJsaWMta2V5cyI6WyJzc2gtZWQyNTUxOSBBQUFBQzNOemFDMWxaREkxTlRFNUFBQUFJS0tCam5lY3hsaFlMbW1xT2IrYkJ1WkhiZ1F4Z3JDWU5uN2gvTy96MWhqRiJdfQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"), disks: [DiskRequest { device: "nvme", name: "disk-30e3dd3f278", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 6364a4bb-4411-48dc-b68e-c9426e8b27f5, read_only_parent: Some(Volume { block_size: 512, id: 34c7e218-1a16-4d12-af8a-65e92675126f, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 0e4ac7e1-9472-4f66-954e-6ec94ea1b92a, key: Some("syMSLYwARdVwIcfCgl7f/Gz+sViPuhy4XcurX9XWsZs="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19001", "[fd00:1122:3344:101::12]:19001", "[fd00:1122:3344:101::13]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 6364a4bb-4411-48dc-b68e-c9426e8b27f5, key: Some("WkrMz60P67sO+4kKRVzLC22lhF9EFIdkKIrYhLUKN/8="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19002", "[fd00:1122:3344:101::14]:19000", "[fd00:1122:3344:101::16]:19000"] } }] } }], migrate: None, nics: [NetworkInterfaceRequest { name: "opte6", slot: Slot(0) }], properties: InstanceProperties { bootrom_id: 00000000-0000-0000-0000-000000000000, description: "Test description", id: 6751d36e-e2f3-4c4e-9403-88826a35df91, image_id: 00000000-0000-0000-0000-000000000000, memory: 1024, metadata: InstanceMetadata { project_id: d30a2c80-385c-41cf-8ae0-f7bb2786d469, silo_id: 20403355-95b9-42f5-9d9e-91bac4880358, sled_id: 6e18d78c-496f-48cb-b07e-b18dbc665f73, sled_model: "i86pc", sled_revision: 0, sled_serial: "sock" }, name: "localshark", vcpus: 2 } } file = sled-agent/src/instance.rs:735 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24752024-08-12T22:39:30.646ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:737 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24762024-08-12T22:39:30.649ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-08-12T22:39:30.649943742Z } file = sled-agent/src/instance.rs:605 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24772024-08-12T22:39:30.649ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:627 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-08-12T22:39:30.649943742Z } propolis_id = da3cbda3-0737-4959-962d-a7999be1623b
24782024-08-12T22:39:30.650ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:525 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 state = SledInstanceState { propolis_id: da3cbda3-0737-4959-962d-a7999be1623b (propolis), vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-08-12T22:39:30.649943742Z }, migration_in: None, migration_out: None }
24792024-08-12T22:39:30.680ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-08-12T22:39:30.680501141Z } file = sled-agent/src/instance.rs:605 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
24802024-08-12T22:39:30.680ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:627 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-08-12T22:39:30.680501141Z } propolis_id = da3cbda3-0737-4959-962d-a7999be1623b
24812024-08-12T22:39:30.680ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:525 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 state = SledInstanceState { propolis_id: da3cbda3-0737-4959-962d-a7999be1623b (propolis), vmm_state: VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-08-12T22:39:30.680501141Z }, migration_in: None, migration_out: None }
24822024-08-12T22:39:37.241ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46815
24832024-08-12T22:39:37.242ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 344 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46815 req_id = 8ac7d15d-1ef1-4a20-a282-b7297a4607a7 response_code = 200 uri = /instances/6751d36e-e2f3-4c4e-9403-88826a35df91/state
24842024-08-12T22:39:37.299ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43232
24852024-08-12T22:39:37.300ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43232 req_id = 5c0c0424-ed4b-470e-8766-bc40b89e708d response_code = 200 uri = /v2p
24862024-08-12T22:39:37.346ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37005
24872024-08-12T22:39:37.346ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37005 req_id = f57a3e35-f187-42de-9dcf-a7542cbf2a27 response_code = 200 uri = /vpc-routes
24882024-08-12T22:39:37.396ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 264 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37005 req_id = a4bc9021-f7ca-4c42-8904-71fe4997b80d response_code = 204 uri = /vpc-routes
24892024-08-12T22:39:37.419ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39650
24902024-08-12T22:39:37.419ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39650 req_id = ec7ad998-5181-4b3c-ae93-251d3783169f response_code = 200 uri = /v2p
24912024-08-12T22:39:37.435ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60392
24922024-08-12T22:39:37.435ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60392 req_id = ca003214-9c2e-467d-9a85-0aa022f1c233 response_code = 200 uri = /instances/6751d36e-e2f3-4c4e-9403-88826a35df91/state
24932024-08-12T22:39:37.476ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56221
24942024-08-12T22:39:37.476ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56221 req_id = 89c1fc11-7886-43cb-8727-32a516ce0d7d response_code = 200 uri = /vpc-routes
24952024-08-12T22:39:37.490ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56221 req_id = 10864905-ff4e-4a7f-9c07-1b1735512756 response_code = 204 uri = /vpc-routes
24962024-08-12T22:39:37.570ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58000
24972024-08-12T22:39:37.571ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
24982024-08-12T22:39:37.594ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
24992024-08-12T22:39:37.614ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 44334 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:58000 req_id = ec94353f-64f2-4afa-891e-ef1fafaacb9f response_code = 204 uri = /switch-ports
25002024-08-12T22:39:37.615ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58000 req_id = ee1db1f3-bebf-4b38-b3bb-0b914b3544c3 response_code = 200 uri = /network-bootstore-config
25012024-08-12T22:39:37.618ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50389
25022024-08-12T22:39:37.619ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
25032024-08-12T22:39:37.640ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
25042024-08-12T22:39:37.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 41175 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50389 req_id = a45f680e-d622-4346-bab1-c2f6be2bfd87 response_code = 204 uri = /switch-ports
25052024-08-12T22:39:37.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50389 req_id = 0fe54408-9b69-412c-8484-b94bff60b3d6 response_code = 200 uri = /network-bootstore-config
25062024-08-12T22:39:38.133ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44977
25072024-08-12T22:39:38.133ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44977 req_id = e7c08cb5-2179-446b-99c3-1fb42eb2cde0 response_code = 200 uri = /vpc-routes
25082024-08-12T22:39:38.148ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44977 req_id = d333a51d-78f4-4371-a247-7df1ce7e695a response_code = 204 uri = /vpc-routes
25092024-08-12T22:39:38.181ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51166
25102024-08-12T22:39:38.181ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51166 req_id = af296e61-3c5b-4cac-abb3-05ecccb8398e response_code = 200 uri = /instances/6751d36e-e2f3-4c4e-9403-88826a35df91/state
25112024-08-12T22:39:38.203ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64789
25122024-08-12T22:39:38.203ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64789 req_id = 45d174c1-5f73-411b-8f7c-a12120a24e4a response_code = 200 uri = /v2p
25132024-08-12T22:39:38.380ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44390
25142024-08-12T22:39:38.380ZINFOSledAgent (ServiceManager): Setting up uplinkd service file = sled-agent/src/services.rs:3885
25152024-08-12T22:39:38.405ZINFOSledAgent (ServiceManager): configuring port: HostPortConfig { port: "qsfp0", addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }] } file = sled-agent/src/services.rs:3896
25162024-08-12T22:39:38.425ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 45218 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:44390 req_id = fce78629-237c-4722-9158-3e189d64f71d response_code = 204 uri = /switch-ports
25172024-08-12T22:39:38.425ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44390 req_id = 0afd2cfc-0262-45ba-ba25-04f99ce217e8 response_code = 200 uri = /network-bootstore-config
25182024-08-12T22:39:40.100ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 6645 local_addr = [fd00:1122:3344:101::1]:53671 method = GET remote_addr = [fd00:1122:3344:101::d]:50227 req_id = 01650fb1-51c2-43e5-904f-3c890744c651 response_code = 200 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73 uri = /6e18d78c-496f-48cb-b07e-b18dbc665f73
25192024-08-12T22:39:40.698ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54805
25202024-08-12T22:39:40.700ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1386 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54805 req_id = 9003a423-1b4a-4506-affd-107cc772fd15 response_code = 200 uri = /instances/6751d36e-e2f3-4c4e-9403-88826a35df91/state
25212024-08-12T22:39:40.700ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-08-12T22:39:40.700266967Z } file = sled-agent/src/instance.rs:605 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
25222024-08-12T22:39:40.700ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:627 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-08-12T22:39:40.700266967Z } propolis_id = da3cbda3-0737-4959-962d-a7999be1623b
25232024-08-12T22:39:40.700ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:525 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 state = SledInstanceState { propolis_id: da3cbda3-0737-4959-962d-a7999be1623b (propolis), vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-08-12T22:39:40.700266967Z }, migration_in: None, migration_out: None }
25242024-08-12T22:39:40.713ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-08-12T22:39:40.713036275Z } file = sled-agent/src/instance.rs:605 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
25252024-08-12T22:39:40.713ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:627 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(6), time_updated: 2024-08-12T22:39:40.713036275Z } propolis_id = da3cbda3-0737-4959-962d-a7999be1623b
25262024-08-12T22:39:40.713ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:641 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
25272024-08-12T22:39:40.713ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
25282024-08-12T22:39:40.713ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:557 id = 13933444234691173538 n_samples = 6 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
25292024-08-12T22:39:40.713ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:557 id = 9003272580222728223 n_samples = 6 sled_id = 6e18d78c-496f-48cb-b07e-b18dbc665f73
25302024-08-12T22:39:40.716ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:405
25312024-08-12T22:39:40.716ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/2acd7d8f-9546-4b2f-a46b-354a4078679d/debug/bundle/zone", "/pool/int/b9292fe7-3702-412e-ade5-90d5eadbfb98/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/537dde5b-089b-4e20-9ef8-bf10127d7dce/crypt/debug", "/pool/ext/90f5d185-2468-460f-8d20-ecb50330e694/crypt/debug", "/pool/ext/9ac4fc8d-4a30-4017-bbdc-a392e988e5c6/crypt/debug", "/pool/ext/fab24dee-2d14-4ad6-8dd1-967cd0dd455c/crypt/debug", "/pool/ext/37abb3ae-dbff-4acb-a865-55b4ed0374e9/crypt/debug", "/pool/ext/ca45bffe-f952-4bbd-bcc4-be5de210718a/crypt/debug", "/pool/ext/65eb96fa-5b90-4e6e-968e-0b7f2a4634d3/crypt/debug", "/pool/ext/ef93b2f7-5e79-4766-bedb-2c6d5ba93538/crypt/debug", "/pool/ext/09108500-899b-4c15-aa1f-0c8295b5001c/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:445 zone_name = oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b
25322024-08-12T22:39:41.315ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1249 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b", bundle_id: 9473b532-4195-4924-ac3a-31793ac73220 }, time_created: 2024-08-12T22:39:40.716403878Z, version: 0, cause: TerminatedInstance }
25332024-08-12T22:39:41.316ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_da3cbda3-0737-4959-962d-a7999be1623b file = sled-agent/src/instance.rs:836 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
25342024-08-12T22:39:43.056ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:270 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
25352024-08-12T22:39:43.072ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(1237126), kind: System }
25362024-08-12T22:39:43.072ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(1237126), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
25372024-08-12T22:39:43.073ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:525 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 state = SledInstanceState { propolis_id: da3cbda3-0737-4959-962d-a7999be1623b (propolis), vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-12T22:39:43.073233192Z }, migration_in: None, migration_out: None }
25382024-08-12T22:39:43.096ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:525 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91 state = SledInstanceState { propolis_id: da3cbda3-0737-4959-962d-a7999be1623b (propolis), vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-12T22:39:43.073233192Z }, migration_in: None, migration_out: None }
25392024-08-12T22:39:43.101ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:773 instance_id = 6751d36e-e2f3-4c4e-9403-88826a35df91
25402024-08-12T22:39:43.484ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37333
25412024-08-12T22:39:43.484ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37333 req_id = 712e4e09-9fa0-4cf1-b57d-28e9eed806f3 response_code = 200 uri = /vpc-routes
25422024-08-12T22:39:43.492ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:37333 req_id = b1270610-a113-4fbc-8cb0-1503bcc3fcd5 response_code = 204 uri = /vpc-routes
25432024-08-12T22:39:43.501ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34723
25442024-08-12T22:39:43.501ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34723 req_id = f66f07cd-94c6-47d5-900d-322e569a574e response_code = 200 uri = /v2p
25452024-08-12T22:39:43.640ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47244
25462024-08-12T22:39:43.641ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 825 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47244 req_id = f834cbcd-3b70-46a1-b8ac-696ca64cdfc1 response_code = 200 uri = /v2p
25472024-08-12T22:39:44.709ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53016
25482024-08-12T22:39:44.709ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53016 req_id = 71e1d497-6a50-4a43-9a45-d8db6fb98cab response_code = 200 uri = /vpc-routes
25492024-08-12T22:39:44.717ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53016 req_id = 8d2647d1-8281-4d7e-9a1d-8d06a471bfad response_code = 204 uri = /vpc-routes