|
|
|
| 1 | 2025-12-16T23:51:29.917Z | job dependencies complete; ready to run (waiting for 42 m 43 s) |
| 2 | 2025-12-17T08:35:22.122Z | job assigned to worker 01KCNPX0Y8TJMCMXPRYF78HVS5 [factory lab, buskin/18172] (queued for 8 h 43 m 52 s) |
| 3 | 2025-12-17T08:35:22.144Z | http://172.20.0.5:9969/ipxe/buskin... ok |
| 4 | 2025-12-17T08:35:22.186Z | http://172.20.0.5:9969/os/buskin/platform/i86pc/kernel/amd64/unix... ok |
| 5 | 2025-12-17T08:35:22.220Z | http://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive... ok |
| 6 | 2025-12-17T08:35:22.253Z | http://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive.hash... ok |
| 7 | 2025-12-17T08:35:22.287Z | http://172.20.0.5:9969/postboot/buskin... ok |
| 8 | 2025-12-17T08:35:22.320Z | wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U |
| 9 | 2025-12-17T08:35:22.354Z | cpu0: microcode has been updated from version 0xa001133 to 0xa0011d5 |
| 10 | 2025-12-17T08:35:22.388Z | Oxide Helios Version helios-2.0.23457 64-bit |
| 11 | 2025-12-17T08:35:22.421Z | Configuring devices. |
| 12 | 2025-12-17T08:35:22.455Z | ################################################## |
| 13 | 2025-12-17T08:35:22.488Z | #### ############# ## |
| 14 | 2025-12-17T08:35:22.522Z | ### ### ############ ## |
| 15 | 2025-12-17T08:35:22.555Z | ## ### # ## ### ## Oxide Computer Company ## |
| 16 | 2025-12-17T08:35:22.589Z | ## ## ## ### # ### ## |
| 17 | 2025-12-17T08:35:22.650Z | ## # ### #### #### This Station Under ## |
| 18 | 2025-12-17T08:35:22.684Z | ### ### #### # ### Computer Control ## |
| 19 | 2025-12-17T08:35:22.717Z | #### #### ### ## ## |
| 20 | 2025-12-17T08:35:22.751Z | ################################################## |
| 21 | 2025-12-17T08:35:22.785Z | Hostname: buskin |
| 22 | 2025-12-17T08:35:22.822Z | Loading smf(7) service descriptions: |
| 23 | 2025-12-17T08:35:23.648Z | downloading input: /input/package/work/package.tar.gz |
| 24 | 2025-12-17T09:07:47.686Z | downloaded input: /input/package/work/package.tar.gz |
| |
| 25 | 2025-12-17T09:07:47.711Z | starting task 0: "setup" |
| 26 | 2025-12-17T09:07:47.711Z | ++ uname -s |
| 27 | 2025-12-17T09:07:47.711Z | + kern=SunOS |
| 28 | 2025-12-17T09:07:47.711Z | + build_user=build |
| 29 | 2025-12-17T09:07:47.711Z | + build_uid=12345 |
| 30 | 2025-12-17T09:07:47.711Z | + work_dir=/work |
| 31 | 2025-12-17T09:07:47.711Z | + input_dir=/input |
| 32 | 2025-12-17T09:07:47.711Z | + [[ 0 == 12345 ]] |
| 33 | 2025-12-17T09:07:47.711Z | + case "$kern" in |
| 34 | 2025-12-17T09:07:47.711Z | + groupadd -g 12345 build |
| 35 | 2025-12-17T09:07:47.711Z | + useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build |
| 36 | 2025-12-17T09:07:49.704Z | + zfs create -o mountpoint=/work rpool/work |
| 37 | 2025-12-17T09:07:49.736Z | ++ awk '$2 == "/home" { print $3 }' /etc/mnttab |
| 38 | 2025-12-17T09:07:49.760Z | + home_fs=zfs |
| 39 | 2025-12-17T09:07:49.760Z | + [[ zfs == autofs ]] |
| 40 | 2025-12-17T09:07:49.760Z | + mkdir -p /home/build |
| 41 | 2025-12-17T09:07:49.760Z | + chown build:build /home/build /work |
| 42 | 2025-12-17T09:07:51.743Z | + chmod 0700 /home/build /work |
| 43 | 2025-12-17T09:07:51.767Z | process exited: duration 4059 ms, exit code 0 |
| |
| 44 | 2025-12-17T09:07:51.815Z | starting task 1: "authentication" |
| 45 | 2025-12-17T09:07:51.860Z | process exited: duration 44 ms, exit code 0 |
| |
| 46 | 2025-12-17T09:07:51.908Z | starting task 2: "build" |
| 47 | 2025-12-17T09:07:51.932Z | + trap _exit_trap EXIT |
| 48 | 2025-12-17T09:07:51.933Z | + OPTE_COMMIT= |
| 49 | 2025-12-17T09:07:51.933Z | + [[ x != \x ]] |
| 50 | 2025-12-17T09:07:51.933Z | + pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3' |
| 51 | 2025-12-17T09:07:52.027Z | secpolicy_fs_linkdir: 0x55 = 0x55 |
| 52 | 2025-12-17T09:07:52.027Z | secpolicy_fs_linkdir+1: 0xb8 = 0x48 |
| 53 | 2025-12-17T09:07:52.027Z | secpolicy_fs_linkdir+2: 0x1 = 0x89 |
| 54 | 2025-12-17T09:07:52.027Z | secpolicy_fs_linkdir+3: 0 = 0xe5 |
| 55 | 2025-12-17T09:07:52.027Z | secpolicy_fs_linkdir+4: 0 = 0xb8 |
| 56 | 2025-12-17T09:07:52.027Z | secpolicy_fs_linkdir+5: 0 = 0x1 |
| 57 | 2025-12-17T09:07:52.027Z | secpolicy_fs_linkdir+6: 0x48 = 0x0 |
| 58 | 2025-12-17T09:07:52.027Z | secpolicy_fs_linkdir+7: 0x89 = 0x0 |
| 59 | 2025-12-17T09:07:52.027Z | secpolicy_fs_linkdir+8: 0xe5 = 0x0 |
| 60 | 2025-12-17T09:07:52.027Z | secpolicy_fs_linkdir+9: 0x48 = 0x5d |
| 61 | 2025-12-17T09:07:52.027Z | secpolicy_fs_linkdir+0xa: 0x83 = 0xc3 |
| 62 | 2025-12-17T09:07:52.051Z | + [[ -d /opt/oxide ]] |
| 63 | 2025-12-17T09:07:52.051Z | + pfexec mv /opt/oxide /opt/oxide-underneath |
| 64 | 2025-12-17T09:07:52.051Z | + pfexec mkdir /opt/oxide |
| 65 | 2025-12-17T09:07:52.052Z | + pfexec mount -F tmpfs -O swap /opt/oxide |
| 66 | 2025-12-17T09:07:52.052Z | + [[ -d /opt/oxide-underneath ]] |
| 67 | 2025-12-17T09:07:52.052Z | + cd /opt/oxide-underneath |
| 68 | 2025-12-17T09:07:52.052Z | + pfexec tar ceEp@/f - . |
| 69 | 2025-12-17T09:07:52.052Z | + cd /opt/oxide |
| 70 | 2025-12-17T09:07:52.052Z | + pfexec tar xveEp@/f - |
| 71 | 2025-12-17T09:07:52.092Z | x ., 0 bytes, 0 tape blocks |
| 72 | 2025-12-17T09:07:52.092Z | x ./opte, 0 bytes, 0 tape blocks |
| 73 | 2025-12-17T09:07:52.093Z | x ./opte/bin, 0 bytes, 0 tape blocks |
| 74 | 2025-12-17T09:07:52.093Z | x ./opte/bin/opteadm, 20875880 bytes, 40774 tape blocks |
| 75 | 2025-12-17T09:07:52.117Z | + pfexec rm -rf /opt/oxide-underneath |
| 76 | 2025-12-17T09:07:52.117Z | + pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone |
| 77 | 2025-12-17T09:07:52.141Z | + pfexec mkdir /opt/oxide/work |
| 78 | 2025-12-17T09:07:52.141Z | + pfexec chown build:build /opt/oxide/work |
| 79 | 2025-12-17T09:07:52.141Z | + cd /opt/oxide/work |
| 80 | 2025-12-17T09:07:52.141Z | + ptime -m tar xvzf /input/package/work/package.tar.gz |
| 81 | 2025-12-17T09:08:13.698Z | Decompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'... |
| 82 | 2025-12-17T09:08:13.698Z | x .github/buildomat/ci-env.sh, 196 bytes, 1 tape blocks |
| 83 | 2025-12-17T09:08:13.699Z | x out/target/test, 107 bytes, 1 tape blocks |
| 84 | 2025-12-17T09:08:13.699Z | x out/npuzone/npuzone, 23699288 bytes, 46288 tape blocks |
| 85 | 2025-12-17T09:08:13.699Z | x package-manifest.toml, 34403 bytes, 68 tape blocks |
| 86 | 2025-12-17T09:08:13.699Z | x smf/sled-agent/non-gimlet/config.toml, 5158 bytes, 11 tape blocks |
| 87 | 2025-12-17T09:08:13.699Z | x target/release/omicron-package, 43805728 bytes, 85559 tape blocks |
| 88 | 2025-12-17T09:08:13.699Z | x target/release/xtask, 11738288 bytes, 22927 tape blocks |
| 89 | 2025-12-17T09:08:13.699Z | x target/debug/bootstrap, 321528504 bytes, 627986 tape blocks |
| 90 | 2025-12-17T09:08:13.699Z | x tests/bootstrap-fcf42bdac5012dcb, 31868752 bytes, 62244 tape blocks |
| 91 | 2025-12-17T09:08:13.699Z | x tests/commtest-64d34f41dc6c9ef7, 32732592 bytes, 63931 tape blocks |
| 92 | 2025-12-17T09:08:13.699Z | x tests/dhcp_server-66261b5397a21974, 31868768 bytes, 62244 tape blocks |
| 93 | 2025-12-17T09:08:13.699Z | x tests/end_to_end_tests-bc05e9f1e2f5bdce, 366753632 bytes, 716316 tape blocks |
| 94 | 2025-12-17T09:08:13.699Z | x out/clickhouse.tar.gz, 327859639 bytes, 640351 tape blocks |
| 95 | 2025-12-17T09:08:13.699Z | x out/clickhouse_keeper.tar.gz, 310453867 bytes, 606356 tape blocks |
| 96 | 2025-12-17T09:08:13.699Z | x out/clickhouse_server.tar.gz, 327858731 bytes, 640350 tape blocks |
| 97 | 2025-12-17T09:08:13.699Z | x out/cockroachdb.tar.gz, 171830228 bytes, 335606 tape blocks |
| 98 | 2025-12-17T09:08:13.699Z | x out/crucible-pantry-zone.tar.gz, 45524448 bytes, 88915 tape blocks |
| 99 | 2025-12-17T09:08:13.699Z | x out/crucible-utils.tar, 18169344 bytes, 35487 tape blocks |
| 100 | 2025-12-17T09:08:13.699Z | x out/crucible-zone.tar.gz, 60001867 bytes, 117192 tape blocks |
| 101 | 2025-12-17T09:08:13.699Z | x out/external-dns.tar.gz, 54771961 bytes, 106977 tape blocks |
| 102 | 2025-12-17T09:08:13.699Z | x out/internal-dns.tar.gz, 54771561 bytes, 106976 tape blocks |
| 103 | 2025-12-17T09:08:13.699Z | x out/mg-ddm-gz.tar, 116766720 bytes, 228060 tape blocks |
| 104 | 2025-12-17T09:08:13.699Z | x out/nexus.tar.gz, 199707198 bytes, 390054 tape blocks |
| 105 | 2025-12-17T09:08:13.699Z | x out/ntp.tar.gz, 35635174 bytes, 69600 tape blocks |
| 106 | 2025-12-17T09:08:13.699Z | x out/omicron-gateway-softnpu.tar.gz, 28896946 bytes, 56440 tape blocks |
| 107 | 2025-12-17T09:08:13.699Z | x out/omicron-sled-agent.tar, 197443584 bytes, 385632 tape blocks |
| 108 | 2025-12-17T09:08:13.699Z | x out/overlay.tar.gz, 3030 bytes, 6 tape blocks |
| 109 | 2025-12-17T09:08:13.700Z | x out/oximeter.tar.gz, 67783882 bytes, 132391 tape blocks |
| 110 | 2025-12-17T09:08:13.700Z | x out/oxlog.tar, 4036608 bytes, 7884 tape blocks |
| 111 | 2025-12-17T09:08:13.700Z | x out/probe.tar.gz, 3109568 bytes, 6074 tape blocks |
| 112 | 2025-12-17T09:08:13.700Z | x out/propolis-server.tar.gz, 75246872 bytes, 146967 tape blocks |
| 113 | 2025-12-17T09:08:13.700Z | x out/pumpkind-gz.tar, 27286528 bytes, 53294 tape blocks |
| 114 | 2025-12-17T09:08:13.700Z | x out/switch-softnpu.tar.gz, 400806153 bytes, 782825 tape blocks |
| 115 | 2025-12-17T09:08:13.725Z | |
| 116 | 2025-12-17T09:08:13.725Z | real 21.570319500 |
| 117 | 2025-12-17T09:08:13.725Z | user 19.255220753 |
| 118 | 2025-12-17T09:08:13.725Z | sys 3.920214920 |
| 119 | 2025-12-17T09:08:13.725Z | trap 0.000891904 |
| 120 | 2025-12-17T09:08:13.725Z | tflt 0.000000000 |
| 121 | 2025-12-17T09:08:13.725Z | dflt 0.000000000 |
| 122 | 2025-12-17T09:08:13.725Z | kflt 0.000000000 |
| 123 | 2025-12-17T09:08:13.725Z | lock 0.000000000 |
| 124 | 2025-12-17T09:08:13.725Z | slp 19.417837364 |
| 125 | 2025-12-17T09:08:13.725Z | lat 0.544466917 |
| 126 | 2025-12-17T09:08:13.725Z | stop 0.000125093 |
| 127 | 2025-12-17T09:08:13.725Z | + source .github/buildomat/ci-env.sh |
| 128 | 2025-12-17T09:08:13.725Z | ++ export CARGO_TERM_COLOR=always |
| 129 | 2025-12-17T09:08:13.725Z | ++ CARGO_TERM_COLOR=always |
| 130 | 2025-12-17T09:08:13.725Z | + bmat address ls |
| 131 | 2025-12-17T09:08:13.725Z | NAME CIDR FIRST LAST COUNT |
| 132 | 2025-12-17T09:08:13.725Z | extra 10.151.1.0/24 10.151.1.164 10.151.1.227 64 |
| 133 | 2025-12-17T09:08:13.725Z | ++ bmat address ls -f extra -Ho first,last,count |
| 134 | 2025-12-17T09:08:13.725Z | + set -- 10.151.1.164 10.151.1.227 64 |
| 135 | 2025-12-17T09:08:13.725Z | + EXTRA_IP_START=10.151.1.164 |
| 136 | 2025-12-17T09:08:13.726Z | + EXTRA_IP_END=10.151.1.227 |
| 137 | 2025-12-17T09:08:13.726Z | + EXTRA_IP_COUNT=64 |
| 138 | 2025-12-17T09:08:13.726Z | + (( EXTRA_IP_COUNT >= 32 )) |
| 139 | 2025-12-17T09:08:13.726Z | + EXTRA_IP_BASE=10.151.1 |
| 140 | 2025-12-17T09:08:13.726Z | + EXTRA_IP_FOCTET=164 |
| 141 | 2025-12-17T09:08:13.726Z | + EXTRA_IP_LOCTET=227 |
| 142 | 2025-12-17T09:08:13.726Z | + SERVICE_IP_POOL_START=10.151.1.164 |
| 143 | 2025-12-17T09:08:13.726Z | + SERVICE_IP_POOL_END=10.151.1.173 |
| 144 | 2025-12-17T09:08:13.726Z | + DNS_IP1=10.151.1.164 |
| 145 | 2025-12-17T09:08:13.726Z | + DNS_IP2=10.151.1.165 |
| 146 | 2025-12-17T09:08:13.726Z | + UPLINK_IP=10.151.1.174 |
| 147 | 2025-12-17T09:08:13.726Z | + IPPOOL_START=10.151.1.175 |
| 148 | 2025-12-17T09:08:13.726Z | + IPPOOL_END=10.151.1.227 |
| 149 | 2025-12-17T09:08:13.751Z | ++ ipadm show-addr -po type,addr |
| 150 | 2025-12-17T09:08:13.751Z | ++ awk '-F[:/]' '$1 == "dhcp" {print $2}' |
| 151 | 2025-12-17T09:08:13.751Z | + GATEWAY_IP=10.151.1.100 |
| 152 | 2025-12-17T09:08:13.751Z | + [[ -n 10.151.1.100 ]] |
| 153 | 2025-12-17T09:08:13.751Z | + ping -s 10.151.1.100 56 1 |
| 154 | 2025-12-17T09:08:13.751Z | PING 10.151.1.100: 56 data bytes |
| 155 | 2025-12-17T09:08:13.751Z | 64 bytes from 10.151.1.100: icmp_seq=0. time=0.011 ms |
| 156 | 2025-12-17T09:08:13.751Z | |
| 157 | 2025-12-17T09:08:13.751Z | ----10.151.1.100 PING Statistics---- |
| 158 | 2025-12-17T09:08:13.751Z | 1 packets transmitted, 1 packets received, 0% packet loss |
| 159 | 2025-12-17T09:08:13.752Z | round-trip (ms) min/avg/max/stddev = 0.011/0.011/0.011/-nan |
| 160 | 2025-12-17T09:08:13.752Z | ++ arp -an |
| 161 | 2025-12-17T09:08:13.752Z | ++ awk -vgw=10.151.1.100 '$2 == gw {print $NF}' |
| 162 | 2025-12-17T09:08:13.752Z | + GATEWAY_MAC=18:c0:4d:0d:9f:b2 |
| 163 | 2025-12-17T09:08:13.752Z | + routeadm -e ipv4-forwarding -u |
| 164 | 2025-12-17T09:08:13.854Z | + PXA_START=10.151.1.164 |
| 165 | 2025-12-17T09:08:13.854Z | + PXA_END=10.151.1.227 |
| 166 | 2025-12-17T09:08:13.854Z | + pfexec zpool create -f scratch c1t1d0 c2t1d0 |
| 167 | 2025-12-17T09:08:14.033Z | + ptime -m pfexec ./target/release/xtask virtual-hardware --vdev-dir /scratch create --gateway-ip 10.151.1.100 --gateway-mac 18:c0:4d:0d:9f:b2 --pxa-start 10.151.1.164 --pxa-end 10.151.1.227 |
| 168 | 2025-12-17T09:08:14.057Z | Couldn't find Cargo.toml, using /opt/oxide/work as workspace root |
| 169 | 2025-12-17T09:08:14.125Z | creating virtual hardware |
| 170 | 2025-12-17T09:08:14.149Z | creating /scratch/m2_0.vdev |
| 171 | 2025-12-17T09:08:14.149Z | creating /scratch/m2_1.vdev |
| 172 | 2025-12-17T09:08:14.149Z | creating /scratch/u2_0.vdev |
| 173 | 2025-12-17T09:08:14.149Z | creating /scratch/u2_1.vdev |
| 174 | 2025-12-17T09:08:14.149Z | creating /scratch/u2_2.vdev |
| 175 | 2025-12-17T09:08:14.149Z | creating /scratch/u2_3.vdev |
| 176 | 2025-12-17T09:08:14.149Z | creating /scratch/u2_4.vdev |
| 177 | 2025-12-17T09:08:14.149Z | creating /scratch/u2_5.vdev |
| 178 | 2025-12-17T09:08:14.149Z | creating /scratch/u2_6.vdev |
| 179 | 2025-12-17T09:08:14.149Z | creating /scratch/u2_7.vdev |
| 180 | 2025-12-17T09:08:14.149Z | creating /scratch/u2_8.vdev |
| 181 | 2025-12-17T09:08:14.149Z | Simnet net0/sc0_0 exists |
| 182 | 2025-12-17T09:08:14.173Z | Simnet net1/sc1_0 exists |
| 183 | 2025-12-17T09:08:14.174Z | Vnic sc0_1 exists |
| 184 | 2025-12-17T09:08:29.931Z | Using 10.151.1.100 as gateway ip |
| 185 | 2025-12-17T09:08:29.931Z | using 18:C0:4D:0D:9F:B2 as gateway mac |
| 186 | 2025-12-17T09:08:29.931Z | configuring SoftNPU ARP entry |
| 187 | 2025-12-17T09:08:29.958Z | configuring SoftNPU proxy ARP |
| 188 | 2025-12-17T09:08:30.011Z | SoftNPU state: |
| 189 | 2025-12-17T09:08:30.011Z | local v6: |
| 190 | 2025-12-17T09:08:30.011Z | local v4: |
| 191 | 2025-12-17T09:08:30.011Z | router v6_idx: |
| 192 | 2025-12-17T09:08:30.011Z | router v6_routes: |
| 193 | 2025-12-17T09:08:30.011Z | router v4_idx: |
| 194 | 2025-12-17T09:08:30.011Z | router v4_routes: |
| 195 | 2025-12-17T09:08:30.011Z | resolver v4: |
| 196 | 2025-12-17T09:08:30.011Z | 10.151.1.100 -> 18:c0:4d:0d:9f:b2 |
| 197 | 2025-12-17T09:08:30.011Z | resolver v6: |
| 198 | 2025-12-17T09:08:30.011Z | nat v4: |
| 199 | 2025-12-17T09:08:30.011Z | nat_v6: |
| 200 | 2025-12-17T09:08:30.011Z | port_mac: |
| 201 | 2025-12-17T09:08:30.011Z | proxy arp: |
| 202 | 2025-12-17T09:08:30.011Z | 10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d |
| 203 | 2025-12-17T09:08:30.011Z | created virtual hardware |
| 204 | 2025-12-17T09:08:30.036Z | |
| 205 | 2025-12-17T09:08:30.036Z | real 15.975697085 |
| 206 | 2025-12-17T09:08:30.036Z | user 0.639146835 |
| 207 | 2025-12-17T09:08:30.036Z | sys 3.119433278 |
| 208 | 2025-12-17T09:08:30.036Z | trap 0.000073708 |
| 209 | 2025-12-17T09:08:30.036Z | tflt 0.000060348 |
| 210 | 2025-12-17T09:08:30.036Z | dflt 0.000540214 |
| 211 | 2025-12-17T09:08:30.036Z | kflt 0.000005299 |
| 212 | 2025-12-17T09:08:30.036Z | lock 12:24.101232540 |
| 213 | 2025-12-17T09:08:30.036Z | slp 48.227930079 |
| 214 | 2025-12-17T09:08:30.036Z | lat 0.092365762 |
| 215 | 2025-12-17T09:08:30.036Z | stop 0.368319974 |
| 216 | 2025-12-17T09:08:30.036Z | + tar xf out/omicron-sled-agent.tar pkg/config-rss.toml pkg/config.toml |
| 217 | 2025-12-17T09:08:30.067Z | + sed -E -i~ 's/(m2|u2)(.*\.vdev)/\/scratch\/\1\2/g' pkg/config.toml |
| 218 | 2025-12-17T09:08:30.113Z | + diff -u pkg/config.toml~ pkg/config.toml |
| 219 | 2025-12-17T09:08:30.113Z | --- pkg/config.toml~ Mon Jul 24 01:21:28 2006 |
| 220 | 2025-12-17T09:08:30.113Z | +++ pkg/config.toml Wed Dec 17 09:08:30 2025 |
| 221 | 2025-12-17T09:08:30.113Z | @@ -29,18 +29,18 @@ |
| 222 | 2025-12-17T09:08:30.113Z | # These paths have the prefix of either "u2" or "m2", followed by an underscore, |
| 223 | 2025-12-17T09:08:30.114Z | # followed by a string that is embedded into their fake serial values. |
| 224 | 2025-12-17T09:08:30.114Z | vdevs = [ |
| 225 | 2025-12-17T09:08:30.114Z | - "m2_0.vdev", |
| 226 | 2025-12-17T09:08:30.114Z | - "m2_1.vdev", |
| 227 | 2025-12-17T09:08:30.114Z | + "/scratch/m2_0.vdev", |
| 228 | 2025-12-17T09:08:30.114Z | + "/scratch/m2_1.vdev", |
| 229 | 2025-12-17T09:08:30.114Z | |
| 230 | 2025-12-17T09:08:30.114Z | - "u2_0.vdev", |
| 231 | 2025-12-17T09:08:30.114Z | - "u2_1.vdev", |
| 232 | 2025-12-17T09:08:30.114Z | - "u2_2.vdev", |
| 233 | 2025-12-17T09:08:30.114Z | - "u2_3.vdev", |
| 234 | 2025-12-17T09:08:30.114Z | - "u2_4.vdev", |
| 235 | 2025-12-17T09:08:30.114Z | - "u2_5.vdev", |
| 236 | 2025-12-17T09:08:30.114Z | - "u2_6.vdev", |
| 237 | 2025-12-17T09:08:30.114Z | - "u2_7.vdev", |
| 238 | 2025-12-17T09:08:30.114Z | - "u2_8.vdev", |
| 239 | 2025-12-17T09:08:30.114Z | + "/scratch/u2_0.vdev", |
| 240 | 2025-12-17T09:08:30.114Z | + "/scratch/u2_1.vdev", |
| 241 | 2025-12-17T09:08:30.114Z | + "/scratch/u2_2.vdev", |
| 242 | 2025-12-17T09:08:30.114Z | + "/scratch/u2_3.vdev", |
| 243 | 2025-12-17T09:08:30.114Z | + "/scratch/u2_4.vdev", |
| 244 | 2025-12-17T09:08:30.114Z | + "/scratch/u2_5.vdev", |
| 245 | 2025-12-17T09:08:30.114Z | + "/scratch/u2_6.vdev", |
| 246 | 2025-12-17T09:08:30.114Z | + "/scratch/u2_7.vdev", |
| 247 | 2025-12-17T09:08:30.114Z | + "/scratch/u2_8.vdev", |
| 248 | 2025-12-17T09:08:30.114Z | ] |
| 249 | 2025-12-17T09:08:30.114Z | |
| 250 | 2025-12-17T09:08:30.114Z | # The amount of memory held back for services which exist between zero and one |
| 251 | 2025-12-17T09:08:30.114Z | + true |
| 252 | 2025-12-17T09:08:30.114Z | ++ grep -c -E 'u2.*\.vdev' pkg/config.toml |
| 253 | 2025-12-17T09:08:30.114Z | + EXPECTED_ZPOOL_COUNT=9 |
| 254 | 2025-12-17T09:08:30.114Z | + echo 'expected number of zpools is 9' |
| 255 | 2025-12-17T09:08:30.114Z | expected number of zpools is 9 |
| 256 | 2025-12-17T09:08:30.114Z | ++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml |
| 257 | 2025-12-17T09:08:30.114Z | + SILO_NAME=recovery |
| 258 | 2025-12-17T09:08:30.114Z | ++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml |
| 259 | 2025-12-17T09:08:30.114Z | + EXTERNAL_DNS_DOMAIN=oxide.test |
| 260 | 2025-12-17T09:08:30.114Z | + sed -i~ ' |
| 261 | 2025-12-17T09:08:30.115Z | /^external_dns_ips/c\ |
| 262 | 2025-12-17T09:08:30.115Z | external_dns_ips = [ "10.151.1.164", "10.151.1.165" ] |
| 263 | 2025-12-17T09:08:30.115Z | /^\[\[internal_services_ip_pool_ranges/,/^$/ { |
| 264 | 2025-12-17T09:08:30.115Z | /^first/c\ |
| 265 | 2025-12-17T09:08:30.115Z | first = "10.151.1.164" |
| 266 | 2025-12-17T09:08:30.115Z | /^last/c\ |
| 267 | 2025-12-17T09:08:30.115Z | last = "10.151.1.173" |
| 268 | 2025-12-17T09:08:30.115Z | } |
| 269 | 2025-12-17T09:08:30.115Z | /^infra_ip_first/c\ |
| 270 | 2025-12-17T09:08:30.115Z | infra_ip_first = "10.151.1.174" |
| 271 | 2025-12-17T09:08:30.115Z | /^infra_ip_last/c\ |
| 272 | 2025-12-17T09:08:30.115Z | infra_ip_last = "10.151.1.174" |
| 273 | 2025-12-17T09:08:30.115Z | /^\[\[rack_network_config.ports/,/^$/ { |
| 274 | 2025-12-17T09:08:30.115Z | /^routes/c\ |
| 275 | 2025-12-17T09:08:30.115Z | routes = \[{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}\] |
| 276 | 2025-12-17T09:08:30.115Z | /^addresses/c\ |
| 277 | 2025-12-17T09:08:30.115Z | addresses = \[{address = "10.151.1.174/24"} \] |
| 278 | 2025-12-17T09:08:30.116Z | } |
| 279 | 2025-12-17T09:08:30.116Z | ' pkg/config-rss.toml |
| 280 | 2025-12-17T09:08:30.116Z | + diff -u pkg/config-rss.toml~ pkg/config-rss.toml |
| 281 | 2025-12-17T09:08:30.116Z | --- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006 |
| 282 | 2025-12-17T09:08:30.116Z | +++ pkg/config-rss.toml Wed Dec 17 09:08:30 2025 |
| 283 | 2025-12-17T09:08:30.116Z | @@ -33,7 +33,7 @@ |
| 284 | 2025-12-17T09:08:30.116Z | # the DNS domain delegated to the rack by the customer. Each of these addresses |
| 285 | 2025-12-17T09:08:30.116Z | # must be contained in one of the "internal services" IP Pool ranges listed |
| 286 | 2025-12-17T09:08:30.116Z | # below. |
| 287 | 2025-12-17T09:08:30.116Z | -external_dns_ips = [ "192.168.1.20", "192.168.1.21" ] |
| 288 | 2025-12-17T09:08:30.116Z | +external_dns_ips = [ "10.151.1.164", "10.151.1.165" ] |
| 289 | 2025-12-17T09:08:30.116Z | |
| 290 | 2025-12-17T09:08:30.116Z | # Initial TLS certificates for the external API |
| 291 | 2025-12-17T09:08:30.116Z | # |
| 292 | 2025-12-17T09:08:30.116Z | @@ -69,8 +69,8 @@ |
| 293 | 2025-12-17T09:08:30.116Z | # |
| 294 | 2025-12-17T09:08:30.116Z | # For more on this and what to put here, see docs/how-to-run.adoc. |
| 295 | 2025-12-17T09:08:30.116Z | [[internal_services_ip_pool_ranges]] |
| 296 | 2025-12-17T09:08:30.116Z | -first = "192.168.1.20" |
| 297 | 2025-12-17T09:08:30.116Z | -last = "192.168.1.29" |
| 298 | 2025-12-17T09:08:30.116Z | +first = "10.151.1.164" |
| 299 | 2025-12-17T09:08:30.116Z | +last = "10.151.1.173" |
| 300 | 2025-12-17T09:08:30.116Z | |
| 301 | 2025-12-17T09:08:30.116Z | # TODO - this configuration is subject to change going forward. Ultimately these |
| 302 | 2025-12-17T09:08:30.116Z | # parameters should be provided to the control plane via wicket, but we need to |
| 303 | 2025-12-17T09:08:30.116Z | @@ -91,8 +91,8 @@ |
| 304 | 2025-12-17T09:08:30.116Z | # A range of IP addresses used by Boundary Services on the external network. In |
| 305 | 2025-12-17T09:08:30.116Z | # a real system, these would be addresses of the uplink ports on the Sidecar. |
| 306 | 2025-12-17T09:08:30.116Z | # With softnpu, only one address is used. |
| 307 | 2025-12-17T09:08:30.116Z | -infra_ip_first = "192.168.1.30" |
| 308 | 2025-12-17T09:08:30.116Z | -infra_ip_last = "192.168.1.30" |
| 309 | 2025-12-17T09:08:30.116Z | +infra_ip_first = "10.151.1.174" |
| 310 | 2025-12-17T09:08:30.116Z | +infra_ip_last = "10.151.1.174" |
| 311 | 2025-12-17T09:08:30.117Z | |
| 312 | 2025-12-17T09:08:30.117Z | # Configurations for BGP routers to run on the scrimlets. |
| 313 | 2025-12-17T09:08:30.117Z | bgp = [] |
| 314 | 2025-12-17T09:08:30.117Z | @@ -100,9 +100,9 @@ |
| 315 | 2025-12-17T09:08:30.117Z | # You can configure multiple uplinks by repeating the following stanza |
| 316 | 2025-12-17T09:08:30.117Z | [[rack_network_config.ports]] |
| 317 | 2025-12-17T09:08:30.117Z | # Routes associated with this port. |
| 318 | 2025-12-17T09:08:30.141Z | -routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}] |
| 319 | 2025-12-17T09:08:30.141Z | +routes = [{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}] |
| 320 | 2025-12-17T09:08:30.141Z | # Addresses associated with this port. |
| 321 | 2025-12-17T09:08:30.141Z | -addresses = [{address = "192.168.1.30/24"}] |
| 322 | 2025-12-17T09:08:30.141Z | +addresses = [{address = "10.151.1.174/24"} ] |
| 323 | 2025-12-17T09:08:30.141Z | # Name of the uplink port. This should always be "qsfp0" when using softnpu. |
| 324 | 2025-12-17T09:08:30.141Z | port = "qsfp0" |
| 325 | 2025-12-17T09:08:30.141Z | # The speed of this port. |
| 326 | 2025-12-17T09:08:30.141Z | + true |
| 327 | 2025-12-17T09:08:30.141Z | + tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml pkg/config.toml |
| 328 | 2025-12-17T09:08:30.141Z | a pkg/config-rss.toml 8K |
| 329 | 2025-12-17T09:08:30.141Z | a pkg/config.toml 6K |
| 330 | 2025-12-17T09:08:30.141Z | + rm -f pkg/config-rss.toml pkg/config-rss.toml~ pkg/config.toml pkg/config.toml~ |
| 331 | 2025-12-17T09:08:30.141Z | + TLS_NAME=recovery.sys.oxide.test |
| 332 | 2025-12-17T09:08:30.141Z | + openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin |
| 333 | 2025-12-17T09:08:30.594Z | .......+...+...........+.+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+............+.+.....+.......+........+......+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+...+...+.....+......+.........+.+...+.................+......+.......+.....+.+.................+...+................+.....+..........+......+.....+...+.........+.......+....................+...+...+.........+...+.......+........+......+..........+.................+.......+...............+...+..............+.+.....+....+...+.....+.......+...........+...+....+...+.........+.....................+..+............+......+.........+...+...+.........+.........+....+......+..+....+.....+...............+...+...............................+..........................+.........+....+.....+.+.....+.+...+...+.........+.........+.................+..................+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 334 | 2025-12-17T09:08:31.257Z | ..+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*........+.+......+......+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+...+.....+.........+................+.........+.....+...................+.........+..+.+........+.+.....+....+..+..........+............+..+.........+.......+........+.+......+.........+....................+.+.....+..........+............+..+.+...+..+....+.....+....+......+..............................+........+.+..+....+.....+...+......+............+.........+....+...........+.....................+...+....+.....+.+.....+.........+...+....+........................+.....+....+.....+........................................+...........+.........+......+......+.........+....+...+........+.........+.......+........+...+...+.+......+..............+.+...+......+........+...+....+.....+...+...+.......+...........+.......+...+..+.......+........+...+.......+........+....+...+...+...............+.....+....+...........+...................+........+..................+.+......+.........+...+...+.........+.........+......+...+..+..........+...........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 335 | 2025-12-17T09:08:31.281Z | ----- |
| 336 | 2025-12-17T09:08:31.305Z | + tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem |
| 337 | 2025-12-17T09:08:31.306Z | a pkg/initial-tls-cert.pem 2K |
| 338 | 2025-12-17T09:08:31.306Z | a pkg/initial-tls-key.pem 4K |
| 339 | 2025-12-17T09:08:31.306Z | + rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem |
| 340 | 2025-12-17T09:08:31.306Z | + rmdir pkg |
| 341 | 2025-12-17T09:08:31.306Z | + E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem |
| 342 | 2025-12-17T09:08:31.306Z | + pfexec mkdir -p /usr/oxide |
| 343 | 2025-12-17T09:08:31.306Z | + pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v2025112000 |
| 344 | 2025-12-17T09:08:31.926Z | + pfexec chmod +x /usr/oxide/oxide |
| 345 | 2025-12-17T09:08:31.951Z | + curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw |
| 346 | 2025-12-17T09:08:50.553Z | + OMICRON_NO_UNINSTALL=1 |
| 347 | 2025-12-17T09:08:50.553Z | + ptime -m pfexec ./target/release/omicron-package -t test install |
| 348 | 2025-12-17T09:08:50.579Z | Logging to: /opt/oxide/work/out/LOG |
| 349 | 2025-12-17T09:08:51.397Z | |
| 350 | 2025-12-17T09:08:51.421Z | real 0.843213049 |
| 351 | 2025-12-17T09:08:51.421Z | user 0.178038341 |
| 352 | 2025-12-17T09:08:51.421Z | sys 4.800828963 |
| 353 | 2025-12-17T09:08:51.421Z | trap 0.000014979 |
| 354 | 2025-12-17T09:08:51.421Z | tflt 0.000044449 |
| 355 | 2025-12-17T09:08:51.421Z | dflt 0.000182126 |
| 356 | 2025-12-17T09:08:51.421Z | kflt 0.000000000 |
| 357 | 2025-12-17T09:08:51.421Z | lock 1:16.095742832 |
| 358 | 2025-12-17T09:08:51.421Z | slp 1.051684198 |
| 359 | 2025-12-17T09:08:51.421Z | lat 0.039212104 |
| 360 | 2025-12-17T09:08:51.421Z | stop 0.085745855 |
| 361 | 2025-12-17T09:08:51.421Z | + retry=0 |
| 362 | 2025-12-17T09:08:51.421Z | + curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/' |
| 363 | 2025-12-17T09:08:51.421Z | + [[ 0 -gt 30 ]] |
| 364 | 2025-12-17T09:08:51.421Z | + sleep 1 |
| 365 | 2025-12-17T09:08:52.412Z | + retry=1 |
| 366 | 2025-12-17T09:08:52.412Z | + curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/' |
| 367 | 2025-12-17T09:08:52.436Z | + [[ 1 -gt 30 ]] |
| 368 | 2025-12-17T09:08:52.436Z | + sleep 1 |
| 369 | 2025-12-17T09:08:53.426Z | + retry=2 |
| 370 | 2025-12-17T09:08:53.426Z | + curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/' |
| 371 | 2025-12-17T09:08:53.451Z | + [[ 2 -gt 30 ]] |
| 372 | 2025-12-17T09:08:53.451Z | + sleep 1 |
| 373 | 2025-12-17T09:08:54.456Z | + retry=3 |
| 374 | 2025-12-17T09:08:54.456Z | + curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/' |
| 375 | 2025-12-17T09:09:14.304Z | + [[ 3 -gt 30 ]] |
| 376 | 2025-12-17T09:09:14.304Z | + sleep 1 |
| 377 | 2025-12-17T09:09:15.307Z | + retry=4 |
| 378 | 2025-12-17T09:09:15.307Z | + curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/' |
| 379 | 2025-12-17T09:09:15.331Z | + pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state |
| 380 | 2025-12-17T09:09:15.355Z | local v6: |
| 381 | 2025-12-17T09:09:15.355Z | fd61:a127:3fee:5ab1::1 |
| 382 | 2025-12-17T09:09:15.355Z | fe80::889a:c8ff:fe1d:175e |
| 383 | 2025-12-17T09:09:15.355Z | local v4: |
| 384 | 2025-12-17T09:09:15.355Z | 10.151.1.174 |
| 385 | 2025-12-17T09:09:15.356Z | router v6_idx: |
| 386 | 2025-12-17T09:09:15.356Z | router v6_routes: |
| 387 | 2025-12-17T09:09:15.356Z | router v4_idx: |
| 388 | 2025-12-17T09:09:15.356Z | router v4_routes: |
| 389 | 2025-12-17T09:09:15.356Z | resolver v4: |
| 390 | 2025-12-17T09:09:15.356Z | 10.151.1.100 -> 18:c0:4d:0d:9f:b2 |
| 391 | 2025-12-17T09:09:15.356Z | resolver v6: |
| 392 | 2025-12-17T09:09:15.356Z | fe80::889a:c8ff:fe1d:175e -> 8a:9a:c8:1d:17:5e |
| 393 | 2025-12-17T09:09:15.356Z | fe80::589a:89ff:fee7:53c5 -> 5a:9a:89:e7:53:c5 |
| 394 | 2025-12-17T09:09:15.356Z | nat v4: |
| 395 | 2025-12-17T09:09:15.356Z | nat_v6: |
| 396 | 2025-12-17T09:09:15.356Z | port_mac: |
| 397 | 2025-12-17T09:09:15.356Z | 1: a8:40:25:5f:c1:2d |
| 398 | 2025-12-17T09:09:15.356Z | 1000: a8:40:25:5f:c1:2c |
| 399 | 2025-12-17T09:09:15.356Z | 2: a8:40:25:5f:c1:4d |
| 400 | 2025-12-17T09:09:15.356Z | proxy arp: |
| 401 | 2025-12-17T09:09:15.356Z | 10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d |
| 402 | 2025-12-17T09:09:15.356Z | + retry=0 |
| 403 | 2025-12-17T09:09:15.380Z | +++ zoneadm list -n |
| 404 | 2025-12-17T09:09:15.380Z | +++ grep oxz_ntp |
| 405 | 2025-12-17T09:09:15.380Z | ++ pfexec svcs -z -Hostate oxide/ntp |
| 406 | 2025-12-17T09:09:15.380Z | svcs: zone '-Hostate': entity not found |
| 407 | 2025-12-17T09:09:15.380Z | ++ true |
| 408 | 2025-12-17T09:09:15.380Z | + [[ '' != online ]] |
| 409 | 2025-12-17T09:09:15.380Z | + [[ 0 -gt 60 ]] |
| 410 | 2025-12-17T09:09:15.380Z | + sleep 1 |
| 411 | 2025-12-17T09:09:16.376Z | + retry=1 |
| 412 | 2025-12-17T09:09:16.401Z | +++ zoneadm list -n |
| 413 | 2025-12-17T09:09:16.401Z | +++ grep oxz_ntp |
| 414 | 2025-12-17T09:09:16.401Z | ++ pfexec svcs -z -Hostate oxide/ntp |
| 415 | 2025-12-17T09:09:16.401Z | svcs: zone '-Hostate': entity not found |
| 416 | 2025-12-17T09:09:16.401Z | ++ true |
| 417 | 2025-12-17T09:09:16.401Z | + [[ '' != online ]] |
| 418 | 2025-12-17T09:09:16.401Z | + [[ 1 -gt 60 ]] |
| 419 | 2025-12-17T09:09:16.401Z | + sleep 1 |
| 420 | 2025-12-17T09:09:17.398Z | + retry=2 |
| 421 | 2025-12-17T09:09:17.422Z | +++ zoneadm list -n |
| 422 | 2025-12-17T09:09:17.422Z | +++ grep oxz_ntp |
| 423 | 2025-12-17T09:09:17.422Z | ++ pfexec svcs -z -Hostate oxide/ntp |
| 424 | 2025-12-17T09:09:17.446Z | svcs: zone '-Hostate': entity not found |
| 425 | 2025-12-17T09:09:17.446Z | ++ true |
| 426 | 2025-12-17T09:09:17.446Z | + [[ '' != online ]] |
| 427 | 2025-12-17T09:09:17.446Z | + [[ 2 -gt 60 ]] |
| 428 | 2025-12-17T09:09:17.446Z | + sleep 1 |
| 429 | 2025-12-17T09:09:18.428Z | + retry=3 |
| 430 | 2025-12-17T09:09:18.453Z | +++ zoneadm list -n |
| 431 | 2025-12-17T09:09:18.453Z | +++ grep oxz_ntp |
| 432 | 2025-12-17T09:09:18.453Z | ++ pfexec svcs -z oxz_ntp_4696b333-67e2-4ed1-b7d3-d5ddcaa23769 -Hostate oxide/ntp |
| 433 | 2025-12-17T09:09:18.477Z | svcs: Pattern 'oxide/ntp' doesn't match any instances |
| 434 | 2025-12-17T09:09:18.477Z | ++ true |
| 435 | 2025-12-17T09:09:18.477Z | + [[ '' != online ]] |
| 436 | 2025-12-17T09:09:18.478Z | + [[ 3 -gt 60 ]] |
| 437 | 2025-12-17T09:09:18.478Z | + sleep 1 |
| 438 | 2025-12-17T09:09:19.461Z | + retry=4 |
| 439 | 2025-12-17T09:09:19.485Z | +++ zoneadm list -n |
| 440 | 2025-12-17T09:09:19.485Z | +++ grep oxz_ntp |
| 441 | 2025-12-17T09:09:19.485Z | ++ pfexec svcs -z oxz_ntp_4696b333-67e2-4ed1-b7d3-d5ddcaa23769 -Hostate oxide/ntp |
| 442 | 2025-12-17T09:09:19.509Z | svcs: Pattern 'oxide/ntp' doesn't match any instances |
| 443 | 2025-12-17T09:09:19.509Z | ++ true |
| 444 | 2025-12-17T09:09:19.509Z | + [[ '' != online ]] |
| 445 | 2025-12-17T09:09:19.509Z | + [[ 4 -gt 60 ]] |
| 446 | 2025-12-17T09:09:19.509Z | + sleep 1 |
| 447 | 2025-12-17T09:09:20.493Z | + retry=5 |
| 448 | 2025-12-17T09:09:20.517Z | +++ zoneadm list -n |
| 449 | 2025-12-17T09:09:20.518Z | +++ grep oxz_ntp |
| 450 | 2025-12-17T09:09:20.518Z | ++ pfexec svcs -z oxz_ntp_4696b333-67e2-4ed1-b7d3-d5ddcaa23769 -Hostate oxide/ntp |
| 451 | 2025-12-17T09:09:20.542Z | svcs: Pattern 'oxide/ntp' doesn't match any instances |
| 452 | 2025-12-17T09:09:20.542Z | ++ true |
| 453 | 2025-12-17T09:09:20.542Z | + [[ '' != online ]] |
| 454 | 2025-12-17T09:09:20.542Z | + [[ 5 -gt 60 ]] |
| 455 | 2025-12-17T09:09:20.542Z | + sleep 1 |
| 456 | 2025-12-17T09:09:21.523Z | + retry=6 |
| 457 | 2025-12-17T09:09:21.547Z | +++ zoneadm list -n |
| 458 | 2025-12-17T09:09:21.547Z | +++ grep oxz_ntp |
| 459 | 2025-12-17T09:09:21.547Z | ++ pfexec svcs -z oxz_ntp_4696b333-67e2-4ed1-b7d3-d5ddcaa23769 -Hostate oxide/ntp |
| 460 | 2025-12-17T09:09:21.571Z | svcs: Pattern 'oxide/ntp' doesn't match any instances |
| 461 | 2025-12-17T09:09:21.571Z | ++ true |
| 462 | 2025-12-17T09:09:21.571Z | + [[ '' != online ]] |
| 463 | 2025-12-17T09:09:21.571Z | + [[ 6 -gt 60 ]] |
| 464 | 2025-12-17T09:09:21.571Z | + sleep 1 |
| 465 | 2025-12-17T09:09:22.558Z | + retry=7 |
| 466 | 2025-12-17T09:09:22.583Z | +++ zoneadm list -n |
| 467 | 2025-12-17T09:09:22.583Z | +++ grep oxz_ntp |
| 468 | 2025-12-17T09:09:22.583Z | ++ pfexec svcs -z oxz_ntp_4696b333-67e2-4ed1-b7d3-d5ddcaa23769 -Hostate oxide/ntp |
| 469 | 2025-12-17T09:09:22.583Z | + [[ offline != online ]] |
| 470 | 2025-12-17T09:09:22.583Z | + [[ 7 -gt 60 ]] |
| 471 | 2025-12-17T09:09:22.583Z | + sleep 1 |
| 472 | 2025-12-17T09:09:23.581Z | + retry=8 |
| 473 | 2025-12-17T09:09:23.630Z | +++ zoneadm list -n |
| 474 | 2025-12-17T09:09:23.630Z | +++ grep oxz_ntp |
| 475 | 2025-12-17T09:09:23.630Z | ++ pfexec svcs -z oxz_ntp_4696b333-67e2-4ed1-b7d3-d5ddcaa23769 -Hostate oxide/ntp |
| 476 | 2025-12-17T09:09:23.630Z | + [[ offline != online ]] |
| 477 | 2025-12-17T09:09:23.630Z | + [[ 8 -gt 60 ]] |
| 478 | 2025-12-17T09:09:23.630Z | + sleep 1 |
| 479 | 2025-12-17T09:09:24.603Z | + retry=9 |
| 480 | 2025-12-17T09:09:24.627Z | +++ zoneadm list -n |
| 481 | 2025-12-17T09:09:24.627Z | +++ grep oxz_ntp |
| 482 | 2025-12-17T09:09:24.627Z | ++ pfexec svcs -z oxz_ntp_4696b333-67e2-4ed1-b7d3-d5ddcaa23769 -Hostate oxide/ntp |
| 483 | 2025-12-17T09:09:24.627Z | + [[ offline != online ]] |
| 484 | 2025-12-17T09:09:24.627Z | + [[ 9 -gt 60 ]] |
| 485 | 2025-12-17T09:09:24.627Z | + sleep 1 |
| 486 | 2025-12-17T09:09:25.626Z | + retry=10 |
| 487 | 2025-12-17T09:09:25.650Z | +++ zoneadm list -n |
| 488 | 2025-12-17T09:09:25.650Z | +++ grep oxz_ntp |
| 489 | 2025-12-17T09:09:25.650Z | ++ pfexec svcs -z oxz_ntp_4696b333-67e2-4ed1-b7d3-d5ddcaa23769 -Hostate oxide/ntp |
| 490 | 2025-12-17T09:09:25.650Z | + [[ offline != online ]] |
| 491 | 2025-12-17T09:09:25.650Z | + [[ 10 -gt 60 ]] |
| 492 | 2025-12-17T09:09:25.650Z | + sleep 1 |
| 493 | 2025-12-17T09:09:26.647Z | + retry=11 |
| 494 | 2025-12-17T09:09:26.671Z | +++ zoneadm list -n |
| 495 | 2025-12-17T09:09:26.671Z | +++ grep oxz_ntp |
| 496 | 2025-12-17T09:09:26.671Z | ++ pfexec svcs -z oxz_ntp_4696b333-67e2-4ed1-b7d3-d5ddcaa23769 -Hostate oxide/ntp |
| 497 | 2025-12-17T09:09:26.671Z | + [[ offline != online ]] |
| 498 | 2025-12-17T09:09:26.671Z | + [[ 11 -gt 60 ]] |
| 499 | 2025-12-17T09:09:26.671Z | + sleep 1 |
| 500 | 2025-12-17T09:09:27.669Z | + retry=12 |
| 501 | 2025-12-17T09:09:27.692Z | +++ zoneadm list -n |
| 502 | 2025-12-17T09:09:27.692Z | +++ grep oxz_ntp |
| 503 | 2025-12-17T09:09:27.692Z | ++ pfexec svcs -z oxz_ntp_4696b333-67e2-4ed1-b7d3-d5ddcaa23769 -Hostate oxide/ntp |
| 504 | 2025-12-17T09:09:27.692Z | + [[ offline != online ]] |
| 505 | 2025-12-17T09:09:27.692Z | + [[ 12 -gt 60 ]] |
| 506 | 2025-12-17T09:09:27.692Z | + sleep 1 |
| |
| 507 | 2025-12-17T09:09:28.652Z | buskin console login: Dec 17 09:09:28 buskin ip: ip: joining multicasts failed (4) on opte0 - will use link layer broadcasts for multicast |
| |
| 508 | 2025-12-17T09:09:28.690Z | + retry=13 |
| 509 | 2025-12-17T09:09:28.714Z | +++ zoneadm list -n |
| 510 | 2025-12-17T09:09:28.715Z | +++ grep oxz_ntp |
| 511 | 2025-12-17T09:09:28.715Z | ++ pfexec svcs -z oxz_ntp_4696b333-67e2-4ed1-b7d3-d5ddcaa23769 -Hostate oxide/ntp |
| 512 | 2025-12-17T09:09:28.715Z | + [[ offline != online ]] |
| 513 | 2025-12-17T09:09:28.715Z | + [[ 13 -gt 60 ]] |
| 514 | 2025-12-17T09:09:28.715Z | + sleep 1 |
| 515 | 2025-12-17T09:09:29.713Z | + retry=14 |
| 516 | 2025-12-17T09:09:29.737Z | +++ zoneadm list -n |
| 517 | 2025-12-17T09:09:29.737Z | +++ grep oxz_ntp |
| 518 | 2025-12-17T09:09:29.737Z | ++ pfexec svcs -z oxz_ntp_4696b333-67e2-4ed1-b7d3-d5ddcaa23769 -Hostate oxide/ntp |
| 519 | 2025-12-17T09:09:29.737Z | + [[ online != online ]] |
| 520 | 2025-12-17T09:09:29.737Z | + echo 'Waited for chrony: 14s' |
| 521 | 2025-12-17T09:09:29.737Z | + retry=0 |
| 522 | 2025-12-17T09:09:29.737Z | Waited for chrony: 14s |
| 523 | 2025-12-17T09:09:29.737Z | + zoneadm list |
| 524 | 2025-12-17T09:09:29.737Z | + grep nexus |
| 525 | 2025-12-17T09:09:29.761Z | + [[ 0 -gt 300 ]] |
| 526 | 2025-12-17T09:09:29.762Z | + sleep 1 |
| 527 | 2025-12-17T09:09:30.742Z | + retry=1 |
| 528 | 2025-12-17T09:09:30.766Z | + zoneadm list |
| 529 | 2025-12-17T09:09:30.766Z | + grep nexus |
| 530 | 2025-12-17T09:09:30.766Z | + [[ 1 -gt 300 ]] |
| 531 | 2025-12-17T09:09:30.766Z | + sleep 1 |
| 532 | 2025-12-17T09:09:31.753Z | + retry=2 |
| 533 | 2025-12-17T09:09:31.778Z | + zoneadm list |
| 534 | 2025-12-17T09:09:31.778Z | + grep nexus |
| 535 | 2025-12-17T09:09:31.778Z | + [[ 2 -gt 300 ]] |
| 536 | 2025-12-17T09:09:31.778Z | + sleep 1 |
| 537 | 2025-12-17T09:09:32.764Z | + retry=3 |
| 538 | 2025-12-17T09:09:32.788Z | + zoneadm list |
| 539 | 2025-12-17T09:09:32.788Z | + grep nexus |
| 540 | 2025-12-17T09:09:32.788Z | + [[ 3 -gt 300 ]] |
| 541 | 2025-12-17T09:09:32.788Z | + sleep 1 |
| 542 | 2025-12-17T09:09:33.776Z | + retry=4 |
| 543 | 2025-12-17T09:09:33.801Z | + zoneadm list |
| 544 | 2025-12-17T09:09:33.801Z | + grep nexus |
| 545 | 2025-12-17T09:09:33.801Z | + [[ 4 -gt 300 ]] |
| 546 | 2025-12-17T09:09:33.801Z | + sleep 1 |
| 547 | 2025-12-17T09:09:34.785Z | + retry=5 |
| 548 | 2025-12-17T09:09:34.809Z | + zoneadm list |
| 549 | 2025-12-17T09:09:34.809Z | + grep nexus |
| 550 | 2025-12-17T09:09:34.809Z | + [[ 5 -gt 300 ]] |
| 551 | 2025-12-17T09:09:34.809Z | + sleep 1 |
| 552 | 2025-12-17T09:09:35.795Z | + retry=6 |
| 553 | 2025-12-17T09:09:35.819Z | + zoneadm list |
| 554 | 2025-12-17T09:09:35.819Z | + grep nexus |
| 555 | 2025-12-17T09:09:35.819Z | + [[ 6 -gt 300 ]] |
| 556 | 2025-12-17T09:09:35.819Z | + sleep 1 |
| 557 | 2025-12-17T09:09:36.805Z | + retry=7 |
| 558 | 2025-12-17T09:09:36.829Z | + zoneadm list |
| 559 | 2025-12-17T09:09:36.829Z | + grep nexus |
| 560 | 2025-12-17T09:09:36.829Z | + [[ 7 -gt 300 ]] |
| 561 | 2025-12-17T09:09:36.829Z | + sleep 1 |
| 562 | 2025-12-17T09:09:37.815Z | + retry=8 |
| 563 | 2025-12-17T09:09:37.839Z | + zoneadm list |
| 564 | 2025-12-17T09:09:37.839Z | + grep nexus |
| 565 | 2025-12-17T09:09:37.839Z | + [[ 8 -gt 300 ]] |
| 566 | 2025-12-17T09:09:37.839Z | + sleep 1 |
| 567 | 2025-12-17T09:09:38.825Z | + retry=9 |
| 568 | 2025-12-17T09:09:38.849Z | + zoneadm list |
| 569 | 2025-12-17T09:09:38.849Z | + grep nexus |
| 570 | 2025-12-17T09:09:38.849Z | + [[ 9 -gt 300 ]] |
| 571 | 2025-12-17T09:09:38.849Z | + sleep 1 |
| 572 | 2025-12-17T09:09:39.836Z | + retry=10 |
| 573 | 2025-12-17T09:09:39.860Z | + zoneadm list |
| 574 | 2025-12-17T09:09:39.860Z | + grep nexus |
| 575 | 2025-12-17T09:09:39.860Z | + [[ 10 -gt 300 ]] |
| 576 | 2025-12-17T09:09:39.860Z | + sleep 1 |
| 577 | 2025-12-17T09:09:40.846Z | + retry=11 |
| 578 | 2025-12-17T09:09:40.870Z | + zoneadm list |
| 579 | 2025-12-17T09:09:40.870Z | + grep nexus |
| 580 | 2025-12-17T09:09:40.870Z | + [[ 11 -gt 300 ]] |
| 581 | 2025-12-17T09:09:40.870Z | + sleep 1 |
| 582 | 2025-12-17T09:09:41.860Z | + retry=12 |
| 583 | 2025-12-17T09:09:41.885Z | + zoneadm list |
| 584 | 2025-12-17T09:09:41.885Z | + grep nexus |
| 585 | 2025-12-17T09:09:41.885Z | + [[ 12 -gt 300 ]] |
| 586 | 2025-12-17T09:09:41.885Z | + sleep 1 |
| 587 | 2025-12-17T09:09:42.875Z | + retry=13 |
| 588 | 2025-12-17T09:09:42.900Z | + zoneadm list |
| 589 | 2025-12-17T09:09:42.900Z | + grep nexus |
| 590 | 2025-12-17T09:09:42.900Z | + [[ 13 -gt 300 ]] |
| 591 | 2025-12-17T09:09:42.900Z | + sleep 1 |
| 592 | 2025-12-17T09:09:43.893Z | + retry=14 |
| 593 | 2025-12-17T09:09:43.917Z | + zoneadm list |
| 594 | 2025-12-17T09:09:43.917Z | + grep nexus |
| 595 | 2025-12-17T09:09:43.917Z | + [[ 14 -gt 300 ]] |
| 596 | 2025-12-17T09:09:43.917Z | + sleep 1 |
| 597 | 2025-12-17T09:09:44.943Z | + retry=15 |
| 598 | 2025-12-17T09:09:45.034Z | + grep nexus |
| 599 | 2025-12-17T09:09:45.068Z | + zoneadm list |
| 600 | 2025-12-17T09:09:45.224Z | + [[ 15 -gt 300 ]] |
| 601 | 2025-12-17T09:09:45.224Z | + sleep 1 |
| 602 | 2025-12-17T09:09:46.256Z | + retry=16 |
| 603 | 2025-12-17T09:09:46.280Z | + zoneadm list |
| 604 | 2025-12-17T09:09:46.280Z | + grep nexus |
| 605 | 2025-12-17T09:09:46.280Z | + [[ 16 -gt 300 ]] |
| 606 | 2025-12-17T09:09:46.280Z | + sleep 1 |
| 607 | 2025-12-17T09:09:47.273Z | + retry=17 |
| 608 | 2025-12-17T09:09:47.297Z | + zoneadm list |
| 609 | 2025-12-17T09:09:47.297Z | + grep nexus |
| 610 | 2025-12-17T09:09:47.297Z | + [[ 17 -gt 300 ]] |
| 611 | 2025-12-17T09:09:47.297Z | + sleep 1 |
| 612 | 2025-12-17T09:09:48.288Z | + retry=18 |
| 613 | 2025-12-17T09:09:48.312Z | + zoneadm list |
| 614 | 2025-12-17T09:09:48.312Z | + grep nexus |
| 615 | 2025-12-17T09:09:48.312Z | + [[ 18 -gt 300 ]] |
| 616 | 2025-12-17T09:09:48.312Z | + sleep 1 |
| 617 | 2025-12-17T09:09:49.304Z | + retry=19 |
| 618 | 2025-12-17T09:09:49.328Z | + zoneadm list |
| 619 | 2025-12-17T09:09:49.328Z | + grep nexus |
| 620 | 2025-12-17T09:09:49.328Z | + [[ 19 -gt 300 ]] |
| 621 | 2025-12-17T09:09:49.328Z | + sleep 1 |
| 622 | 2025-12-17T09:09:50.315Z | + retry=20 |
| 623 | 2025-12-17T09:09:50.341Z | + zoneadm list |
| 624 | 2025-12-17T09:09:50.341Z | + grep nexus |
| 625 | 2025-12-17T09:09:50.341Z | + [[ 20 -gt 300 ]] |
| 626 | 2025-12-17T09:09:50.341Z | + sleep 1 |
| 627 | 2025-12-17T09:09:51.327Z | + retry=21 |
| 628 | 2025-12-17T09:09:51.351Z | + zoneadm list |
| 629 | 2025-12-17T09:09:51.351Z | + grep nexus |
| 630 | 2025-12-17T09:09:51.351Z | + [[ 21 -gt 300 ]] |
| 631 | 2025-12-17T09:09:51.351Z | + sleep 1 |
| 632 | 2025-12-17T09:09:52.340Z | + retry=22 |
| 633 | 2025-12-17T09:09:52.364Z | + zoneadm list |
| 634 | 2025-12-17T09:09:52.364Z | + grep nexus |
| 635 | 2025-12-17T09:09:52.365Z | + [[ 22 -gt 300 ]] |
| 636 | 2025-12-17T09:09:52.365Z | + sleep 1 |
| 637 | 2025-12-17T09:09:53.353Z | + retry=23 |
| 638 | 2025-12-17T09:09:53.377Z | + zoneadm list |
| 639 | 2025-12-17T09:09:53.377Z | + grep nexus |
| 640 | 2025-12-17T09:09:53.377Z | + [[ 23 -gt 300 ]] |
| 641 | 2025-12-17T09:09:53.377Z | + sleep 1 |
| 642 | 2025-12-17T09:09:54.365Z | + retry=24 |
| 643 | 2025-12-17T09:09:54.392Z | + zoneadm list |
| 644 | 2025-12-17T09:09:54.392Z | + grep nexus |
| 645 | 2025-12-17T09:09:54.392Z | + [[ 24 -gt 300 ]] |
| 646 | 2025-12-17T09:09:54.392Z | + sleep 1 |
| 647 | 2025-12-17T09:09:55.385Z | + retry=25 |
| 648 | 2025-12-17T09:09:55.409Z | + zoneadm list |
| 649 | 2025-12-17T09:09:55.410Z | + grep nexus |
| 650 | 2025-12-17T09:09:55.410Z | + [[ 25 -gt 300 ]] |
| 651 | 2025-12-17T09:09:55.410Z | + sleep 1 |
| 652 | 2025-12-17T09:09:56.397Z | + retry=26 |
| 653 | 2025-12-17T09:09:56.421Z | + zoneadm list |
| 654 | 2025-12-17T09:09:56.421Z | + grep nexus |
| 655 | 2025-12-17T09:09:56.421Z | + [[ 26 -gt 300 ]] |
| 656 | 2025-12-17T09:09:56.421Z | + sleep 1 |
| 657 | 2025-12-17T09:09:57.409Z | + retry=27 |
| 658 | 2025-12-17T09:09:57.433Z | + zoneadm list |
| 659 | 2025-12-17T09:09:57.433Z | + grep nexus |
| 660 | 2025-12-17T09:09:57.433Z | + [[ 27 -gt 300 ]] |
| 661 | 2025-12-17T09:09:57.433Z | + sleep 1 |
| 662 | 2025-12-17T09:09:58.420Z | + retry=28 |
| 663 | 2025-12-17T09:09:58.445Z | + zoneadm list |
| 664 | 2025-12-17T09:09:58.445Z | + grep nexus |
| 665 | 2025-12-17T09:09:58.445Z | + [[ 28 -gt 300 ]] |
| 666 | 2025-12-17T09:09:58.445Z | + sleep 1 |
| 667 | 2025-12-17T09:09:59.431Z | + retry=29 |
| 668 | 2025-12-17T09:09:59.455Z | + zoneadm list |
| 669 | 2025-12-17T09:09:59.455Z | + grep nexus |
| 670 | 2025-12-17T09:09:59.455Z | + [[ 29 -gt 300 ]] |
| 671 | 2025-12-17T09:09:59.455Z | + sleep 1 |
| 672 | 2025-12-17T09:10:00.442Z | + retry=30 |
| 673 | 2025-12-17T09:10:00.466Z | + zoneadm list |
| 674 | 2025-12-17T09:10:00.466Z | + grep nexus |
| 675 | 2025-12-17T09:10:00.466Z | + [[ 30 -gt 300 ]] |
| 676 | 2025-12-17T09:10:00.466Z | + sleep 1 |
| 677 | 2025-12-17T09:10:01.454Z | + retry=31 |
| 678 | 2025-12-17T09:10:01.478Z | + zoneadm list |
| 679 | 2025-12-17T09:10:01.478Z | + grep nexus |
| 680 | 2025-12-17T09:10:01.479Z | + [[ 31 -gt 300 ]] |
| 681 | 2025-12-17T09:10:01.479Z | + sleep 1 |
| 682 | 2025-12-17T09:10:02.486Z | + retry=32 |
| 683 | 2025-12-17T09:10:02.511Z | + zoneadm list |
| 684 | 2025-12-17T09:10:02.511Z | + grep nexus |
| 685 | 2025-12-17T09:10:02.511Z | + [[ 32 -gt 300 ]] |
| 686 | 2025-12-17T09:10:02.511Z | + sleep 1 |
| 687 | 2025-12-17T09:10:03.478Z | + retry=33 |
| 688 | 2025-12-17T09:10:03.502Z | + zoneadm list |
| 689 | 2025-12-17T09:10:03.502Z | + grep nexus |
| 690 | 2025-12-17T09:10:03.502Z | + [[ 33 -gt 300 ]] |
| 691 | 2025-12-17T09:10:03.502Z | + sleep 1 |
| 692 | 2025-12-17T09:10:04.492Z | + retry=34 |
| 693 | 2025-12-17T09:10:04.519Z | + zoneadm list |
| 694 | 2025-12-17T09:10:04.519Z | + grep nexus |
| 695 | 2025-12-17T09:10:04.520Z | + [[ 34 -gt 300 ]] |
| 696 | 2025-12-17T09:10:04.520Z | + sleep 1 |
| 697 | 2025-12-17T09:10:05.511Z | + retry=35 |
| 698 | 2025-12-17T09:10:05.535Z | + zoneadm list |
| 699 | 2025-12-17T09:10:05.535Z | + grep nexus |
| 700 | 2025-12-17T09:10:05.535Z | + [[ 35 -gt 300 ]] |
| 701 | 2025-12-17T09:10:05.535Z | + sleep 1 |
| 702 | 2025-12-17T09:10:06.529Z | + retry=36 |
| 703 | 2025-12-17T09:10:06.553Z | + zoneadm list |
| 704 | 2025-12-17T09:10:06.553Z | + grep nexus |
| 705 | 2025-12-17T09:10:06.553Z | + [[ 36 -gt 300 ]] |
| 706 | 2025-12-17T09:10:06.553Z | + sleep 1 |
| 707 | 2025-12-17T09:10:07.547Z | + retry=37 |
| 708 | 2025-12-17T09:10:07.571Z | + zoneadm list |
| 709 | 2025-12-17T09:10:07.572Z | + grep nexus |
| 710 | 2025-12-17T09:10:07.572Z | + [[ 37 -gt 300 ]] |
| 711 | 2025-12-17T09:10:07.572Z | + sleep 1 |
| 712 | 2025-12-17T09:10:08.565Z | + retry=38 |
| 713 | 2025-12-17T09:10:08.589Z | + zoneadm list |
| 714 | 2025-12-17T09:10:08.589Z | + grep nexus |
| 715 | 2025-12-17T09:10:08.589Z | + [[ 38 -gt 300 ]] |
| 716 | 2025-12-17T09:10:08.589Z | + sleep 1 |
| 717 | 2025-12-17T09:10:09.641Z | + retry=39 |
| 718 | 2025-12-17T09:10:09.665Z | + zoneadm list |
| 719 | 2025-12-17T09:10:09.665Z | + grep nexus |
| 720 | 2025-12-17T09:10:09.665Z | + [[ 39 -gt 300 ]] |
| 721 | 2025-12-17T09:10:09.665Z | + sleep 1 |
| 722 | 2025-12-17T09:10:10.594Z | + retry=40 |
| 723 | 2025-12-17T09:10:10.617Z | + zoneadm list |
| 724 | 2025-12-17T09:10:10.617Z | + grep nexus |
| 725 | 2025-12-17T09:10:10.617Z | + [[ 40 -gt 300 ]] |
| 726 | 2025-12-17T09:10:10.617Z | + sleep 1 |
| 727 | 2025-12-17T09:10:11.605Z | + retry=41 |
| 728 | 2025-12-17T09:10:11.629Z | + zoneadm list |
| 729 | 2025-12-17T09:10:11.629Z | + grep nexus |
| 730 | 2025-12-17T09:10:11.629Z | + [[ 41 -gt 300 ]] |
| 731 | 2025-12-17T09:10:11.629Z | + sleep 1 |
| 732 | 2025-12-17T09:10:12.617Z | + retry=42 |
| 733 | 2025-12-17T09:10:12.641Z | + zoneadm list |
| 734 | 2025-12-17T09:10:12.641Z | + grep nexus |
| 735 | 2025-12-17T09:10:12.641Z | + [[ 42 -gt 300 ]] |
| 736 | 2025-12-17T09:10:12.641Z | + sleep 1 |
| 737 | 2025-12-17T09:10:13.630Z | + retry=43 |
| 738 | 2025-12-17T09:10:13.654Z | + zoneadm list |
| 739 | 2025-12-17T09:10:13.654Z | + grep nexus |
| 740 | 2025-12-17T09:10:13.654Z | + [[ 43 -gt 300 ]] |
| 741 | 2025-12-17T09:10:13.654Z | + sleep 1 |
| 742 | 2025-12-17T09:10:14.641Z | + retry=44 |
| 743 | 2025-12-17T09:10:14.665Z | + zoneadm list |
| 744 | 2025-12-17T09:10:14.665Z | + grep nexus |
| 745 | 2025-12-17T09:10:14.665Z | + [[ 44 -gt 300 ]] |
| 746 | 2025-12-17T09:10:14.665Z | + sleep 1 |
| 747 | 2025-12-17T09:10:15.658Z | + retry=45 |
| 748 | 2025-12-17T09:10:15.681Z | + zoneadm list |
| 749 | 2025-12-17T09:10:15.681Z | + grep nexus |
| 750 | 2025-12-17T09:10:15.681Z | + [[ 45 -gt 300 ]] |
| 751 | 2025-12-17T09:10:15.681Z | + sleep 1 |
| 752 | 2025-12-17T09:10:16.665Z | + retry=46 |
| 753 | 2025-12-17T09:10:16.688Z | + zoneadm list |
| 754 | 2025-12-17T09:10:16.688Z | + grep nexus |
| 755 | 2025-12-17T09:10:16.688Z | + [[ 46 -gt 300 ]] |
| 756 | 2025-12-17T09:10:16.688Z | + sleep 1 |
| 757 | 2025-12-17T09:10:17.678Z | + retry=47 |
| 758 | 2025-12-17T09:10:17.702Z | + zoneadm list |
| 759 | 2025-12-17T09:10:17.702Z | + grep nexus |
| 760 | 2025-12-17T09:10:17.702Z | + [[ 47 -gt 300 ]] |
| 761 | 2025-12-17T09:10:17.702Z | + sleep 1 |
| 762 | 2025-12-17T09:10:18.691Z | + retry=48 |
| 763 | 2025-12-17T09:10:18.714Z | + zoneadm list |
| 764 | 2025-12-17T09:10:18.714Z | + grep nexus |
| 765 | 2025-12-17T09:10:18.714Z | + [[ 48 -gt 300 ]] |
| 766 | 2025-12-17T09:10:18.714Z | + sleep 1 |
| 767 | 2025-12-17T09:10:19.703Z | + retry=49 |
| 768 | 2025-12-17T09:10:19.726Z | + zoneadm list |
| 769 | 2025-12-17T09:10:19.726Z | + grep nexus |
| 770 | 2025-12-17T09:10:19.726Z | + [[ 49 -gt 300 ]] |
| 771 | 2025-12-17T09:10:19.726Z | + sleep 1 |
| 772 | 2025-12-17T09:10:20.715Z | + retry=50 |
| 773 | 2025-12-17T09:10:20.739Z | + zoneadm list |
| 774 | 2025-12-17T09:10:20.739Z | + grep nexus |
| 775 | 2025-12-17T09:10:20.739Z | + [[ 50 -gt 300 ]] |
| 776 | 2025-12-17T09:10:20.739Z | + sleep 1 |
| 777 | 2025-12-17T09:10:21.729Z | + retry=51 |
| 778 | 2025-12-17T09:10:21.752Z | + zoneadm list |
| 779 | 2025-12-17T09:10:21.752Z | + grep nexus |
| 780 | 2025-12-17T09:10:21.752Z | + [[ 51 -gt 300 ]] |
| 781 | 2025-12-17T09:10:21.752Z | + sleep 1 |
| 782 | 2025-12-17T09:10:22.742Z | + retry=52 |
| 783 | 2025-12-17T09:10:22.765Z | + zoneadm list |
| 784 | 2025-12-17T09:10:22.765Z | + grep nexus |
| 785 | 2025-12-17T09:10:22.765Z | + [[ 52 -gt 300 ]] |
| 786 | 2025-12-17T09:10:22.765Z | + sleep 1 |
| 787 | 2025-12-17T09:10:23.765Z | + retry=53 |
| 788 | 2025-12-17T09:10:23.789Z | + zoneadm list |
| 789 | 2025-12-17T09:10:23.789Z | + grep nexus |
| 790 | 2025-12-17T09:10:23.789Z | + [[ 53 -gt 300 ]] |
| 791 | 2025-12-17T09:10:23.789Z | + sleep 1 |
| 792 | 2025-12-17T09:10:24.769Z | + retry=54 |
| 793 | 2025-12-17T09:10:24.792Z | + zoneadm list |
| 794 | 2025-12-17T09:10:24.792Z | + grep nexus |
| 795 | 2025-12-17T09:10:24.792Z | + [[ 54 -gt 300 ]] |
| 796 | 2025-12-17T09:10:24.792Z | + sleep 1 |
| 797 | 2025-12-17T09:10:25.782Z | + retry=55 |
| 798 | 2025-12-17T09:10:25.805Z | + zoneadm list |
| 799 | 2025-12-17T09:10:25.806Z | + grep nexus |
| 800 | 2025-12-17T09:10:25.806Z | + [[ 55 -gt 300 ]] |
| 801 | 2025-12-17T09:10:25.806Z | + sleep 1 |
| 802 | 2025-12-17T09:10:26.794Z | + retry=56 |
| 803 | 2025-12-17T09:10:26.818Z | + zoneadm list |
| 804 | 2025-12-17T09:10:26.819Z | + grep nexus |
| 805 | 2025-12-17T09:10:26.819Z | + [[ 56 -gt 300 ]] |
| 806 | 2025-12-17T09:10:26.819Z | + sleep 1 |
| 807 | 2025-12-17T09:10:27.806Z | + retry=57 |
| 808 | 2025-12-17T09:10:27.830Z | + zoneadm list |
| 809 | 2025-12-17T09:10:27.830Z | + grep nexus |
| 810 | 2025-12-17T09:10:27.830Z | + [[ 57 -gt 300 ]] |
| 811 | 2025-12-17T09:10:27.830Z | + sleep 1 |
| 812 | 2025-12-17T09:10:28.819Z | + retry=58 |
| 813 | 2025-12-17T09:10:28.843Z | + zoneadm list |
| 814 | 2025-12-17T09:10:28.843Z | + grep nexus |
| 815 | 2025-12-17T09:10:28.866Z | + [[ 58 -gt 300 ]] |
| 816 | 2025-12-17T09:10:28.866Z | + sleep 1 |
| 817 | 2025-12-17T09:10:29.880Z | + retry=59 |
| 818 | 2025-12-17T09:10:29.880Z | + zoneadm list |
| 819 | 2025-12-17T09:10:29.880Z | + grep nexus |
| 820 | 2025-12-17T09:10:29.908Z | + [[ 59 -gt 300 ]] |
| 821 | 2025-12-17T09:10:29.908Z | + sleep 1 |
| |
| 822 | 2025-12-17T09:12:02.745Z | panic[cpu0]/thread=fffffe00b7ef2c20: pageout_deadman: stuck pushing the same page for 90 seconds (freemem is 107538) |
| 823 | 2025-12-17T09:12:02.794Z | fffffe00b7ef2a60 genunix:pageout_deadman+62 () |
| 824 | 2025-12-17T09:12:02.840Z | fffffe00b7ef2ad0 genunix:clock+7b3 () |
| 825 | 2025-12-17T09:12:02.885Z | fffffe00b7ef2b60 genunix:cyclic_softint+e1 () |
| 826 | 2025-12-17T09:12:02.930Z | fffffe00b7ef2b80 unix:cbe_softclock+23 () |
| 827 | 2025-12-17T09:12:03.121Z | fffffe00b7ef2bd0 unix:av_dispatch_softvect+72 () |
| 828 | 2025-12-17T09:12:03.154Z | fffffe00b7ef2c00 apix:apix_dispatch_softint+35 () |
| 829 | 2025-12-17T09:12:03.188Z | fffffe00b9f15840 unix:switch_sp_and_call+15 () |
| 830 | 2025-12-17T09:12:03.221Z | fffffe00b9f15890 apix:apix_do_softint+5a () |
| 831 | 2025-12-17T09:12:03.255Z | fffffe00b9f158f0 apix:apix_do_interrupt+2bf () |
| 832 | 2025-12-17T09:12:03.329Z | fffffe00b9f15900 unix:cmnint+c3 () |
| 833 | 2025-12-17T09:12:03.367Z | fffffe00b9f15a40 genunix:checkpage+4e () |
| 834 | 2025-12-17T09:12:03.399Z | fffffe00b9f15b10 genunix:pageout_scanner+1e5 () |
| 835 | 2025-12-17T09:12:03.433Z | fffffe00b9f15b20 unix:thread_start+b () |
| 836 | 2025-12-17T09:12:03.466Z | skipping system dump - no dump device configured |
| 837 | 2025-12-17T09:12:03.500Z | rebooting... |
| 838 | 2025-12-17T09:12:50.460Z | ^[[2J^[[1;1H^[[8;25;80t^[[2J^[[1;1H^[[1;1H^[[0;37;40m^[[2J^[[1;1H^[[25; |
| 839 | 2025-12-17T09:12:51.671Z | 8H98^[[1;1H^[[25;78H9D^[[1;1H^[[25;78H9A^[[1;1H^[[25;78H9C^[[1;1H^[[25;78 |
| 840 | 2025-12-17T09:12:51.708Z | 9D^[[1;1H^[[25;78H9A^[[1;1H^[[25;78H9C^[[1;1H^[[25;78HB4^[[1;1H^[[25;78HB |
| 841 | 2025-12-17T09:12:54.086Z | ^[[1;1H^[[25;78HB4^[[1;1H^[[1;1H^[[1;1HPress <DEL> to enter setup, <F1 |
| 842 | 2025-12-17T09:12:55.299Z | > Display Boot Menu, <F12> Force Network Boot ^[[25;78H92^[[2;1H^[ |
| 843 | 2025-12-17T09:12:55.335Z | 25;78HA0^[[2;1H^[[25;78HA0^[[2;1H^[[25;78H99^[[2;1H^[[25;78H92^[[2;1H^[[2 |
| 844 | 2025-12-17T09:12:55.369Z | ;78H92^[[2;1H^[[25;78H72^[[2;1H^[[25;78HA0^[[2;1H^[[25;78HA2^[[2;1H^[[25; |
| 845 | 2025-12-17T09:12:55.402Z | 8HA2^[[2;1H^[[25;78HA2^[[2;1H^[[25;78HA2^[[2;1H^[[25;78HA2^[[2;1H^[[25;78 |
| 846 | 2025-12-17T09:12:55.727Z | A2^[[2;1H^[[25;78HA2^[[2;1H^[[25;78HA2^[[2;1H^[[25;78H92^[[2;1H^[[25; |
| 847 | 2025-12-17T09:12:56.007Z | 8HB2^[[2;1H^[[2J^[[1;1H^[[01;00H^[[0m^[[2;37;40mInitializing Intel(R) B |
| 848 | 2025-12-17T09:12:56.082Z | ot Agent GE v1.5.53 ^[[02;00HP |
| 849 | 2025-12-17T09:12:56.193Z | E 2.1 Build 092 (WfM 2.0) ^[ |
| 850 | 2025-12-17T09:12:56.235Z | 03;00HPress Ctrl+S to enter the Setup Menu. ^[ |
| 851 | 2025-12-17T09:12:56.338Z | 04;00H ^[ |
| 852 | 2025-12-17T09:12:56.374Z | 05;00H ^[ |
| 853 | 2025-12-17T09:12:56.407Z | 06;00H ^[ |
| 854 | 2025-12-17T09:12:56.441Z | 07;00H ^[ |
| 855 | 2025-12-17T09:12:56.474Z | 08;00H ^[ |
| 856 | 2025-12-17T09:12:56.508Z | 09;00H ^[ |
| 857 | 2025-12-17T09:12:56.541Z | 10;00H ^[ |
| 858 | 2025-12-17T09:12:56.620Z | 11;00H ^[ |
| 859 | 2025-12-17T09:12:56.832Z | 12;00H ^[ |
| 860 | 2025-12-17T09:12:56.877Z | 13;00H ^[ |
| 861 | 2025-12-17T09:12:57.001Z | 14;00H ^[ |
| 862 | 2025-12-17T09:12:57.034Z | 15;00H ^[ |
| 863 | 2025-12-17T09:12:57.068Z | 16;00H ^[ |
| 864 | 2025-12-17T09:12:57.101Z | 17;00H ^[ |
| 865 | 2025-12-17T09:12:57.135Z | 18;00H ^[ |
| 866 | 2025-12-17T09:12:57.169Z | 19;00H ^[ |
| 867 | 2025-12-17T09:12:57.202Z | 20;00H ^[ |
| 868 | 2025-12-17T09:12:57.236Z | 21;00H ^[ |
| 869 | 2025-12-17T09:12:57.269Z | 22;00H ^[ |
| 870 | 2025-12-17T09:12:57.303Z | 23;00H ^[ |
| 871 | 2025-12-17T09:12:57.370Z | 24;00H ^[ |
| 872 | 2025-12-17T09:12:57.403Z | 24;00H^[[03;39H^[[03;00HPress Ctrl+S to enter the Setup Menu.. |
| 873 | 2025-12-17T09:12:58.351Z | ^[[03;39H^[[01;00HInitializ |
| 874 | 2025-12-17T09:12:58.424Z | ng Intel(R) Boot Agent GE v1.5.53 ^[ |
| 875 | 2025-12-17T09:12:58.459Z | 02;00HPXE 2.1 Build 092 (WfM 2.0) ^[ |
| 876 | 2025-12-17T09:12:58.493Z | 03;00H ^[ |
| 877 | 2025-12-17T09:12:58.526Z | 04;00H ^[ |
| 878 | 2025-12-17T09:12:58.560Z | 05;00HInitializing Intel(R) Boot Agent GE v1.5.53 ^[ |
| 879 | 2025-12-17T09:12:58.593Z | 06;00HPXE 2.1 Build 092 (WfM 2.0) ^[ |
| 880 | 2025-12-17T09:12:59.073Z | 07;00HPress Ctrl+S to enter the Setup Menu. ^[ |
| 881 | 2025-12-17T09:12:59.109Z | 08;00H ^[ |
| 882 | 2025-12-17T09:12:59.141Z | 09;00H ^[ |
| 883 | 2025-12-17T09:12:59.175Z | 10;00H ^[ |
| 884 | 2025-12-17T09:12:59.209Z | 11;00H ^[ |
| 885 | 2025-12-17T09:12:59.242Z | 12;00H ^[ |
| 886 | 2025-12-17T09:12:59.332Z | 13;00H ^[ |
| 887 | 2025-12-17T09:12:59.466Z | 14;00H ^[ |
| 888 | 2025-12-17T09:12:59.556Z | 15;00H ^[ |
| 889 | 2025-12-17T09:12:59.623Z | 16;00H ^[ |
| 890 | 2025-12-17T09:12:59.680Z | 17;00H ^[ |
| 891 | 2025-12-17T09:12:59.723Z | 18;00H ^[ |
| 892 | 2025-12-17T09:12:59.757Z | 19;00H ^[ |
| 893 | 2025-12-17T09:12:59.790Z | 20;00H ^[ |
| 894 | 2025-12-17T09:12:59.824Z | 21;00H ^[ |
| 895 | 2025-12-17T09:12:59.858Z | 22;00H ^[ |
| 896 | 2025-12-17T09:12:59.891Z | 23;00H ^[ |
| 897 | 2025-12-17T09:12:59.925Z | 24;00H ^[ |
| 898 | 2025-12-17T09:12:59.958Z | 24;00H^[[07;39H^[[07;00HPress Ctrl+S to enter the Setup Menu.. |
| 899 | 2025-12-17T09:13:03.217Z | ^[[07;39H^[[2J^[[1;1H^[[8;25;80t^[ |
| 900 | 2025-12-17T09:13:03.250Z | 2J^[[1;1H^[[1;1H^[[0;37;40mPress <DEL> to enter setup, <F10> Display |
| 901 | 2025-12-17T09:13:03.285Z | Boot Menu, <F12> Force Network Boot B2^[ |
| 902 | 2025-12-17T09:13:03.317Z | 2;1H^[[2;1H^[[2;1H |
| 903 | 2025-12-17T09:13:03.350Z | ^[[3;1H^[[3;1HBIOS Version: MZ32-AR0- |
| 904 | 2025-12-17T09:13:03.384Z | 0-M06 ^[[4;1H^[[4 |
| 905 | 2025-12-17T09:13:03.418Z | 1HBIOS Date: 07/10/2021 ^[ |
| 906 | 2025-12-17T09:13:03.451Z | 5;1H^[[5;1HProcessor Type: AMD EPYC 7413 24-Core Processor |
| 907 | 2025-12-17T09:13:04.023Z | ^[[6;1H^[[6;1HProcessor Speed: 2650 MHz |
| 908 | 2025-12-17T09:13:04.056Z | ^[[7;1H^[[7;1HTot |
| 909 | 2025-12-17T09:13:04.090Z | l Memory: 131072MB (DDR4 3200) ^[ |
| 910 | 2025-12-17T09:13:04.123Z | 8;1H^[[8;1HBMC Firmware Version: 12.53.07 |
| 911 | 2025-12-17T09:13:04.157Z | ^[[9;1H^[[9;1HBMC IP : 172.20.4.57 |
| 912 | 2025-12-17T09:13:04.191Z | ^[[10;1H |
| 913 | 2025-12-17T09:13:04.249Z | ^[ |
| 914 | 2025-12-17T09:13:04.290Z | 11;1H^[[11;1H |
| 915 | 2025-12-17T09:13:04.325Z | ^[[12;1H^[[12;1HUSB Devices total: 0 KBDs |
| 916 | 2025-12-17T09:13:04.358Z | 0 MICE, 0 MASS, 3 HUBs ^[[13;1H^[[13 |
| 917 | 2025-12-17T09:13:04.392Z | 1H ^[ |
| 918 | 2025-12-17T09:13:04.427Z | 14;1H^[[14;1HDetected ATA/ATAPI Devices... |
| 919 | 2025-12-17T09:13:04.459Z | ^[[15;1H^[[15;1H |
| 920 | 2025-12-17T09:13:04.492Z | ^[[16;1H^[[16 |
| 921 | 2025-12-17T09:13:04.526Z | 1HDetected NVME Devices... ^[ |
| 922 | 2025-12-17T09:13:04.559Z | 17;1H^[[17;1HHP SSD EX950 1TB Size: 1024.2GB |
| 923 | 2025-12-17T09:13:04.592Z | ^[[18;1H^[[18;1HHP SSD EX950 1TB Size: 10 |
| 924 | 2025-12-17T09:13:07.255Z | 4.2GB ^[[19;1H^[[19 |
| 925 | 2025-12-17T09:13:07.294Z | 1HPreparing Bios Setup configuration.... ^[ |
| 926 | 2025-12-17T09:13:07.326Z | 20;1H^[[20;1HImporting Bios settings from BMC. |
| 927 | 2025-12-17T09:13:08.527Z | ^[[21;1H^[[21;1H |
| 928 | 2025-12-17T09:13:11.420Z | ^[[2J^[[1;1H^[ |
| 929 | 2025-12-17T09:13:12.037Z | 2J^[[1;1H^[[2J^[[1;1H^[[01;00H |
| 930 | 2025-12-17T09:13:29.288Z | iPXE initialising devices...ok |
| 931 | 2025-12-17T09:13:29.318Z | ^[[0m |
| 932 | 2025-12-17T09:13:29.351Z | ^[[1miPXE 1.21.1+ (g8bff5)^[[0m -- Open Source Network Boot Firmwar |
| 933 | 2025-12-17T09:13:29.385Z | -- ^[[36mhttp://ipxe.org^[[0m |
| 934 | 2025-12-17T09:13:29.418Z | Features: DNS HTTP iSCSI TFTP SRP AoE ELF MBOOT PXE bzImage Menu PXEXT |
| 935 | 2025-12-17T09:13:31.197Z | net0: 18:c0:4d:0d:9f:b2 using i350 on 0000:c3:00.0 (Ethernet) [open] |
| 936 | 2025-12-17T09:13:31.232Z | [Link:up, TX:0 TXE:1 RX:0 RXE:0] |
| 937 | 2025-12-17T09:13:31.266Z | [TXE: 1 x "Network unreachable (http://ipxe.org/28086011)"] |
| 938 | 2025-12-17T09:13:34.468Z | Configuring (net0 18:c0:4d:0d:9f:b2)...... ok |
| 939 | 2025-12-17T09:13:34.500Z | net0: 10.151.1.100/255.255.255.0 gw 10.151.1.1 |
| 940 | 2025-12-17T09:13:34.928Z | Filename: http://172.20.0.5:9969/ipxe/buskin |
| 941 | 2025-12-17T09:13:34.965Z | http://172.20.0.5:9969/ipxe/buskin... ok |
| 942 | 2025-12-17T09:13:35.002Z | buskin : 557 bytes [script] |
| 943 | 2025-12-17T09:15:24.424Z | worker failed without completing job |
| 944 | 2025-12-17T09:15:24.427Z | task 2 was incomplete, marked failed |