01KM23E73QH0KSFDJFG01QV2PC: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-*.log*",
        "%/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
        "/tmp/kstat/*.kstat",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.39",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01KM23FTE1R2N3TZ02DJKWCG13

Tags:

Output:

SEQ GLOBAL TIME DETAILS
12026-03-19T04:14:21.398Zjob dependencies complete; ready to run (waiting for 23 m 14 s)
22026-03-19T08:45:01.733Zjob assigned to worker 01KM2M882Z5MC67EPVB8D36GFE [factory lab, buskin/19630] (queued for 4 h 30 m 40 s)
32026-03-19T08:45:01.751Zhttp://172.20.0.5:9969/ipxe/buskin... ok
42026-03-19T08:45:01.853Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/kernel/amd64/unix... ok
52026-03-19T08:45:01.904Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive... ok
62026-03-19T08:45:01.943Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive.hash... ok
72026-03-19T08:45:01.976Zhttp://172.20.0.5:9969/postboot/buskin... ok
82026-03-19T08:45:02.010ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
92026-03-19T08:45:02.043ZWARNING: cpu0: failed to update microcode from version 0xa001133 to 0xa0011de
102026-03-19T08:45:02.077ZWARNING: cpu0: using older fallback microcode; update the system firmware
112026-03-19T08:45:02.110Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d5
122026-03-19T08:45:02.144ZOxide Helios Version helios-2.0.23743 64-bit
132026-03-19T08:45:02.178ZConfiguring devices.
142026-03-19T08:45:02.212Z##################################################
152026-03-19T08:45:02.246Z#### ############# ##
162026-03-19T08:45:02.278Z### ### ############ ##
172026-03-19T08:45:02.312Z## ### # ## ### ## Oxide Computer Company ##
182026-03-19T08:45:02.347Z## ## ## ### # ### ##
192026-03-19T08:45:02.380Z## # ### #### #### This Station Under ##
202026-03-19T08:45:02.413Z### ### #### # ### Computer Control ##
212026-03-19T08:45:02.452Z#### #### ### ## ##
222026-03-19T08:45:02.509Z##################################################
232026-03-19T08:45:02.542ZHostname: buskin
242026-03-19T08:45:02.575ZLoading smf(7) service descriptions:
252026-03-19T08:45:03.593Zdownloading input: /input/package/work/package.tar.gz
262026-03-19T08:51:45.936Zdownloaded input: /input/package/work/package.tar.gz
 
272026-03-19T08:51:45.960Zstarting task 0: "setup"
282026-03-19T08:51:45.960Z++ uname -s
292026-03-19T08:51:45.960Z+ kern=SunOS
302026-03-19T08:51:45.961Z+ build_user=build
312026-03-19T08:51:45.961Z+ build_uid=12345
322026-03-19T08:51:45.961Z+ work_dir=/work
332026-03-19T08:51:45.961Z+ input_dir=/input
342026-03-19T08:51:45.961Z+ [[ 0 == 12345 ]]
352026-03-19T08:51:45.961Z+ case "$kern" in
362026-03-19T08:51:45.961Z+ groupadd -g 12345 build
372026-03-19T08:51:45.961Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
382026-03-19T08:51:47.954Z+ zfs create -o mountpoint=/work rpool/work
392026-03-19T08:51:47.979Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
402026-03-19T08:51:47.979Z+ home_fs=zfs
412026-03-19T08:51:48.003Z+ [[ zfs == autofs ]]
422026-03-19T08:51:48.004Z+ mkdir -p /home/build
432026-03-19T08:51:48.004Z+ chown build:build /home/build /work
442026-03-19T08:51:49.985Z+ chmod 0700 /home/build /work
452026-03-19T08:51:50.009Zprocess exited: duration 4052 ms, exit code 0
 
462026-03-19T08:51:50.058Zstarting task 1: "authentication"
472026-03-19T08:51:50.105Zprocess exited: duration 46 ms, exit code 0
 
482026-03-19T08:51:50.154Zstarting task 2: "build"
492026-03-19T08:51:50.179Z+ trap _exit_trap EXIT
502026-03-19T08:51:50.179Z+ OPTE_COMMIT=
512026-03-19T08:51:50.180Z+ [[ x != \x ]]
522026-03-19T08:51:50.180Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
532026-03-19T08:51:50.271Zsecpolicy_fs_linkdir: 0x55 = 0x55
542026-03-19T08:51:50.271Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
552026-03-19T08:51:50.271Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
562026-03-19T08:51:50.272Zsecpolicy_fs_linkdir+3: 0 = 0xe5
572026-03-19T08:51:50.272Zsecpolicy_fs_linkdir+4: 0 = 0xb8
582026-03-19T08:51:50.272Zsecpolicy_fs_linkdir+5: 0 = 0x1
592026-03-19T08:51:50.272Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
602026-03-19T08:51:50.272Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
612026-03-19T08:51:50.272Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
622026-03-19T08:51:50.272Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
632026-03-19T08:51:50.272Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
642026-03-19T08:51:50.296Z+ [[ -d /opt/oxide ]]
652026-03-19T08:51:50.297Z+ pfexec mv /opt/oxide /opt/oxide-underneath
662026-03-19T08:51:50.297Z+ pfexec mkdir /opt/oxide
672026-03-19T08:51:50.297Z+ pfexec mount -F tmpfs -O swap /opt/oxide
682026-03-19T08:51:50.297Z+ [[ -d /opt/oxide-underneath ]]
692026-03-19T08:51:50.297Z+ cd /opt/oxide-underneath
702026-03-19T08:51:50.297Z+ pfexec tar ceEp@/f - .
712026-03-19T08:51:50.297Z+ cd /opt/oxide
722026-03-19T08:51:50.297Z+ pfexec tar xveEp@/f -
732026-03-19T08:51:50.338Zx ., 0 bytes, 0 tape blocks
742026-03-19T08:51:50.339Zx ./opte, 0 bytes, 0 tape blocks
752026-03-19T08:51:50.339Zx ./opte/bin, 0 bytes, 0 tape blocks
762026-03-19T08:51:50.339Zx ./opte/bin/opteadm, 22792392 bytes, 44517 tape blocks
772026-03-19T08:51:50.363Z+ pfexec rm -rf /opt/oxide-underneath
782026-03-19T08:51:50.364Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
792026-03-19T08:51:50.364Z+ pfexec mkdir /opt/oxide/work
802026-03-19T08:51:50.388Z+ pfexec chown build:build /opt/oxide/work
812026-03-19T08:51:50.388Z+ cd /opt/oxide/work
822026-03-19T08:51:50.388Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
832026-03-19T08:52:11.075ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
842026-03-19T08:52:11.075Zx .github/buildomat/ci-env.sh, 196 bytes, 1 tape blocks
852026-03-19T08:52:11.075Zx out/target/test, 107 bytes, 1 tape blocks
862026-03-19T08:52:11.075Zx out/npuzone/npuzone, 28059200 bytes, 54804 tape blocks
872026-03-19T08:52:11.075Zx package-manifest.toml, 34895 bytes, 69 tape blocks
882026-03-19T08:52:11.075Zx smf/sled-agent/non-gimlet/config.toml, 5328 bytes, 11 tape blocks
892026-03-19T08:52:11.075Zx target/release/omicron-package, 41533528 bytes, 81121 tape blocks
902026-03-19T08:52:11.075Zx target/release/xtask, 11707104 bytes, 22866 tape blocks
912026-03-19T08:52:11.075Zx target/debug/bootstrap, 343445864 bytes, 670793 tape blocks
922026-03-19T08:52:11.075Zx tests/bootstrap-e6f8ef4052bfb0c2, 32061232 bytes, 62620 tape blocks
932026-03-19T08:52:11.075Zx tests/commtest-24d786529c44a77c, 33172184 bytes, 64790 tape blocks
942026-03-19T08:52:11.075Zx tests/dhcp_server-58b78f45b9570bbf, 32061248 bytes, 62620 tape blocks
952026-03-19T08:52:11.075Zx tests/end_to_end_tests-a99e61b3cc7d9b0e, 386756056 bytes, 755383 tape blocks
962026-03-19T08:52:11.075Zx out/clickhouse.tar.gz, 308949345 bytes, 603417 tape blocks
972026-03-19T08:52:11.075Zx out/clickhouse_keeper.tar.gz, 294064996 bytes, 574346 tape blocks
982026-03-19T08:52:11.075Zx out/clickhouse_server.tar.gz, 308948327 bytes, 603415 tape blocks
992026-03-19T08:52:11.075Zx out/cockroachdb.tar.gz, 160290155 bytes, 313067 tape blocks
1002026-03-19T08:52:11.075Zx out/crucible-pantry-zone.tar.gz, 39925388 bytes, 77980 tape blocks
1012026-03-19T08:52:11.075Zx out/crucible-utils.tar, 18169344 bytes, 35487 tape blocks
1022026-03-19T08:52:11.075Zx out/crucible-zone.tar.gz, 54402848 bytes, 106256 tape blocks
1032026-03-19T08:52:11.075Zx out/external-dns.tar.gz, 44650404 bytes, 87208 tape blocks
1042026-03-19T08:52:11.075Zx out/internal-dns.tar.gz, 44650000 bytes, 87208 tape blocks
1052026-03-19T08:52:11.075Zx out/mg-ddm-gz.tar, 80258048 bytes, 156754 tape blocks
1062026-03-19T08:52:11.075Zx out/nexus.tar.gz, 214136581 bytes, 418236 tape blocks
1072026-03-19T08:52:11.075Zx out/ntp.tar.gz, 26027900 bytes, 50836 tape blocks
1082026-03-19T08:52:11.075Zx out/omicron-gateway-softnpu.tar.gz, 28902695 bytes, 56451 tape blocks
1092026-03-19T08:52:11.076Zx out/omicron-sled-agent.tar, 191020544 bytes, 373087 tape blocks
1102026-03-19T08:52:11.076Zx out/overlay.tar.gz, 3030 bytes, 6 tape blocks
1112026-03-19T08:52:11.076Zx out/oximeter.tar.gz, 58817109 bytes, 114878 tape blocks
1122026-03-19T08:52:11.076Zx out/oxlog.tar, 3872256 bytes, 7563 tape blocks
1132026-03-19T08:52:11.076Zx out/probe.tar.gz, 3109568 bytes, 6074 tape blocks
1142026-03-19T08:52:11.076Zx out/propolis-server.tar.gz, 76424187 bytes, 149266 tape blocks
1152026-03-19T08:52:11.076Zx out/pumpkind-gz.tar, 27286528 bytes, 53294 tape blocks
1162026-03-19T08:52:11.076Zx out/switch-softnpu.tar.gz, 376679150 bytes, 735702 tape blocks
1172026-03-19T08:52:11.101Z
1182026-03-19T08:52:11.101Zreal 20.681396158
1192026-03-19T08:52:11.101Zuser 18.570974714
1202026-03-19T08:52:11.101Zsys 3.537630187
1212026-03-19T08:52:11.101Ztrap 0.000668203
1222026-03-19T08:52:11.101Ztflt 0.000000000
1232026-03-19T08:52:11.101Zdflt 0.000000000
1242026-03-19T08:52:11.101Zkflt 0.000000000
1252026-03-19T08:52:11.101Zlock 0.000000000
1262026-03-19T08:52:11.101Zslp 18.708809384
1272026-03-19T08:52:11.101Zlat 0.542612360
1282026-03-19T08:52:11.101Zstop 0.000130761
1292026-03-19T08:52:11.101Z+ source .github/buildomat/ci-env.sh
1302026-03-19T08:52:11.101Z++ export CARGO_TERM_COLOR=always
1312026-03-19T08:52:11.101Z++ CARGO_TERM_COLOR=always
1322026-03-19T08:52:11.101Z+ bmat address ls
1332026-03-19T08:52:11.101ZNAME CIDR FIRST LAST COUNT
1342026-03-19T08:52:11.101Zextra 10.151.1.0/24 10.151.1.164 10.151.1.227 64
1352026-03-19T08:52:11.101Z++ bmat address ls -f extra -Ho first,last,count
1362026-03-19T08:52:11.101Z+ set -- 10.151.1.164 10.151.1.227 64
1372026-03-19T08:52:11.101Z+ EXTRA_IP_START=10.151.1.164
1382026-03-19T08:52:11.101Z+ EXTRA_IP_END=10.151.1.227
1392026-03-19T08:52:11.101Z+ EXTRA_IP_COUNT=64
1402026-03-19T08:52:11.101Z+ (( EXTRA_IP_COUNT >= 32 ))
1412026-03-19T08:52:11.101Z+ EXTRA_IP_BASE=10.151.1
1422026-03-19T08:52:11.101Z+ EXTRA_IP_FOCTET=164
1432026-03-19T08:52:11.101Z+ EXTRA_IP_LOCTET=227
1442026-03-19T08:52:11.101Z+ SERVICE_IP_POOL_START=10.151.1.164
1452026-03-19T08:52:11.102Z+ SERVICE_IP_POOL_END=10.151.1.173
1462026-03-19T08:52:11.102Z+ DNS_IP1=10.151.1.164
1472026-03-19T08:52:11.102Z+ DNS_IP2=10.151.1.165
1482026-03-19T08:52:11.102Z+ UPLINK_IP=10.151.1.174
1492026-03-19T08:52:11.102Z+ IPPOOL_START=10.151.1.175
1502026-03-19T08:52:11.102Z+ IPPOOL_END=10.151.1.227
1512026-03-19T08:52:11.102Z++ ipadm show-addr -po type,addr
1522026-03-19T08:52:11.102Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
1532026-03-19T08:52:11.102Z+ GATEWAY_IP=10.151.1.100
1542026-03-19T08:52:11.102Z+ [[ -n 10.151.1.100 ]]
1552026-03-19T08:52:11.102Z+ ping -s 10.151.1.100 56 1
1562026-03-19T08:52:11.102ZPING 10.151.1.100: 56 data bytes
1572026-03-19T08:52:11.102Z64 bytes from 10.151.1.100: icmp_seq=0. time=0.012 ms
1582026-03-19T08:52:11.102Z
1592026-03-19T08:52:11.102Z----10.151.1.100 PING Statistics----
1602026-03-19T08:52:11.102Z1 packets transmitted, 1 packets received, 0% packet loss
1612026-03-19T08:52:11.102Zround-trip (ms) min/avg/max/stddev = 0.012/0.012/0.012/-nan
1622026-03-19T08:52:11.102Z++ arp -an
1632026-03-19T08:52:11.102Z++ awk -vgw=10.151.1.100 '$2 == gw {print $NF}'
1642026-03-19T08:52:11.127Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
1652026-03-19T08:52:11.127Z+ routeadm -e ipv4-forwarding -u
1662026-03-19T08:52:11.202Z+ PXA_START=10.151.1.164
1672026-03-19T08:52:11.202Z+ PXA_END=10.151.1.227
1682026-03-19T08:52:11.202Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
1692026-03-19T08:52:11.339Z+ ptime -m pfexec ./target/release/xtask virtual-hardware --vdev-dir /scratch create --gateway-ip 10.151.1.100 --gateway-mac 18:c0:4d:0d:9f:b2 --pxa-start 10.151.1.164 --pxa-end 10.151.1.227
1702026-03-19T08:52:11.363ZCouldn't find Cargo.toml, using /opt/oxide/work as workspace root
1712026-03-19T08:52:11.415Zcreating virtual hardware
1722026-03-19T08:52:11.440Zcreating /scratch/m2_0.vdev
1732026-03-19T08:52:11.440Zcreating /scratch/m2_1.vdev
1742026-03-19T08:52:11.440Zcreating /scratch/u2_0.vdev
1752026-03-19T08:52:11.441Zcreating /scratch/u2_1.vdev
1762026-03-19T08:52:11.441Zcreating /scratch/u2_2.vdev
1772026-03-19T08:52:11.441Zcreating /scratch/u2_3.vdev
1782026-03-19T08:52:11.441Zcreating /scratch/u2_4.vdev
1792026-03-19T08:52:11.441Zcreating /scratch/u2_5.vdev
1802026-03-19T08:52:11.441Zcreating /scratch/u2_6.vdev
1812026-03-19T08:52:11.441Zcreating /scratch/u2_7.vdev
1822026-03-19T08:52:11.441Zcreating /scratch/u2_8.vdev
1832026-03-19T08:52:11.465ZSimnet net0/sc0_0 exists
1842026-03-19T08:52:11.490ZSimnet net1/sc1_0 exists
1852026-03-19T08:52:11.490ZVnic sc0_1 exists
1862026-03-19T08:52:24.421ZUsing 10.151.1.100 as gateway ip
1872026-03-19T08:52:24.421Zusing 18:C0:4D:0D:9F:B2 as gateway mac
1882026-03-19T08:52:24.421Zconfiguring SoftNPU ARP entry
1892026-03-19T08:52:24.447Zconfiguring SoftNPU proxy ARP
1902026-03-19T08:52:24.499ZSoftNPU state:
1912026-03-19T08:52:24.499Z local v6:
1922026-03-19T08:52:24.499Z local v4:
1932026-03-19T08:52:24.499Z router v6_idx:
1942026-03-19T08:52:24.499Z router v6_routes:
1952026-03-19T08:52:24.499Z router v4_idx:
1962026-03-19T08:52:24.499Z router v4_routes:
1972026-03-19T08:52:24.499Z resolver v4:
1982026-03-19T08:52:24.499Z 10.151.1.100 -> 18:c0:4d:0d:9f:b2
1992026-03-19T08:52:24.499Z resolver v6:
2002026-03-19T08:52:24.499Z nat v4:
2012026-03-19T08:52:24.499Z nat_v6:
2022026-03-19T08:52:24.499Z attached subnet v4:
2032026-03-19T08:52:24.499Z attached subnet v6:
2042026-03-19T08:52:24.499Z port_mac:
2052026-03-19T08:52:24.499Z proxy arp:
2062026-03-19T08:52:24.499Z 10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
2072026-03-19T08:52:24.499Zcreated virtual hardware
2082026-03-19T08:52:24.524Z
2092026-03-19T08:52:24.524Zreal 13.157073566
2102026-03-19T08:52:24.524Zuser 0.664162460
2112026-03-19T08:52:24.524Zsys 3.404621492
2122026-03-19T08:52:24.524Ztrap 0.000091640
2132026-03-19T08:52:24.524Ztflt 0.000212635
2142026-03-19T08:52:24.524Zdflt 0.000474919
2152026-03-19T08:52:24.524Zkflt 0.000000000
2162026-03-19T08:52:24.524Zlock 10:11.178783654
2172026-03-19T08:52:24.524Zslp 39.723971604
2182026-03-19T08:52:24.524Zlat 0.127811735
2192026-03-19T08:52:24.524Zstop 0.258346328
2202026-03-19T08:52:24.524Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml pkg/config.toml
2212026-03-19T08:52:24.557Z+ sed -E -i~ 's/(m2|u2)(.*\.vdev)/\/scratch\/\1\2/g' pkg/config.toml
2222026-03-19T08:52:24.604Z+ diff -u pkg/config.toml~ pkg/config.toml
2232026-03-19T08:52:24.604Z--- pkg/config.toml~ Mon Jul 24 01:21:28 2006
2242026-03-19T08:52:24.604Z+++ pkg/config.toml Thu Mar 19 08:52:24 2026
2252026-03-19T08:52:24.604Z@@ -29,18 +29,18 @@
2262026-03-19T08:52:24.604Z # These paths have the prefix of either "u2" or "m2", followed by an underscore,
2272026-03-19T08:52:24.604Z # followed by a string that is embedded into their fake serial values.
2282026-03-19T08:52:24.604Z vdevs = [
2292026-03-19T08:52:24.604Z- "m2_0.vdev",
2302026-03-19T08:52:24.604Z- "m2_1.vdev",
2312026-03-19T08:52:24.604Z+ "/scratch/m2_0.vdev",
2322026-03-19T08:52:24.604Z+ "/scratch/m2_1.vdev",
2332026-03-19T08:52:24.604Z
2342026-03-19T08:52:24.604Z- "u2_0.vdev",
2352026-03-19T08:52:24.604Z- "u2_1.vdev",
2362026-03-19T08:52:24.604Z- "u2_2.vdev",
2372026-03-19T08:52:24.604Z- "u2_3.vdev",
2382026-03-19T08:52:24.604Z- "u2_4.vdev",
2392026-03-19T08:52:24.604Z- "u2_5.vdev",
2402026-03-19T08:52:24.604Z- "u2_6.vdev",
2412026-03-19T08:52:24.604Z- "u2_7.vdev",
2422026-03-19T08:52:24.604Z- "u2_8.vdev",
2432026-03-19T08:52:24.604Z+ "/scratch/u2_0.vdev",
2442026-03-19T08:52:24.604Z+ "/scratch/u2_1.vdev",
2452026-03-19T08:52:24.604Z+ "/scratch/u2_2.vdev",
2462026-03-19T08:52:24.604Z+ "/scratch/u2_3.vdev",
2472026-03-19T08:52:24.604Z+ "/scratch/u2_4.vdev",
2482026-03-19T08:52:24.604Z+ "/scratch/u2_5.vdev",
2492026-03-19T08:52:24.604Z+ "/scratch/u2_6.vdev",
2502026-03-19T08:52:24.604Z+ "/scratch/u2_7.vdev",
2512026-03-19T08:52:24.604Z+ "/scratch/u2_8.vdev",
2522026-03-19T08:52:24.605Z ]
2532026-03-19T08:52:24.605Z
2542026-03-19T08:52:24.605Z # The amount of memory held back for services which exist between zero and one
2552026-03-19T08:52:24.605Z+ true
2562026-03-19T08:52:24.605Z++ grep -c -E 'u2.*\.vdev' pkg/config.toml
2572026-03-19T08:52:24.605Z+ EXPECTED_ZPOOL_COUNT=9
2582026-03-19T08:52:24.605Z+ echo 'expected number of zpools is 9'
2592026-03-19T08:52:24.605Zexpected number of zpools is 9
2602026-03-19T08:52:24.605Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
2612026-03-19T08:52:24.605Z+ SILO_NAME=recovery
2622026-03-19T08:52:24.605Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
2632026-03-19T08:52:24.605Z+ EXTERNAL_DNS_DOMAIN=oxide.test
2642026-03-19T08:52:24.605Z+ sed -i~ '
2652026-03-19T08:52:24.605Z /^external_dns_ips/c\
2662026-03-19T08:52:24.605Zexternal_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
2672026-03-19T08:52:24.605Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
2682026-03-19T08:52:24.605Z /^first/c\
2692026-03-19T08:52:24.605Zfirst = "10.151.1.164"
2702026-03-19T08:52:24.605Z /^last/c\
2712026-03-19T08:52:24.605Zlast = "10.151.1.173"
2722026-03-19T08:52:24.605Z }
2732026-03-19T08:52:24.605Z /^infra_ip_first/c\
2742026-03-19T08:52:24.605Zinfra_ip_first = "10.151.1.174"
2752026-03-19T08:52:24.605Z /^infra_ip_last/c\
2762026-03-19T08:52:24.605Zinfra_ip_last = "10.151.1.174"
2772026-03-19T08:52:24.605Z /^\[\[rack_network_config.ports/,/^$/ {
2782026-03-19T08:52:24.605Z /^routes/c\
2792026-03-19T08:52:24.605Zroutes = \[{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}\]
2802026-03-19T08:52:24.605Z /^addresses/c\
2812026-03-19T08:52:24.605Zaddresses = \[{address = "10.151.1.174/24"} \]
2822026-03-19T08:52:24.605Z }
2832026-03-19T08:52:24.605Z' pkg/config-rss.toml
2842026-03-19T08:52:24.605Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
2852026-03-19T08:52:24.605Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
2862026-03-19T08:52:24.605Z+++ pkg/config-rss.toml Thu Mar 19 08:52:24 2026
2872026-03-19T08:52:24.605Z@@ -33,7 +33,7 @@
2882026-03-19T08:52:24.605Z # the DNS domain delegated to the rack by the customer. Each of these addresses
2892026-03-19T08:52:24.605Z # must be contained in one of the "internal services" IP Pool ranges listed
2902026-03-19T08:52:24.605Z # below.
2912026-03-19T08:52:24.605Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
2922026-03-19T08:52:24.606Z+external_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
2932026-03-19T08:52:24.606Z
2942026-03-19T08:52:24.606Z # Initial TLS certificates for the external API
2952026-03-19T08:52:24.606Z #
2962026-03-19T08:52:24.606Z@@ -69,8 +69,8 @@
2972026-03-19T08:52:24.606Z #
2982026-03-19T08:52:24.606Z # For more on this and what to put here, see docs/how-to-run.adoc.
2992026-03-19T08:52:24.606Z [[internal_services_ip_pool_ranges]]
3002026-03-19T08:52:24.606Z-first = "192.168.1.20"
3012026-03-19T08:52:24.606Z-last = "192.168.1.29"
3022026-03-19T08:52:24.606Z+first = "10.151.1.164"
3032026-03-19T08:52:24.606Z+last = "10.151.1.173"
3042026-03-19T08:52:24.606Z
3052026-03-19T08:52:24.606Z # TODO - this configuration is subject to change going forward. Ultimately these
3062026-03-19T08:52:24.606Z # parameters should be provided to the control plane via wicket, but we need to
3072026-03-19T08:52:24.606Z@@ -91,8 +91,8 @@
3082026-03-19T08:52:24.606Z # A range of IP addresses used by Boundary Services on the external network. In
3092026-03-19T08:52:24.606Z # a real system, these would be addresses of the uplink ports on the Sidecar.
3102026-03-19T08:52:24.606Z # With softnpu, only one address is used.
3112026-03-19T08:52:24.606Z-infra_ip_first = "192.168.1.30"
3122026-03-19T08:52:24.606Z-infra_ip_last = "192.168.1.30"
3132026-03-19T08:52:24.606Z+infra_ip_first = "10.151.1.174"
3142026-03-19T08:52:24.606Z+infra_ip_last = "10.151.1.174"
3152026-03-19T08:52:24.606Z
3162026-03-19T08:52:24.606Z # Configurations for BGP routers to run on the scrimlets.
3172026-03-19T08:52:24.606Z bgp = []
3182026-03-19T08:52:24.606Z@@ -100,9 +100,9 @@
3192026-03-19T08:52:24.606Z # You can configure multiple uplinks by repeating the following stanza
3202026-03-19T08:52:24.606Z [[rack_network_config.ports]]
3212026-03-19T08:52:24.606Z # Routes associated with this port.
3222026-03-19T08:52:24.631Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
3232026-03-19T08:52:24.631Z+routes = [{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}]
3242026-03-19T08:52:24.631Z # Addresses associated with this port.
3252026-03-19T08:52:24.631Z-addresses = [{address = "192.168.1.30/24"}]
3262026-03-19T08:52:24.631Z+addresses = [{address = "10.151.1.174/24"} ]
3272026-03-19T08:52:24.631Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
3282026-03-19T08:52:24.631Z port = "qsfp0"
3292026-03-19T08:52:24.631Z # The speed of this port.
3302026-03-19T08:52:24.632Z+ true
3312026-03-19T08:52:24.632Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml pkg/config.toml
3322026-03-19T08:52:24.632Za pkg/config-rss.toml 8K
3332026-03-19T08:52:24.632Za pkg/config.toml 6K
3342026-03-19T08:52:24.632Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~ pkg/config.toml pkg/config.toml~
3352026-03-19T08:52:24.632Z+ TLS_NAME=recovery.sys.oxide.test
3362026-03-19T08:52:24.632Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
3372026-03-19T08:52:24.974Z.....+..............+...+......+.+......+...+.....+.+...+.....+......+.......+........+.+...+...+..+...+..........+........+......+...+.......+.....+......+............+.+.....+..........+.....+......+......+....+...+..+......+...+.+...+......+.....+............+...................+...+...+........+....+.....+......+.+......+..+.......+........+...+.......+...+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*......................+.+...............+..................+..+...+....+..+....+...............+..+...+.......+..+...+.......+...+...+.....+.........+.......+...+...+.....+...+...............+..........+..............................+...........+.+.........+..................+........+.........+.+.....+...+....+.....+.........................+...............+...+..+............+.........+....+........+..........+............+........+....+...+..+................+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3382026-03-19T08:52:25.523Z.+....+......+........+...+.......+......+...........+...+.+.........+.....+...+...+.+..............+....+...+...+.....+......+....+..+....+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*......+.....+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*......+..+................+.......................+.+.........+..+............+..........+..+...+......+....+............+...+...+...+.....+............+..................+.+.........+..+.........+....+......+.........+..+..........+.........+..............................+...+..+...+......+.+.....+...............+.+.....+...............+.......+.....+.............+.........+....................................+...+...........+......+....+.....+...+.+............+.....+...+......+.........+............+.......+...+........+..........+.....+......+...+......................+......+.....+...+......+.+........+............+.........+...+.......+.....+.............+......+...............+.....+.+..............+..........+.................+.......+..+.+.....+..................+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3392026-03-19T08:52:25.548Z-----
3402026-03-19T08:52:25.548Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
3412026-03-19T08:52:25.572Za pkg/initial-tls-cert.pem 2K
3422026-03-19T08:52:25.572Za pkg/initial-tls-key.pem 4K
3432026-03-19T08:52:25.572Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
3442026-03-19T08:52:25.573Z+ rmdir pkg
3452026-03-19T08:52:25.573Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
3462026-03-19T08:52:25.573Z+ pfexec mkdir -p /usr/oxide
3472026-03-19T08:52:25.573Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v2025112000
3482026-03-19T08:52:26.198Z+ pfexec chmod +x /usr/oxide/oxide
3492026-03-19T08:52:26.223Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
 
3502026-03-19T09:03:07.136Zworker agent experienced a fatal error; aborting job
3512026-03-19T09:03:07.136Ztask 2 was incomplete, marked failed