01HQ90C60H98TW6425KH3EZXJS: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log*",
        "%/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.28",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01HQ90CBDN2VF6ZG94G2NN0M1Z

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12024-02-22T18:13:57.326Zjob dependencies complete; ready to run (waiting for 0 s)
22024-02-22T18:19:20.890Zjob assigned to worker 01HQ90CCEPCH9ZB4E87H2HK5ZY (queued for 5 m 23 s)
32024-02-22T18:19:20.954Zhttp://172.20.0.5:9969/ipxe/buskin... ok
42024-02-22T18:19:21.003Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/kernel/amd64/unix... ok
52024-02-22T18:19:21.039Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive... ok
62024-02-22T18:19:21.079Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive.hash... ok
72024-02-22T18:19:21.113Zhttp://172.20.0.5:9969/postboot/buskin... ok
82024-02-22T18:19:21.145ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
92024-02-22T18:19:21.184Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d1
102024-02-22T18:19:21.218ZOxide Helios Version helios-2.0.22451 64-bit
112024-02-22T18:19:21.252ZConfiguring devices.
122024-02-22T18:19:21.286Z##################################################
132024-02-22T18:19:21.319Z#### ############# ##
142024-02-22T18:19:21.354Z### ### ############ ##
152024-02-22T18:19:21.388Z## ### # ## ### ## Oxide Computer Company ##
162024-02-22T18:19:21.420Z## ## ## ### # ### ##
172024-02-22T18:19:21.454Z## # ### #### #### This Station Under ##
182024-02-22T18:19:21.488Z### ### #### # ### Computer Control ##
192024-02-22T18:19:21.520Z#### #### ### ## ##
202024-02-22T18:19:21.554Z##################################################
212024-02-22T18:19:21.588ZHostname: buskin
222024-02-22T18:19:21.621ZLoading smf(7) service descriptions:
232024-02-22T18:19:22.040Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap-22e5a97eec9bb5c6.gz
242024-02-22T18:19:23.973Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap-22e5a97eec9bb5c6.gz
252024-02-22T18:19:23.998Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
262024-02-22T18:19:42.412Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
272024-02-22T18:19:42.412Zdownloading input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-25e7d53a8e782c84.gz
282024-02-22T18:20:01.383Zdownloaded input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-25e7d53a8e782c84.gz
292024-02-22T18:20:01.383Zdownloading input: /input/ci-tools/work/caboose-util.gz
302024-02-22T18:20:02.393Zdownloaded input: /input/ci-tools/work/caboose-util.gz
312024-02-22T18:20:02.393Zdownloading input: /input/ci-tools/work/tufaceous.gz
322024-02-22T18:20:05.468Zdownloaded input: /input/ci-tools/work/tufaceous.gz
332024-02-22T18:20:05.493Zdownloading input: /input/package/work/version.txt
342024-02-22T18:20:05.517Zdownloaded input: /input/package/work/version.txt
352024-02-22T18:20:05.517Zdownloading input: /input/package/work/package.tar.gz
362024-02-22T18:20:21.032Zdownloaded input: /input/package/work/package.tar.gz
372024-02-22T18:20:21.032Zdownloading input: /input/package/work/global-zone-packages.tar.gz
382024-02-22T18:20:43.688Zdownloaded input: /input/package/work/global-zone-packages.tar.gz
392024-02-22T18:20:43.688Zdownloading input: /input/package/work/trampoline-global-zone-packages.tar.gz
402024-02-22T18:20:50.635Zdownloaded input: /input/package/work/trampoline-global-zone-packages.tar.gz
412024-02-22T18:20:50.661Zdownloading input: /input/package/work/zones/clickhouse.tar.gz
422024-02-22T18:21:23.282Zdownloaded input: /input/package/work/zones/clickhouse.tar.gz
432024-02-22T18:21:23.282Zdownloading input: /input/package/work/zones/clickhouse_keeper.tar.gz
442024-02-22T18:21:55.820Zdownloaded input: /input/package/work/zones/clickhouse_keeper.tar.gz
452024-02-22T18:21:55.821Zdownloading input: /input/package/work/zones/cockroachdb.tar.gz
462024-02-22T18:22:18.066Zdownloaded input: /input/package/work/zones/cockroachdb.tar.gz
472024-02-22T18:22:18.066Zdownloading input: /input/package/work/zones/crucible-pantry-zone.tar.gz
482024-02-22T18:22:22.892Zdownloaded input: /input/package/work/zones/crucible-pantry-zone.tar.gz
492024-02-22T18:22:22.892Zdownloading input: /input/package/work/zones/crucible-zone.tar.gz
502024-02-22T18:22:29.698Zdownloaded input: /input/package/work/zones/crucible-zone.tar.gz
512024-02-22T18:22:29.698Zdownloading input: /input/package/work/zones/external-dns.tar.gz
522024-02-22T18:22:33.864Zdownloaded input: /input/package/work/zones/external-dns.tar.gz
532024-02-22T18:22:33.864Zdownloading input: /input/package/work/zones/internal-dns.tar.gz
542024-02-22T18:22:37.978Zdownloaded input: /input/package/work/zones/internal-dns.tar.gz
552024-02-22T18:22:37.978Zdownloading input: /input/package/work/zones/ntp.tar.gz
562024-02-22T18:22:38.028Zdownloaded input: /input/package/work/zones/ntp.tar.gz
572024-02-22T18:22:38.053Zdownloading input: /input/package/work/zones/omicron-gateway-asic.tar.gz
582024-02-22T18:22:41.404Zdownloaded input: /input/package/work/zones/omicron-gateway-asic.tar.gz
592024-02-22T18:22:41.404Zdownloading input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
602024-02-22T18:22:44.790Zdownloaded input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
612024-02-22T18:22:44.815Zdownloading input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
622024-02-22T18:22:59.663Zdownloaded input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
632024-02-22T18:22:59.663Zdownloading input: /input/package/work/zones/omicron-nexus.tar.gz
642024-02-22T18:23:14.486Zdownloaded input: /input/package/work/zones/omicron-nexus.tar.gz
652024-02-22T18:23:14.486Zdownloading input: /input/package/work/zones/overlay.tar.gz
662024-02-22T18:23:14.580Zdownloaded input: /input/package/work/zones/overlay.tar.gz
672024-02-22T18:23:14.581Zdownloading input: /input/package/work/zones/oximeter.tar.gz
682024-02-22T18:23:24.032Zdownloaded input: /input/package/work/zones/oximeter.tar.gz
692024-02-22T18:23:24.032Zdownloading input: /input/package/work/zones/propolis-server.tar.gz
702024-02-22T18:23:35.328Zdownloaded input: /input/package/work/zones/propolis-server.tar.gz
712024-02-22T18:23:35.329Zdownloading input: /input/package/work/zones/switch-asic.tar.gz
722024-02-22T18:24:23.035Zdownloaded input: /input/package/work/zones/switch-asic.tar.gz
732024-02-22T18:24:23.060Zdownloading input: /input/package/work/zones/switch-softnpu.tar.gz
742024-02-22T18:24:58.095Zdownloaded input: /input/package/work/zones/switch-softnpu.tar.gz
 
752024-02-22T18:24:58.095Zstarting task 0: "setup"
762024-02-22T18:24:58.119Z++ uname -s
772024-02-22T18:24:58.119Z+ kern=SunOS
782024-02-22T18:24:58.119Z+ case "$kern" in
792024-02-22T18:24:58.119Z+ groupadd -g 12345 build
802024-02-22T18:24:58.120Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
812024-02-22T18:25:00.112Z+ zfs create -o mountpoint=/work rpool/work
822024-02-22T18:25:00.155Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
832024-02-22T18:25:00.179Z+ home_fs=zfs
842024-02-22T18:25:00.179Z+ [[ zfs == autofs ]]
852024-02-22T18:25:00.179Z+ mkdir -p /home/build
862024-02-22T18:25:00.179Z+ chown build:build /home/build /work
872024-02-22T18:25:02.163Z+ chmod 0700 /home/build /work
882024-02-22T18:25:02.188Zprocess exited: duration 4071 ms, exit code 0
 
892024-02-22T18:25:02.238Zstarting task 1: "authentication"
902024-02-22T18:25:02.283Zprocess exited: duration 43 ms, exit code 0
 
912024-02-22T18:25:02.336Zstarting task 2: "build"
922024-02-22T18:25:02.362Z+ trap _exit_trap EXIT
932024-02-22T18:25:02.362Z+ set +x
942024-02-22T18:25:29.844Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
952024-02-22T18:25:30.089Zsecpolicy_fs_linkdir: 0x55 = 0x55
962024-02-22T18:25:30.090Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
972024-02-22T18:25:30.090Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
982024-02-22T18:25:30.090Zsecpolicy_fs_linkdir+3: 0 = 0xe5
992024-02-22T18:25:30.115Zsecpolicy_fs_linkdir+4: 0 = 0xb8
1002024-02-22T18:25:30.115Zsecpolicy_fs_linkdir+5: 0 = 0x1
1012024-02-22T18:25:30.115Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
1022024-02-22T18:25:30.116Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
1032024-02-22T18:25:30.116Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
1042024-02-22T18:25:30.116Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
1052024-02-22T18:25:30.116Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
1062024-02-22T18:25:30.116Z+ [[ -d /opt/oxide ]]
1072024-02-22T18:25:30.116Z+ pfexec mv /opt/oxide /opt/oxide-underneath
1082024-02-22T18:25:30.116Z+ pfexec mkdir /opt/oxide
1092024-02-22T18:25:30.116Z+ pfexec mount -F tmpfs -O swap /opt/oxide
1102024-02-22T18:25:30.116Z+ [[ -d /opt/oxide-underneath ]]
1112024-02-22T18:25:30.116Z+ cd /opt/oxide-underneath
1122024-02-22T18:25:30.116Z+ pfexec tar ceEp@/f - .
1132024-02-22T18:25:30.116Z+ cd /opt/oxide
1142024-02-22T18:25:30.116Z+ pfexec tar xveEp@/f -
1152024-02-22T18:25:30.150Zx ., 0 bytes, 0 tape blocks
1162024-02-22T18:25:30.150Zx ./opte, 0 bytes, 0 tape blocks
1172024-02-22T18:25:30.150Zx ./opte/bin, 0 bytes, 0 tape blocks
1182024-02-22T18:25:30.150Zx ./opte/bin/opteadm, 51799224 bytes, 101171 tape blocks
1192024-02-22T18:25:30.176Z+ pfexec rm -rf /opt/oxide-underneath
1202024-02-22T18:25:30.176Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
1212024-02-22T18:25:30.201Z+ pfexec mkdir /opt/oxide/work
1222024-02-22T18:25:30.202Z+ pfexec chown build:build /opt/oxide/work
1232024-02-22T18:25:30.202Z+ cd /opt/oxide/work
1242024-02-22T18:25:30.202Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
1252024-02-22T18:25:31.905ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
1262024-02-22T18:25:31.905Zx out/mg-ddm-gz.tar, 69308416 bytes, 135368 tape blocks
1272024-02-22T18:25:31.905Zx out/omicron-sled-agent.tar, 123022848 bytes, 240279 tape blocks
1282024-02-22T18:25:31.905Zx out/oxlog.tar, 8557568 bytes, 16714 tape blocks
1292024-02-22T18:25:31.905Zx out/target/test, 75 bytes, 1 tape blocks
1302024-02-22T18:25:31.906Zx out/npuzone/npuzone, 37184104 bytes, 72626 tape blocks
1312024-02-22T18:25:31.906Zx package-manifest.toml, 24242 bytes, 48 tape blocks
1322024-02-22T18:25:31.906Zx smf/sled-agent/non-gimlet/config.toml, 3448 bytes, 7 tape blocks
1332024-02-22T18:25:31.906Zx target/release/omicron-package, 45890040 bytes, 89629 tape blocks
1342024-02-22T18:25:31.906Zx tools/create_virtual_hardware.sh, 3184 bytes, 7 tape blocks
1352024-02-22T18:25:31.906Zx tools/virtual_hardware.sh, 3416 bytes, 7 tape blocks
1362024-02-22T18:25:31.906Zx tools/scrimlet/create-softnpu-zone.sh, 627 bytes, 2 tape blocks
1372024-02-22T18:25:31.906Zx tools/scrimlet/destroy-softnpu-zone.sh, 203 bytes, 1 tape blocks
1382024-02-22T18:25:31.906Zx tools/scrimlet/softnpu-init.sh, 2157 bytes, 5 tape blocks
1392024-02-22T18:25:31.906Zx tools/scrimlet/softnpu-zone.txt, 330 bytes, 1 tape blocks
1402024-02-22T18:25:31.906Zx tools/scrimlet/softnpu.toml, 171 bytes, 1 tape blocks
1412024-02-22T18:25:31.931Z
1422024-02-22T18:25:31.931Zreal 1.714193090
1432024-02-22T18:25:31.931Zuser 1.541923383
1442024-02-22T18:25:31.931Zsys 0.297530857
1452024-02-22T18:25:31.931Ztrap 0.000079283
1462024-02-22T18:25:31.931Ztflt 0.000000000
1472024-02-22T18:25:31.931Zdflt 0.000000000
1482024-02-22T18:25:31.932Zkflt 0.000000000
1492024-02-22T18:25:31.932Zlock 0.000000000
1502024-02-22T18:25:31.932Zslp 1.545918824
1512024-02-22T18:25:31.932Zlat 0.040653184
1522024-02-22T18:25:31.932Zstop 0.000124741
1532024-02-22T18:25:31.932Z+ cp /input/package/work/zones/clickhouse_keeper.tar.gz /input/package/work/zones/clickhouse.tar.gz /input/package/work/zones/cockroachdb.tar.gz /input/package/work/zones/crucible-pantry-zone.tar.gz /input/package/work/zones/crucible-zone.tar.gz /input/package/work/zones/external-dns.tar.gz /input/package/work/zones/internal-dns.tar.gz /input/package/work/zones/ntp.tar.gz /input/package/work/zones/omicron-gateway-asic.tar.gz /input/package/work/zones/omicron-gateway-softnpu.tar.gz /input/package/work/zones/omicron-nexus-single-sled.tar.gz /input/package/work/zones/omicron-nexus.tar.gz /input/package/work/zones/overlay.tar.gz /input/package/work/zones/oximeter.tar.gz /input/package/work/zones/propolis-server.tar.gz /input/package/work/zones/switch-asic.tar.gz /input/package/work/zones/switch-softnpu.tar.gz out/
1542024-02-22T18:25:33.110Z+ mv out/omicron-nexus-single-sled.tar.gz out/omicron-nexus.tar.gz
1552024-02-22T18:25:33.135Z+ mkdir tests
1562024-02-22T18:25:33.135Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1572024-02-22T18:25:33.135Z+ ptime -m gunzip
1582024-02-22T18:25:33.135Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-22e5a97eec9bb5c6
1592024-02-22T18:25:33.371Z
1602024-02-22T18:25:33.396Zreal 0.235841619
1612024-02-22T18:25:33.396Zuser 0.214172775
1622024-02-22T18:25:33.396Zsys 0.021420168
1632024-02-22T18:25:33.397Ztrap 0.000018240
1642024-02-22T18:25:33.397Ztflt 0.000000000
1652024-02-22T18:25:33.397Zdflt 0.000000000
1662024-02-22T18:25:33.397Zkflt 0.000000000
1672024-02-22T18:25:33.397Zlock 0.000000000
1682024-02-22T18:25:33.397Zslp 0.000019550
1692024-02-22T18:25:33.397Zlat 0.000023871
1702024-02-22T18:25:33.397Zstop 0.000127681
1712024-02-22T18:25:33.397Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-22e5a97eec9bb5c6
1722024-02-22T18:25:33.397Z+ chmod a+x tests/bootstrap-22e5a97eec9bb5c6
1732024-02-22T18:25:33.397Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1742024-02-22T18:25:33.397Z+ ptime -m gunzip
1752024-02-22T18:25:33.397Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1762024-02-22T18:25:35.932Z
1772024-02-22T18:25:35.957Zreal 2.550008358
1782024-02-22T18:25:35.957Zuser 2.405321457
1792024-02-22T18:25:35.957Zsys 0.144280693
1802024-02-22T18:25:35.957Ztrap 0.000181924
1812024-02-22T18:25:35.957Ztflt 0.000000000
1822024-02-22T18:25:35.957Zdflt 0.000000000
1832024-02-22T18:25:35.957Zkflt 0.000000000
1842024-02-22T18:25:35.957Zlock 0.000000000
1852024-02-22T18:25:35.957Zslp 0.000020400
1862024-02-22T18:25:35.957Zlat 0.000019410
1872024-02-22T18:25:35.957Zstop 0.000123342
1882024-02-22T18:25:35.957Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1892024-02-22T18:25:35.957Z+ chmod a+x tests/bootstrap
1902024-02-22T18:25:35.957Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1912024-02-22T18:25:35.957Z+ ptime -m gunzip
1922024-02-22T18:25:35.958Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-25e7d53a8e782c84
1932024-02-22T18:25:38.556Z
1942024-02-22T18:25:38.582Zreal 2.612336684
1952024-02-22T18:25:38.582Zuser 2.426383216
1962024-02-22T18:25:38.582Zsys 0.185569691
1972024-02-22T18:25:38.582Ztrap 0.000160743
1982024-02-22T18:25:38.582Ztflt 0.000000000
1992024-02-22T18:25:38.582Zdflt 0.000000000
2002024-02-22T18:25:38.582Zkflt 0.000000000
2012024-02-22T18:25:38.582Zlock 0.000000000
2022024-02-22T18:25:38.582Zslp 0.000024520
2032024-02-22T18:25:38.582Zlat 0.000015970
2042024-02-22T18:25:38.582Zstop 0.000115831
2052024-02-22T18:25:38.582Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-25e7d53a8e782c84
2062024-02-22T18:25:38.582Z+ chmod a+x tests/end_to_end_tests-25e7d53a8e782c84
2072024-02-22T18:25:38.582Z+ bmat address ls
2082024-02-22T18:25:38.582ZNAME CIDR FIRST LAST COUNT
2092024-02-22T18:25:38.582Zextra 10.151.1.0/24 10.151.1.164 10.151.1.227 64
2102024-02-22T18:25:38.582Z++ bmat address ls -f extra -Ho first,last,count
2112024-02-22T18:25:38.608Z+ set -- 10.151.1.164 10.151.1.227 64
2122024-02-22T18:25:38.608Z+ EXTRA_IP_START=10.151.1.164
2132024-02-22T18:25:38.608Z+ EXTRA_IP_END=10.151.1.227
2142024-02-22T18:25:38.608Z+ EXTRA_IP_COUNT=64
2152024-02-22T18:25:38.608Z+ (( EXTRA_IP_COUNT >= 32 ))
2162024-02-22T18:25:38.608Z+ EXTRA_IP_BASE=10.151.1
2172024-02-22T18:25:38.608Z+ EXTRA_IP_FOCTET=164
2182024-02-22T18:25:38.608Z+ EXTRA_IP_LOCTET=227
2192024-02-22T18:25:38.608Z+ SERVICE_IP_POOL_START=10.151.1.164
2202024-02-22T18:25:38.608Z+ SERVICE_IP_POOL_END=10.151.1.173
2212024-02-22T18:25:38.608Z+ DNS_IP1=10.151.1.164
2222024-02-22T18:25:38.608Z+ DNS_IP2=10.151.1.165
2232024-02-22T18:25:38.608Z+ UPLINK_IP=10.151.1.174
2242024-02-22T18:25:38.608Z+ IPPOOL_START=10.151.1.175
2252024-02-22T18:25:38.608Z+ IPPOOL_END=10.151.1.227
2262024-02-22T18:25:38.608Z++ ipadm show-addr -po type,addr
2272024-02-22T18:25:38.609Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
2282024-02-22T18:25:38.609Z+ GATEWAY_IP=10.151.1.100
2292024-02-22T18:25:38.609Z+ [[ -n 10.151.1.100 ]]
2302024-02-22T18:25:38.609Z+ ping -s 10.151.1.100 56 1
2312024-02-22T18:25:38.609ZPING 10.151.1.100: 56 data bytes
2322024-02-22T18:25:38.609Z64 bytes from 10.151.1.100: icmp_seq=0. time=0.011 ms
2332024-02-22T18:25:38.609Z
2342024-02-22T18:25:38.609Z----10.151.1.100 PING Statistics----
2352024-02-22T18:25:38.609Z1 packets transmitted, 1 packets received, 0% packet loss
2362024-02-22T18:25:38.609Zround-trip (ms) min/avg/max/stddev = 0.011/0.011/0.011/-nan
2372024-02-22T18:25:38.609Z++ arp -an
2382024-02-22T18:25:38.609Z++ awk -vgw=10.151.1.100 '$2 == gw {print $NF}'
2392024-02-22T18:25:38.636Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
2402024-02-22T18:25:38.636Z+ routeadm -e ipv4-forwarding -u
2412024-02-22T18:25:38.721Z+ PXA_START=10.151.1.164
2422024-02-22T18:25:38.721Z+ PXA_END=10.151.1.227
2432024-02-22T18:25:38.721Z+ export GATEWAY_IP GATEWAY_MAC PXA_START PXA_END
2442024-02-22T18:25:38.746Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
2452024-02-22T18:25:38.924Z+ ZPOOL_VDEV_DIR=/scratch
2462024-02-22T18:25:38.924Z+ ptime -m pfexec ./tools/create_virtual_hardware.sh
2472024-02-22T18:25:38.949Z+++ dirname ./tools/create_virtual_hardware.sh
2482024-02-22T18:25:38.949Z++ cd ./tools
2492024-02-22T18:25:38.949Z++ pwd
2502024-02-22T18:25:38.949Z+ SOURCE_DIR=/opt/oxide/work/tools
2512024-02-22T18:25:38.949Z+ OMICRON_TOP=/opt/oxide/work/tools/..
2522024-02-22T18:25:38.949Z+ SOFTNPU_MODE=zone
2532024-02-22T18:25:38.949Z+ . /opt/oxide/work/tools/virtual_hardware.sh
2542024-02-22T18:25:38.949Z++ MARKER=/etc/opt/oxide/NO_INSTALL
2552024-02-22T18:25:38.949Z++ [[ -f /etc/opt/oxide/NO_INSTALL ]]
2562024-02-22T18:25:38.950Z++ dladm show-phys -p -o LINK
2572024-02-22T18:25:38.950Z++ head -1
2582024-02-22T18:25:38.950Z+ PHYSICAL_LINK=igb0
2592024-02-22T18:25:38.950Z+ echo 'Using igb0 as physical link'
2602024-02-22T18:25:38.950ZUsing igb0 as physical link
2612024-02-22T18:25:38.950Z+ ensure_run_as_root
2622024-02-22T18:25:38.975Z++ id -u
2632024-02-22T18:25:38.975Z+ [[ 0 -ne 0 ]]
2642024-02-22T18:25:38.975Z+ ensure_zpools
2652024-02-22T18:25:38.975Z+ ZPOOL_TYPES=('oxp_' 'oxi_')
2662024-02-22T18:25:38.975Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
2672024-02-22T18:25:38.975Z+ readarray -t ZPOOLS
2682024-02-22T18:25:38.975Z++ grep '"oxp_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
2692024-02-22T18:25:38.975Z++ sed 's/[ ",]//g'
2702024-02-22T18:25:38.975Z+ for ZPOOL in "${ZPOOLS[@]}"
2712024-02-22T18:25:38.975Z+ echo 'Zpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]'
2722024-02-22T18:25:38.975ZZpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]
2732024-02-22T18:25:38.975Z+ VDEV_PATH=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2742024-02-22T18:25:38.975Z+ [[ -f /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
2752024-02-22T18:25:38.975Z+ dd if=/dev/zero of=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
2762024-02-22T18:25:38.975Z0+0 records in
2772024-02-22T18:25:38.976Z0+0 records out
2782024-02-22T18:25:38.976Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
2792024-02-22T18:25:38.976Z+ success 'ZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
2802024-02-22T18:25:38.976Z+ set +x
2812024-02-22T18:25:38.976ZZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
2822024-02-22T18:25:38.976Z++ zpool list -o name
2832024-02-22T18:25:38.976Z++ grep oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
2842024-02-22T18:25:38.976Z+ [[ -z '' ]]
2852024-02-22T18:25:38.976Z+ zpool create -f oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2862024-02-22T18:25:39.010Z+ success 'ZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
2872024-02-22T18:25:39.035Z+ set +x
2882024-02-22T18:25:39.035ZZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists
2892024-02-22T18:25:39.035Z+ for ZPOOL in "${ZPOOLS[@]}"
2902024-02-22T18:25:39.035Z+ echo 'Zpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
2912024-02-22T18:25:39.035ZZpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]
2922024-02-22T18:25:39.036Z+ VDEV_PATH=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
2932024-02-22T18:25:39.036Z+ [[ -f /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
2942024-02-22T18:25:39.036Z+ dd if=/dev/zero of=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
2952024-02-22T18:25:39.036Z0+0 records in
2962024-02-22T18:25:39.036Z0+0 records out
2972024-02-22T18:25:39.036Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
2982024-02-22T18:25:39.036Z+ success 'ZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
2992024-02-22T18:25:39.036Z+ set +x
3002024-02-22T18:25:39.036ZZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3012024-02-22T18:25:39.036Z++ zpool list -o name
3022024-02-22T18:25:39.036Z++ grep oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
3032024-02-22T18:25:39.036Z+ [[ -z '' ]]
3042024-02-22T18:25:39.036Z+ zpool create -f oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3052024-02-22T18:25:39.061Z+ success 'ZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3062024-02-22T18:25:39.061Z+ set +x
3072024-02-22T18:25:39.061ZZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3082024-02-22T18:25:39.061Z+ for ZPOOL in "${ZPOOLS[@]}"
3092024-02-22T18:25:39.062Z+ echo 'Zpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3102024-02-22T18:25:39.062ZZpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]
3112024-02-22T18:25:39.062Z+ VDEV_PATH=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3122024-02-22T18:25:39.062Z+ [[ -f /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3132024-02-22T18:25:39.062Z+ dd if=/dev/zero of=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3142024-02-22T18:25:39.090Z0+0 records in
3152024-02-22T18:25:39.090Z0+0 records out
3162024-02-22T18:25:39.090Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3172024-02-22T18:25:39.090Z+ success 'ZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3182024-02-22T18:25:39.090Z+ set +x
3192024-02-22T18:25:39.090ZZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3202024-02-22T18:25:39.090Z++ zpool list -o name
3212024-02-22T18:25:39.090Z++ grep oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
3222024-02-22T18:25:39.090Z+ [[ -z '' ]]
3232024-02-22T18:25:39.090Z+ zpool create -f oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3242024-02-22T18:25:39.115Z+ success 'ZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3252024-02-22T18:25:39.115Z+ set +x
3262024-02-22T18:25:39.115ZZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3272024-02-22T18:25:39.116Z+ for ZPOOL in "${ZPOOLS[@]}"
3282024-02-22T18:25:39.116Z+ echo 'Zpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3292024-02-22T18:25:39.116ZZpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]
3302024-02-22T18:25:39.116Z+ VDEV_PATH=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3312024-02-22T18:25:39.116Z+ [[ -f /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3322024-02-22T18:25:39.116Z+ dd if=/dev/zero of=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3332024-02-22T18:25:39.116Z0+0 records in
3342024-02-22T18:25:39.116Z0+0 records out
3352024-02-22T18:25:39.116Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3362024-02-22T18:25:39.116Z+ success 'ZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3372024-02-22T18:25:39.116Z+ set +x
3382024-02-22T18:25:39.116ZZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3392024-02-22T18:25:39.116Z++ zpool list -o name
3402024-02-22T18:25:39.116Z++ grep oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03
3412024-02-22T18:25:39.141Z+ [[ -z '' ]]
3422024-02-22T18:25:39.141Z+ zpool create -f oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3432024-02-22T18:25:39.166Z+ success 'ZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3442024-02-22T18:25:39.166Z+ set +x
3452024-02-22T18:25:39.166ZZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3462024-02-22T18:25:39.166Z+ for ZPOOL in "${ZPOOLS[@]}"
3472024-02-22T18:25:39.166Z+ echo 'Zpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3482024-02-22T18:25:39.166ZZpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]
3492024-02-22T18:25:39.166Z+ VDEV_PATH=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3502024-02-22T18:25:39.167Z+ [[ -f /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3512024-02-22T18:25:39.167Z+ dd if=/dev/zero of=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3522024-02-22T18:25:39.167Z0+0 records in
3532024-02-22T18:25:39.167Z0+0 records out
3542024-02-22T18:25:39.167Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3552024-02-22T18:25:39.167Z+ success 'ZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3562024-02-22T18:25:39.167Z+ set +x
3572024-02-22T18:25:39.167ZZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3582024-02-22T18:25:39.167Z++ zpool list -o name
3592024-02-22T18:25:39.167Z++ grep oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03
3602024-02-22T18:25:39.192Z+ [[ -z '' ]]
3612024-02-22T18:25:39.192Z+ zpool create -f oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3622024-02-22T18:25:39.217Z+ success 'ZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3632024-02-22T18:25:39.218Z+ set +x
3642024-02-22T18:25:39.218ZZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3652024-02-22T18:25:39.218Z+ for ZPOOL in "${ZPOOLS[@]}"
3662024-02-22T18:25:39.218Z+ echo 'Zpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]'
3672024-02-22T18:25:39.218ZZpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]
3682024-02-22T18:25:39.218Z+ VDEV_PATH=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3692024-02-22T18:25:39.218Z+ [[ -f /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev ]]
3702024-02-22T18:25:39.218Z+ dd if=/dev/zero of=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev bs=1 count=0 seek=20G
3712024-02-22T18:25:39.218Z0+0 records in
3722024-02-22T18:25:39.218Z0+0 records out
3732024-02-22T18:25:39.218Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3742024-02-22T18:25:39.218Z+ success 'ZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists'
3752024-02-22T18:25:39.218Z+ set +x
3762024-02-22T18:25:39.218ZZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists
3772024-02-22T18:25:39.218Z++ zpool list -o name
3782024-02-22T18:25:39.218Z++ grep oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930
3792024-02-22T18:25:39.218Z+ [[ -z '' ]]
3802024-02-22T18:25:39.218Z+ zpool create -f oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3812024-02-22T18:25:39.243Z+ success 'ZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists'
3822024-02-22T18:25:39.244Z+ set +x
3832024-02-22T18:25:39.244ZZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists
3842024-02-22T18:25:39.244Z+ for ZPOOL in "${ZPOOLS[@]}"
3852024-02-22T18:25:39.244Z+ echo 'Zpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]'
3862024-02-22T18:25:39.244ZZpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]
3872024-02-22T18:25:39.244Z+ VDEV_PATH=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
3882024-02-22T18:25:39.244Z+ [[ -f /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev ]]
3892024-02-22T18:25:39.244Z+ dd if=/dev/zero of=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev bs=1 count=0 seek=20G
3902024-02-22T18:25:39.244Z0+0 records in
3912024-02-22T18:25:39.244Z0+0 records out
3922024-02-22T18:25:39.244Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3932024-02-22T18:25:39.244Z+ success 'ZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists'
3942024-02-22T18:25:39.244Z+ set +x
3952024-02-22T18:25:39.244ZZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists
3962024-02-22T18:25:39.269Z++ zpool list -o name
3972024-02-22T18:25:39.269Z++ grep oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60
3982024-02-22T18:25:39.269Z+ [[ -z '' ]]
3992024-02-22T18:25:39.269Z+ zpool create -f oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
4002024-02-22T18:25:39.294Z+ success 'ZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists'
4012024-02-22T18:25:39.294Z+ set +x
4022024-02-22T18:25:39.294ZZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists
4032024-02-22T18:25:39.294Z+ for ZPOOL in "${ZPOOLS[@]}"
4042024-02-22T18:25:39.294Z+ echo 'Zpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]'
4052024-02-22T18:25:39.294ZZpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]
4062024-02-22T18:25:39.294Z+ VDEV_PATH=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4072024-02-22T18:25:39.294Z+ [[ -f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev ]]
4082024-02-22T18:25:39.294Z+ dd if=/dev/zero of=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev bs=1 count=0 seek=20G
4092024-02-22T18:25:39.294Z0+0 records in
4102024-02-22T18:25:39.294Z0+0 records out
4112024-02-22T18:25:39.294Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4122024-02-22T18:25:39.295Z+ success 'ZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists'
4132024-02-22T18:25:39.295Z+ set +x
4142024-02-22T18:25:39.295ZZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists
4152024-02-22T18:25:39.295Z++ zpool list -o name
4162024-02-22T18:25:39.295Z++ grep oxp_31bd71cd-4736-4a12-a387-9b74b050396f
4172024-02-22T18:25:39.320Z+ [[ -z '' ]]
4182024-02-22T18:25:39.320Z+ zpool create -f oxp_31bd71cd-4736-4a12-a387-9b74b050396f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4192024-02-22T18:25:39.345Z+ success 'ZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists'
4202024-02-22T18:25:39.345Z+ set +x
4212024-02-22T18:25:39.345ZZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists
4222024-02-22T18:25:39.345Z+ for ZPOOL in "${ZPOOLS[@]}"
4232024-02-22T18:25:39.345Z+ echo 'Zpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]'
4242024-02-22T18:25:39.345ZZpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]
4252024-02-22T18:25:39.345Z+ VDEV_PATH=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4262024-02-22T18:25:39.345Z+ [[ -f /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev ]]
4272024-02-22T18:25:39.345Z+ dd if=/dev/zero of=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev bs=1 count=0 seek=20G
4282024-02-22T18:25:39.346Z0+0 records in
4292024-02-22T18:25:39.346Z0+0 records out
4302024-02-22T18:25:39.346Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4312024-02-22T18:25:39.346Z+ success 'ZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists'
4322024-02-22T18:25:39.346Z+ set +x
4332024-02-22T18:25:39.346ZZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists
4342024-02-22T18:25:39.346Z++ zpool list -o name
4352024-02-22T18:25:39.346Z++ grep oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7
4362024-02-22T18:25:39.346Z+ [[ -z '' ]]
4372024-02-22T18:25:39.346Z+ zpool create -f oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4382024-02-22T18:25:39.373Z+ success 'ZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists'
4392024-02-22T18:25:39.373Z+ set +x
4402024-02-22T18:25:39.398ZZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists
4412024-02-22T18:25:39.399Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
4422024-02-22T18:25:39.399Z+ readarray -t ZPOOLS
4432024-02-22T18:25:39.399Z++ grep '"oxi_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
4442024-02-22T18:25:39.399Z++ sed 's/[ ",]//g'
4452024-02-22T18:25:39.399Z+ for ZPOOL in "${ZPOOLS[@]}"
4462024-02-22T18:25:39.399Z+ echo 'Zpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4472024-02-22T18:25:39.399ZZpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]
4482024-02-22T18:25:39.399Z+ VDEV_PATH=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4492024-02-22T18:25:39.399Z+ [[ -f /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4502024-02-22T18:25:39.399Z+ dd if=/dev/zero of=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4512024-02-22T18:25:39.399Z0+0 records in
4522024-02-22T18:25:39.399Z0+0 records out
4532024-02-22T18:25:39.399Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4542024-02-22T18:25:39.399Z+ success 'ZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4552024-02-22T18:25:39.399Z+ set +x
4562024-02-22T18:25:39.399ZZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4572024-02-22T18:25:39.399Z++ zpool list -o name
4582024-02-22T18:25:39.399Z++ grep oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b
4592024-02-22T18:25:39.399Z+ [[ -z '' ]]
4602024-02-22T18:25:39.399Z+ zpool create -f oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4612024-02-22T18:25:39.428Z+ success 'ZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4622024-02-22T18:25:39.428Z+ set +x
4632024-02-22T18:25:39.453ZZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4642024-02-22T18:25:39.453Z+ for ZPOOL in "${ZPOOLS[@]}"
4652024-02-22T18:25:39.453Z+ echo 'Zpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4662024-02-22T18:25:39.453ZZpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]
4672024-02-22T18:25:39.453Z+ VDEV_PATH=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4682024-02-22T18:25:39.453Z+ [[ -f /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4692024-02-22T18:25:39.453Z+ dd if=/dev/zero of=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4702024-02-22T18:25:39.453Z0+0 records in
4712024-02-22T18:25:39.453Z0+0 records out
4722024-02-22T18:25:39.453Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4732024-02-22T18:25:39.453Z+ success 'ZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4742024-02-22T18:25:39.454Z+ set +x
4752024-02-22T18:25:39.454ZZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4762024-02-22T18:25:39.454Z++ zpool list -o name
4772024-02-22T18:25:39.454Z++ grep oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b
4782024-02-22T18:25:39.454Z+ [[ -z '' ]]
4792024-02-22T18:25:39.454Z+ zpool create -f oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4802024-02-22T18:25:39.479Z+ success 'ZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4812024-02-22T18:25:39.479Z+ set +x
4822024-02-22T18:25:39.479ZZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4832024-02-22T18:25:39.479Z+ [[ zone == \z\o\n\e ]]
4842024-02-22T18:25:39.479Z+ ensure_simulated_links igb0
4852024-02-22T18:25:39.479Z+ local PHYSICAL_LINK=igb0
4862024-02-22T18:25:39.479Z+ INDICES=("0" "1")
4872024-02-22T18:25:39.479Z+ for I in "${INDICES[@]}"
4882024-02-22T18:25:39.479Z++ get_simnet_name_if_exists net0
4892024-02-22T18:25:39.479Z++ dladm show-simnet -p -o LINK net0
4902024-02-22T18:25:39.479Z++ echo ''
4912024-02-22T18:25:39.480Z+ [[ -z '' ]]
4922024-02-22T18:25:39.505Z+ dladm create-simnet -t net0
4932024-02-22T18:25:39.505Z+ dladm create-simnet -t sc0_0
4942024-02-22T18:25:39.505Z+ dladm modify-simnet -t -p net0 sc0_0
4952024-02-22T18:25:39.505Z+ dladm set-linkprop -p mtu=9000 sc0_0
4962024-02-22T18:25:39.505Z+ success 'Simnet net0/sc0_0 exists'
4972024-02-22T18:25:39.505Z+ set +x
4982024-02-22T18:25:39.505ZSimnet net0/sc0_0 exists
4992024-02-22T18:25:39.505Z+ for I in "${INDICES[@]}"
5002024-02-22T18:25:39.505Z++ get_simnet_name_if_exists net1
5012024-02-22T18:25:39.505Z++ dladm show-simnet -p -o LINK net1
5022024-02-22T18:25:39.505Z++ echo ''
5032024-02-22T18:25:39.505Z+ [[ -z '' ]]
5042024-02-22T18:25:39.505Z+ dladm create-simnet -t net1
5052024-02-22T18:25:39.530Z+ dladm create-simnet -t sc1_0
5062024-02-22T18:25:39.530Z+ dladm modify-simnet -t -p net1 sc1_0
5072024-02-22T18:25:39.530Z+ dladm set-linkprop -p mtu=9000 sc1_0
5082024-02-22T18:25:39.531Z+ success 'Simnet net1/sc1_0 exists'
5092024-02-22T18:25:39.531Z+ set +x
5102024-02-22T18:25:39.531ZSimnet net1/sc1_0 exists
5112024-02-22T18:25:39.531Z++ get_vnic_name_if_exists sc0_1
5122024-02-22T18:25:39.531Z++ dladm show-vnic -p -o LINK sc0_1
5132024-02-22T18:25:39.531Z++ echo ''
5142024-02-22T18:25:39.531Z+ [[ -z '' ]]
5152024-02-22T18:25:39.531Z+ dladm create-vnic -t sc0_1 -l igb0 -m a8:e1:de:01:70:1d
5162024-02-22T18:25:39.556Z+ [[ -v PROMISC_FILT_OFF ]]
5172024-02-22T18:25:39.556Z+ success 'Vnic sc0_1 exists'
5182024-02-22T18:25:39.556Z+ set +x
5192024-02-22T18:25:39.556ZVnic sc0_1 exists
5202024-02-22T18:25:39.556Z+ warn_if_no_proxy_arp
5212024-02-22T18:25:39.556Z+ [[ -v PXA_START ]]
5222024-02-22T18:25:39.556Z+ [[ -v PXA_END ]]
5232024-02-22T18:25:39.556Z+ ensure_softnpu_zone
5242024-02-22T18:25:39.556Z+ zoneadm list
5252024-02-22T18:25:39.556Z+ grep -q sidecar_softnpu
5262024-02-22T18:25:39.556Z+ [[ -f out/npuzone/npuzone ]]
5272024-02-22T18:25:39.556Z+ out/npuzone/npuzone create sidecar --omicron-zone --ports sc0_0,tfportrear0_0 --ports sc0_1,tfportqsfp0_0 --sidecar-lite-commit e3ea4b495ba0a71801ded0776ae4bbd31df57e26 --softnpu-commit dbab082dfa89da5db5ca2325c257089d2f130092
5282024-02-22T18:25:39.556Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/softnpu/image/dbab082dfa89da5db5ca2325c257089d2f130092/softnpu.sha256.txt
5292024-02-22T18:25:46.325Zdownloading softnpu ...done
5302024-02-22T18:25:46.350Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/libsidecar_lite.so.sha256.txt
5312024-02-22T18:25:51.365Zdownloading asic_program.so ...done
5322024-02-22T18:25:51.390Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/scadm.sha256.txt
5332024-02-22T18:25:55.838Zdownloading scadm ...done
5342024-02-22T18:25:55.863Zcreating port tfportrear0_0 for sc0_0
5352024-02-22T18:25:55.864Zcreating port tfportqsfp0_0 for sc0_1
5362024-02-22T18:25:55.864Zsoftnpu zone setup
5372024-02-22T18:25:55.864Zconfigure zone
5382024-02-22T18:25:55.888Zinstall zone
5392024-02-22T18:25:56.905Zboot zone
5402024-02-22T18:26:00.016Z[sidecar_softnpu] svcs milestone/network
5412024-02-22T18:26:00.043ZSTATE STIME FMRI
5422024-02-22T18:26:00.043Zonline 18:25:58 svc:/milestone/network:default
5432024-02-22T18:26:00.043Z[sidecar_softnpu] /softnpu/npu start
5442024-02-22T18:26:00.068Z
5452024-02-22T18:26:00.094Z+ /opt/oxide/work/tools/scrimlet/softnpu-init.sh
5462024-02-22T18:26:00.094Z+ GATEWAY_IP=10.151.1.100
5472024-02-22T18:26:00.094Z+ echo 'Using 10.151.1.100 as gateway ip'
5482024-02-22T18:26:00.094ZUsing 10.151.1.100 as gateway ip
5492024-02-22T18:26:00.094Z+ [[ ! -v GATEWAY_MAC ]]
5502024-02-22T18:26:00.094Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
5512024-02-22T18:26:00.094Z++ grep -c -E '^([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}$' /dev/fd/63
5522024-02-22T18:26:00.094Z+++ echo 18:c0:4d:0d:9f:b2
5532024-02-22T18:26:00.094Z+ COUNT=1
5542024-02-22T18:26:00.094Z+ [[ 1 -ne 1 ]]
5552024-02-22T18:26:00.094Z+ echo 'Using 18:c0:4d:0d:9f:b2 as gateway mac'
5562024-02-22T18:26:00.094ZUsing 18:c0:4d:0d:9f:b2 as gateway mac
5572024-02-22T18:26:00.094Z+ z_scadm add-arp-entry 10.151.1.100 18:c0:4d:0d:9f:b2
5582024-02-22T18:26:00.094Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-arp-entry 10.151.1.100 18:c0:4d:0d:9f:b2
5592024-02-22T18:26:00.120Z+ PXA_MAC=a8:e1:de:01:70:1d
5602024-02-22T18:26:00.120Z+ [[ -v PXA_START ]]
5612024-02-22T18:26:00.120Z+ z_scadm add-proxy-arp 10.151.1.164 10.151.1.227 a8:e1:de:01:70:1d
5622024-02-22T18:26:00.120Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-proxy-arp 10.151.1.164 10.151.1.227 a8:e1:de:01:70:1d
5632024-02-22T18:26:00.145Z+ z_scadm dump-state
5642024-02-22T18:26:00.145Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
5652024-02-22T18:26:00.170Zlocal v6:
5662024-02-22T18:26:00.170Zlocal v4:
5672024-02-22T18:26:00.170Zrouter v6:
5682024-02-22T18:26:00.170Zrouter v4_idx:
5692024-02-22T18:26:00.171Zrouter v4_routes:
5702024-02-22T18:26:00.171Zresolver v4:
5712024-02-22T18:26:00.171Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
5722024-02-22T18:26:00.171Zresolver v6:
5732024-02-22T18:26:00.171Znat v4:
5742024-02-22T18:26:00.171Znat_v6:
5752024-02-22T18:26:00.171Zport_mac:
5762024-02-22T18:26:00.171Zproxy arp:
5772024-02-22T18:26:00.171Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
5782024-02-22T18:26:00.171Z+ success 'softnpu zone exists'
5792024-02-22T18:26:00.171Z+ set +x
5802024-02-22T18:26:00.171Zsoftnpu zone exists
5812024-02-22T18:26:00.196Z
5822024-02-22T18:26:00.197Zreal 21.243630118
5832024-02-22T18:26:00.197Zuser 0.712665990
5842024-02-22T18:26:00.197Zsys 1.564854565
5852024-02-22T18:26:00.197Ztrap 0.000099286
5862024-02-22T18:26:00.197Ztflt 0.000104947
5872024-02-22T18:26:00.197Zdflt 0.001611255
5882024-02-22T18:26:00.197Zkflt 0.000000000
5892024-02-22T18:26:00.197Zlock 16:23.525859099
5902024-02-22T18:26:00.197Zslp 1:03.332409484
5912024-02-22T18:26:00.197Zlat 0.088091097
5922024-02-22T18:26:00.197Zstop 0.371111727
5932024-02-22T18:26:00.197Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml
5942024-02-22T18:26:00.223Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
5952024-02-22T18:26:00.223Z+ SILO_NAME=recovery
5962024-02-22T18:26:00.223Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
5972024-02-22T18:26:00.223Z+ EXTERNAL_DNS_DOMAIN=oxide.test
5982024-02-22T18:26:00.223Z+ sed -i~ '
5992024-02-22T18:26:00.223Z /^external_dns_ips/c\
6002024-02-22T18:26:00.223Zexternal_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
6012024-02-22T18:26:00.223Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
6022024-02-22T18:26:00.223Z /^first/c\
6032024-02-22T18:26:00.223Zfirst = "10.151.1.164"
6042024-02-22T18:26:00.223Z /^last/c\
6052024-02-22T18:26:00.223Zlast = "10.151.1.173"
6062024-02-22T18:26:00.223Z }
6072024-02-22T18:26:00.223Z /^\[rack_network_config/,/^$/ {
6082024-02-22T18:26:00.223Z /^infra_ip_first/c\
6092024-02-22T18:26:00.223Zinfra_ip_first = "10.151.1.174"
6102024-02-22T18:26:00.223Z /^infra_ip_last/c\
6112024-02-22T18:26:00.223Zinfra_ip_last = "10.151.1.174"
6122024-02-22T18:26:00.223Z }
6132024-02-22T18:26:00.223Z /^\[\[rack_network_config.ports/,/^$/ {
6142024-02-22T18:26:00.224Z /^routes/c\
6152024-02-22T18:26:00.224Zroutes = \[{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}\]
6162024-02-22T18:26:00.224Z /^addresses/c\
6172024-02-22T18:26:00.224Zaddresses = \["10.151.1.174/24"\]
6182024-02-22T18:26:00.224Z }
6192024-02-22T18:26:00.224Z' pkg/config-rss.toml
6202024-02-22T18:26:00.224Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
6212024-02-22T18:26:00.224Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
6222024-02-22T18:26:00.224Z+++ pkg/config-rss.toml Thu Feb 22 18:26:00 2024
6232024-02-22T18:26:00.224Z@@ -33,7 +33,7 @@
6242024-02-22T18:26:00.224Z # the DNS domain delegated to the rack by the customer. Each of these addresses
6252024-02-22T18:26:00.224Z # must be contained in one of the "internal services" IP Pool ranges listed
6262024-02-22T18:26:00.224Z # below.
6272024-02-22T18:26:00.224Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
6282024-02-22T18:26:00.224Z+external_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
6292024-02-22T18:26:00.224Z
6302024-02-22T18:26:00.224Z # Initial TLS certificates for the external API
6312024-02-22T18:26:00.224Z #
6322024-02-22T18:26:00.224Z@@ -69,8 +69,8 @@
6332024-02-22T18:26:00.224Z #
6342024-02-22T18:26:00.224Z # For more on this and what to put here, see docs/how-to-run.adoc.
6352024-02-22T18:26:00.224Z [[internal_services_ip_pool_ranges]]
6362024-02-22T18:26:00.224Z-first = "192.168.1.20"
6372024-02-22T18:26:00.224Z-last = "192.168.1.29"
6382024-02-22T18:26:00.225Z+first = "10.151.1.164"
6392024-02-22T18:26:00.225Z+last = "10.151.1.173"
6402024-02-22T18:26:00.225Z
6412024-02-22T18:26:00.225Z # TODO - this configuration is subject to change going forward. Ultimately these
6422024-02-22T18:26:00.225Z # parameters should be provided to the control plane via wicket, but we need to
6432024-02-22T18:26:00.225Z@@ -100,9 +100,9 @@
6442024-02-22T18:26:00.225Z # You can configure multiple uplinks by repeating the following stanza
6452024-02-22T18:26:00.225Z [[rack_network_config.ports]]
6462024-02-22T18:26:00.225Z # Routes associated with this port.
6472024-02-22T18:26:00.225Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
6482024-02-22T18:26:00.225Z+routes = [{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}]
6492024-02-22T18:26:00.225Z # Addresses associated with this port.
6502024-02-22T18:26:00.225Z-addresses = ["192.168.1.30/24"]
6512024-02-22T18:26:00.225Z+addresses = ["10.151.1.174/24"]
6522024-02-22T18:26:00.225Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
6532024-02-22T18:26:00.225Z port = "qsfp0"
6542024-02-22T18:26:00.225Z # The speed of this port.
6552024-02-22T18:26:00.225Z+ true
6562024-02-22T18:26:00.225Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml
6572024-02-22T18:26:00.225Za pkg/config-rss.toml 7K
6582024-02-22T18:26:00.225Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~
6592024-02-22T18:26:00.251Z+ TLS_NAME=recovery.sys.oxide.test
6602024-02-22T18:26:00.251Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
6612024-02-22T18:26:00.978Z....+...+......+.+.....+.........+.......+..+.......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.....+......+.....+.............+.....+......+.+.....+....+..............+.+...+...........+.........+...+.......+....................+.+.....+.........+.+......+..........................+....+...+..+......+...................+...+..+...+......+.+..............+...............................+...+........+.......+..+.+............+......+......+........+.+..+....+......+...+.........+...+........+.+..............+.......+.........+...+...............+...+..............+....+..+...+.......+...+..+....+.....+........................+......+..........+.........+..............+.+......+.....+....+..............+......+....+..+.+........+.+......+...+..............+.......+.....+....+..+...+....+..+......+...................+......+........+.......+..............+...+.+....................+......+.............+.....+...+.............+..............+......+.+......+...+.....+.+.....+...+.............+...............+...+.....................+...+.....+.........................+...+...............+........+............+...+............+.......+............+.....+...................+......+.....+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6622024-02-22T18:26:02.547Z.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+.....+..........+......+.....+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+.........+...+.........................+.....+............+..........+......+..+...+..........+..............+.+......+.....+......+....+.....+.+...............+...............+....................+...+.+......+...+......+.....................+............+..+.+.....+.............+.................+....+.........+..+............+......+....+............+......+........+.......+.........+......+..............+...+................+............+..+.+..+...+.+........+.+..............+.......+.....+...+...+.......+..+................+.........+...+.....+.+.....+....+...+.....+...+....+......+.....+..........+.....+......+.........+.+...+..+......+...+................+.......................+......+.......+...+......+.....+...+......+................+.....+......+.......+............+...+......+............+........+...............+.........+.+............+....................+..........+...+........................+............+...+..+....+.....+......+.+...+..+...+............+......................+..+...+...+.........................+..+.+..+....+...+.....+...............+......+.+..............+.+....................+...+..................+.......+...+..+...+.............+.....+.+..............+............+....+.....+.+...+..+...+..........+.........+..+......+....+......+.......................+..........+.....+.......+.....+..................+......+....+.................+...+...................+..+...................+...........+.........+.......+.....+.......+......+......+........+.+.....+.+...+...............+......+..........................+......+..........+...+............+.........+.....+....+.....+.+...+...............+..+.........+.+......+............+..+.........+.......+............+..+...+.+....................+.+...........+............+.+...............+......+...+..+......+.......+..+.+........+....+..+.+.....+...............+....+........+......+...............+.+......+............+.....+.......+..............+.......+..+...+...................+...............+.........+...........+...+...+.......+.....+.......+.....+.+..............+....+.........+........+..........+..+.+.........+..+...+.+............+.................+............+......+..........+............+............+......+..+...+...................+......+...+............+................................+.+......+.....+......+..........+..+...+.........+................+........+.+...+...+..+.......+...+...+.....+............+.+..+.........................+..+............+.+...+............+..+.+.....+.......+.....+.......+...........+....+.....+.+..............+..........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6632024-02-22T18:26:02.573Z-----
6642024-02-22T18:26:02.573Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6652024-02-22T18:26:02.599Za pkg/initial-tls-cert.pem 2K
6662024-02-22T18:26:02.599Za pkg/initial-tls-key.pem 4K
6672024-02-22T18:26:02.599Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6682024-02-22T18:26:02.599Z+ rmdir pkg
6692024-02-22T18:26:02.599Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
6702024-02-22T18:26:02.599Z+ pfexec mkdir -p /usr/oxide
6712024-02-22T18:26:02.599Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0
6722024-02-22T18:26:03.336Z+ pfexec chmod +x /usr/oxide/oxide
6732024-02-22T18:26:03.362Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
6742024-02-22T18:26:21.628Z+ OMICRON_NO_UNINSTALL=1
6752024-02-22T18:26:21.628Z+ ptime -m pfexec ./target/release/omicron-package -t test install
6762024-02-22T18:26:21.657ZLogging to: /opt/oxide/work/out/LOG
6772024-02-22T18:26:22.149Z
6782024-02-22T18:26:22.174Zreal 0.515319190
6792024-02-22T18:26:22.174Zuser 0.110779166
6802024-02-22T18:26:22.174Zsys 2.150338705
6812024-02-22T18:26:22.174Ztrap 0.000008959
6822024-02-22T18:26:22.174Ztflt 0.000031638
6832024-02-22T18:26:22.175Zdflt 0.000913486
6842024-02-22T18:26:22.175Zkflt 0.000007200
6852024-02-22T18:26:22.175Zlock 46.737204193
6862024-02-22T18:26:22.175Zslp 0.724875138
6872024-02-22T18:26:22.175Zlat 0.003070214
6882024-02-22T18:26:22.175Zstop 0.197367519
6892024-02-22T18:26:22.175Z+ retry=0
6902024-02-22T18:26:22.175Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
6912024-02-22T18:26:22.175Z+ [[ 0 -gt 30 ]]
6922024-02-22T18:26:22.175Z+ sleep 1
6932024-02-22T18:26:23.163Z+ retry=1
6942024-02-22T18:26:23.163Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
6952024-02-22T18:26:23.188Z+ [[ 1 -gt 30 ]]
6962024-02-22T18:26:23.188Z+ sleep 1
6972024-02-22T18:26:24.178Z+ retry=2
6982024-02-22T18:26:24.178Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
6992024-02-22T18:26:24.203Z+ [[ 2 -gt 30 ]]
7002024-02-22T18:26:24.203Z+ sleep 1
7012024-02-22T18:26:25.193Z+ retry=3
7022024-02-22T18:26:25.193Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7032024-02-22T18:26:25.218Z+ [[ 3 -gt 30 ]]
7042024-02-22T18:26:25.218Z+ sleep 1
7052024-02-22T18:26:26.206Z+ retry=4
7062024-02-22T18:26:26.207Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7072024-02-22T18:26:51.382Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
7082024-02-22T18:26:51.410Zlocal v6:
7092024-02-22T18:26:51.410Zfe80::c805:13ff:fe91:c524
7102024-02-22T18:26:51.410Zfd7b:6a7b:bad6:7911::1
7112024-02-22T18:26:51.410Zlocal v4:
7122024-02-22T18:26:51.410Z10.151.1.174
7132024-02-22T18:26:51.411Zrouter v6:
7142024-02-22T18:26:51.411Zfdb0:18c0:4d0d:9fb2::/64 -> fe80::f4f7:c1ff:feb1:3a7c (1)
7152024-02-22T18:26:51.411Zfd00:1122:3344:1::/64 -> fe80::f4f7:c1ff:feb1:3a7c (1)
7162024-02-22T18:26:51.411Zfd00:1122:3344:101::/64 -> fe80::f4f7:c1ff:feb1:3a7c (1)
7172024-02-22T18:26:51.411Zfd00:1122:3344:3::/64 -> fe80::f4f7:c1ff:feb1:3a7c (1)
7182024-02-22T18:26:51.411Zfd00:1122:3344:2::/64 -> fe80::f4f7:c1ff:feb1:3a7c (1)
7192024-02-22T18:26:51.411Zrouter v4_idx:
7202024-02-22T18:26:51.411Z0.0.0.0/0 -> 0
7212024-02-22T18:26:51.411Zrouter v4_routes:
7222024-02-22T18:26:51.411Z0 -> 10.151.1.100 (2)
7232024-02-22T18:26:51.411Zresolver v4:
7242024-02-22T18:26:51.411Z10.151.1.174 -> e6:98:06:05:d0:75
7252024-02-22T18:26:51.411Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
7262024-02-22T18:26:51.411Zresolver v6:
7272024-02-22T18:26:51.411Zfe80::c805:13ff:fe91:c524 -> ca:05:13:91:c5:24
7282024-02-22T18:26:51.411Zfe80::f4f7:c1ff:feb1:3a7c -> f6:f7:c1:b1:3a:7c
7292024-02-22T18:26:51.411Znat v4:
7302024-02-22T18:26:51.411Z10.151.1.169 0/16383 -> fd00:1122:3344:101::1 100/a8:40:25:ff:b9:86
7312024-02-22T18:26:51.411Znat_v6:
7322024-02-22T18:26:51.411Zport_mac:
7332024-02-22T18:26:51.411Z1000: a8:40:25:1a:e4:c9
7342024-02-22T18:26:51.436Z1: a8:40:25:1a:e4:ca
7352024-02-22T18:26:51.437Z2: a8:40:25:1a:e4:ea
7362024-02-22T18:26:51.437Zproxy arp:
7372024-02-22T18:26:51.437Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
7382024-02-22T18:26:51.437Z+ retry=0
7392024-02-22T18:26:51.437Z+++ zoneadm list -n
7402024-02-22T18:26:51.437Z+++ grep oxz_ntp
7412024-02-22T18:26:51.437Z++ pfexec svcs -z oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584 -Hostate oxide/ntp
7422024-02-22T18:26:51.462Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7432024-02-22T18:26:51.462Z++ true
7442024-02-22T18:26:51.462Z+ [[ '' != online ]]
7452024-02-22T18:26:51.462Z+ [[ 0 -gt 60 ]]
7462024-02-22T18:26:51.462Z+ sleep 1
7472024-02-22T18:26:52.446Z+ retry=1
7482024-02-22T18:26:52.472Z+++ zoneadm list -n
7492024-02-22T18:26:52.472Z+++ grep oxz_ntp
7502024-02-22T18:26:52.472Z++ pfexec svcs -z oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584 -Hostate oxide/ntp
7512024-02-22T18:26:52.497Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7522024-02-22T18:26:52.497Z++ true
7532024-02-22T18:26:52.497Z+ [[ '' != online ]]
7542024-02-22T18:26:52.497Z+ [[ 1 -gt 60 ]]
7552024-02-22T18:26:52.497Z+ sleep 1
7562024-02-22T18:26:53.476Z+ retry=2
7572024-02-22T18:26:53.501Z+++ zoneadm list -n
7582024-02-22T18:26:53.501Z+++ grep oxz_ntp
7592024-02-22T18:26:53.501Z++ pfexec svcs -z oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584 -Hostate oxide/ntp
7602024-02-22T18:26:53.501Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7612024-02-22T18:26:53.527Z++ true
7622024-02-22T18:26:53.527Z+ [[ '' != online ]]
7632024-02-22T18:26:53.527Z+ [[ 2 -gt 60 ]]
7642024-02-22T18:26:53.527Z+ sleep 1
7652024-02-22T18:26:54.506Z+ retry=3
7662024-02-22T18:26:54.531Z+++ zoneadm list -n
7672024-02-22T18:26:54.531Z+++ grep oxz_ntp
7682024-02-22T18:26:54.531Z++ pfexec svcs -z oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584 -Hostate oxide/ntp
7692024-02-22T18:26:54.531Z+ [[ disabled != online ]]
7702024-02-22T18:26:54.531Z+ [[ 3 -gt 60 ]]
7712024-02-22T18:26:54.531Z+ sleep 1
7722024-02-22T18:26:55.530Z+ retry=4
7732024-02-22T18:26:55.555Z+++ zoneadm list -n
7742024-02-22T18:26:55.555Z+++ grep oxz_ntp
7752024-02-22T18:26:55.555Z++ pfexec svcs -z oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584 -Hostate oxide/ntp
7762024-02-22T18:26:55.555Z+ [[ disabled != online ]]
7772024-02-22T18:26:55.555Z+ [[ 4 -gt 60 ]]
7782024-02-22T18:26:55.555Z+ sleep 1
7792024-02-22T18:26:56.552Z+ retry=5
7802024-02-22T18:26:56.577Z+++ zoneadm list -n
7812024-02-22T18:26:56.577Z+++ grep oxz_ntp
7822024-02-22T18:26:56.577Z++ pfexec svcs -z oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584 -Hostate oxide/ntp
7832024-02-22T18:26:56.577Z+ [[ disabled != online ]]
7842024-02-22T18:26:56.577Z+ [[ 5 -gt 60 ]]
7852024-02-22T18:26:56.577Z+ sleep 1
7862024-02-22T18:26:57.574Z+ retry=6
7872024-02-22T18:26:57.599Z+++ zoneadm list -n
7882024-02-22T18:26:57.600Z+++ grep oxz_ntp
7892024-02-22T18:26:57.600Z++ pfexec svcs -z oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584 -Hostate oxide/ntp
7902024-02-22T18:26:57.600Z+ [[ disabled != online ]]
7912024-02-22T18:26:57.600Z+ [[ 6 -gt 60 ]]
7922024-02-22T18:26:57.600Z+ sleep 1
7932024-02-22T18:26:58.595Z+ retry=7
7942024-02-22T18:26:58.620Z+++ zoneadm list -n
7952024-02-22T18:26:58.620Z+++ grep oxz_ntp
7962024-02-22T18:26:58.620Z++ pfexec svcs -z oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584 -Hostate oxide/ntp
7972024-02-22T18:26:58.620Z+ [[ disabled != online ]]
7982024-02-22T18:26:58.620Z+ [[ 7 -gt 60 ]]
7992024-02-22T18:26:58.620Z+ sleep 1
8002024-02-22T18:26:59.617Z+ retry=8
8012024-02-22T18:26:59.642Z+++ zoneadm list -n
8022024-02-22T18:26:59.642Z+++ grep oxz_ntp
8032024-02-22T18:26:59.642Z++ pfexec svcs -z oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584 -Hostate oxide/ntp
8042024-02-22T18:26:59.642Z+ [[ disabled != online ]]
8052024-02-22T18:26:59.642Z+ [[ 8 -gt 60 ]]
8062024-02-22T18:26:59.642Z+ sleep 1
 
8072024-02-22T18:27:00.534Zbuskin console login: Feb 22 18:27:00 buskin ip: ip: joining multicasts failed (4) on vopte0 - will use link layer broadcasts for multicast
 
8082024-02-22T18:27:00.640Z+ retry=9
8092024-02-22T18:27:00.665Z+++ zoneadm list -n
8102024-02-22T18:27:00.665Z+++ grep oxz_ntp
8112024-02-22T18:27:00.665Z++ pfexec svcs -z oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584 -Hostate oxide/ntp
8122024-02-22T18:27:00.665Z+ [[ online != online ]]
8132024-02-22T18:27:00.665Z+ echo 'Waited for chrony: 9s'
8142024-02-22T18:27:00.665ZWaited for chrony: 9s
8152024-02-22T18:27:00.665Z+ export RUST_BACKTRACE=1
8162024-02-22T18:27:00.665Z+ RUST_BACKTRACE=1
8172024-02-22T18:27:00.665Z+ export E2E_TLS_CERT IPPOOL_START IPPOOL_END
8182024-02-22T18:27:00.665Z++ ./tests/bootstrap
8192024-02-22T18:27:00.690ZUsing external DNS server at 10.151.1.164:53
8202024-02-22T18:27:00.715Z2024-02-22 18:27:01.240592549 UTC: attempting to log into API
8212024-02-22T18:27:15.732Z2024-02-22 18:27:15.349168868 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8222024-02-22T18:27:16.730Z2024-02-22 18:27:16.350145358 UTC: attempting to log into API
8232024-02-22T18:27:31.763Z2024-02-22 18:27:31.382272135 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8242024-02-22T18:27:32.764Z2024-02-22 18:27:32.383419508 UTC: attempting to log into API
8252024-02-22T18:27:47.796Z2024-02-22 18:27:47.416151426 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8262024-02-22T18:27:48.797Z2024-02-22 18:27:48.417283830 UTC: attempting to log into API
8272024-02-22T18:28:03.830Z2024-02-22 18:28:03.448856893 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8282024-02-22T18:28:04.830Z2024-02-22 18:28:04.449846004 UTC: attempting to log into API
8292024-02-22T18:28:19.862Z2024-02-22 18:28:19.504139825 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8302024-02-22T18:28:20.863Z2024-02-22 18:28:20.507877536 UTC: attempting to log into API
8312024-02-22T18:28:35.895Z2024-02-22 18:28:35.581736491 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8322024-02-22T18:28:36.895Z2024-02-22 18:28:36.585466012 UTC: attempting to log into API
8332024-02-22T18:28:51.927Z2024-02-22 18:28:51.658805201 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8342024-02-22T18:28:52.928Z2024-02-22 18:28:52.662536955 UTC: attempting to log into API
8352024-02-22T18:29:07.959Z2024-02-22 18:29:07.735999076 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8362024-02-22T18:29:08.961Z2024-02-22 18:29:08.739903804 UTC: attempting to log into API
8372024-02-22T18:29:23.992Z2024-02-22 18:29:23.813708160 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8382024-02-22T18:29:24.993Z2024-02-22 18:29:24.817428162 UTC: attempting to log into API
8392024-02-22T18:29:40.025Z2024-02-22 18:29:39.890286727 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8402024-02-22T18:29:41.026Z2024-02-22 18:29:40.893963704 UTC: attempting to log into API
8412024-02-22T18:29:56.057Z2024-02-22 18:29:55.965905047 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8422024-02-22T18:29:57.058Z2024-02-22 18:29:56.969550803 UTC: attempting to log into API
8432024-02-22T18:30:12.089Z2024-02-22 18:30:12.042050115 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8442024-02-22T18:30:13.091Z2024-02-22 18:30:13.045667533 UTC: attempting to log into API
8452024-02-22T18:30:28.122Z2024-02-22 18:30:28.112473400 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8462024-02-22T18:30:29.124Z2024-02-22 18:30:29.113626764 UTC: attempting to log into API
8472024-02-22T18:30:44.156Z2024-02-22 18:30:44.146091536 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8482024-02-22T18:30:45.158Z2024-02-22 18:30:45.148073546 UTC: attempting to log into API
8492024-02-22T18:31:00.196Z2024-02-22 18:31:00.184741592 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8502024-02-22T18:31:01.197Z2024-02-22 18:31:01.185335215 UTC: attempting to log into API
8512024-02-22T18:31:16.239Z2024-02-22 18:31:16.228300512 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8522024-02-22T18:31:17.240Z2024-02-22 18:31:17.229089778 UTC: attempting to log into API
8532024-02-22T18:31:32.272Z2024-02-22 18:31:32.261632872 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8542024-02-22T18:31:33.273Z2024-02-22 18:31:33.262718734 UTC: attempting to log into API
 
8552024-02-22T18:31:34.915ZFeb 22 18:31:34 buskin ip: ip: joining multicasts failed (4) on vopte2 - will use link layer broadcasts for multicast
8562024-02-22T18:31:41.954ZFeb 22 18:31:41 buskin ip: ip: joining multicasts failed (4) on vopte1 - will use link layer broadcasts for multicast
 
8572024-02-22T18:31:48.306Z2024-02-22 18:31:48.294613907 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8582024-02-22T18:31:49.306Z2024-02-22 18:31:49.294942064 UTC: attempting to log into API
 
8592024-02-22T18:31:49.583ZFeb 22 18:31:49 buskin ip: ip: joining multicasts failed (4) on vopte3 - will use link layer broadcasts for multicast
8602024-02-22T18:31:56.325ZFeb 22 18:31:55 buskin ip: ip: joining multicasts failed (4) on vopte4 - will use link layer broadcasts for multicast
8612024-02-22T18:32:01.913ZFeb 22 18:32:01 buskin ip: ip: joining multicasts failed (4) on vopte5 - will use link layer broadcasts for multicast
 
8622024-02-22T18:32:04.340Z2024-02-22 18:32:04.328736536 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8632024-02-22T18:32:05.342Z2024-02-22 18:32:05.330732206 UTC: attempting to log into API
8642024-02-22T18:32:20.373Z2024-02-22 18:32:20.361962576 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8652024-02-22T18:32:21.374Z2024-02-22 18:32:21.363047098 UTC: attempting to log into API
8662024-02-22T18:32:27.174Z2024-02-22 18:32:27.162843803 UTC: login succeeded
8672024-02-22T18:32:27.208Zwaiting for nexus to come up...
8682024-02-22T18:32:27.274Zcreating IP pool... 10.151.1.175 - 10.151.1.227
8692024-02-22T18:32:27.451Zsetting up quotas...
8702024-02-22T18:32:27.494Zensuring datasets are ready...
8712024-02-22T18:32:29.304Zlet's roll.
8722024-02-22T18:32:29.330Z+ eval 'OXIDE_HOST=https://recovery.sys.oxide.test
8732024-02-22T18:32:29.330ZOXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
8742024-02-22T18:32:29.330ZOXIDE_TOKEN=oxide-token-8ff8287aeeca5cbd593185332e77209e2a8473cd'
8752024-02-22T18:32:29.330Z++ OXIDE_HOST=https://recovery.sys.oxide.test
8762024-02-22T18:32:29.330Z++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
8772024-02-22T18:32:29.330Z++ OXIDE_TOKEN=oxide-token-8ff8287aeeca5cbd593185332e77209e2a8473cd
8782024-02-22T18:32:29.330Z+ export OXIDE_HOST OXIDE_TOKEN
8792024-02-22T18:32:29.330Z+ retry=0
8802024-02-22T18:32:29.330Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8812024-02-22T18:32:29.405Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images'
8822024-02-22T18:32:29.675Z{"status":"ok"}success
8832024-02-22T18:32:29.675ZProject {
8842024-02-22T18:32:29.675Z description: "some images",
8852024-02-22T18:32:29.675Z id: 99e9f0e5-fb0d-46b4-a7c2-119e718f4169,
8862024-02-22T18:32:29.675Z name: Name(
8872024-02-22T18:32:29.675Z "images",
8882024-02-22T18:32:29.675Z ),
8892024-02-22T18:32:29.700Z time_created: 2024-02-22T18:32:29.465243Z,
8902024-02-22T18:32:29.700Z time_modified: 2024-02-22T18:32:29.465243Z,
8912024-02-22T18:32:29.700Z}
8922024-02-22T18:32:29.700Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11
8932024-02-22T18:37:52.682Zdone!
8942024-02-22T18:37:52.780Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11
8952024-02-22T18:37:52.887Zsuccess
8962024-02-22T18:37:52.887ZImage {
8972024-02-22T18:37:52.887Z block_size: ByteCount(
8982024-02-22T18:37:52.887Z 512,
8992024-02-22T18:37:52.887Z ),
9002024-02-22T18:37:52.912Z description: "debian 11 original base image",
9012024-02-22T18:37:52.912Z digest: None,
9022024-02-22T18:37:52.912Z id: 9914f8dd-3e4b-448f-8396-bffab501d564,
9032024-02-22T18:37:52.912Z name: Name(
9042024-02-22T18:37:52.912Z "debian11",
9052024-02-22T18:37:52.912Z ),
9062024-02-22T18:37:52.912Z os: "debian",
9072024-02-22T18:37:52.912Z project_id: None,
9082024-02-22T18:37:52.912Z size: ByteCount(
9092024-02-22T18:37:52.912Z 2147483648,
9102024-02-22T18:37:52.912Z ),
9112024-02-22T18:37:52.912Z time_created: 2024-02-22T18:37:52.643871Z,
9122024-02-22T18:37:52.912Z time_modified: 2024-02-22T18:37:52.866107Z,
9132024-02-22T18:37:52.912Z url: None,
9142024-02-22T18:37:52.912Z version: "11",
9152024-02-22T18:37:52.912Z}
9162024-02-22T18:37:52.912Z+ rm ./tests/bootstrap
9172024-02-22T18:37:52.965Z+ for test_bin in tests/*
9182024-02-22T18:37:52.965Z+ ./tests/bootstrap-22e5a97eec9bb5c6
9192024-02-22T18:37:53.022Z
9202024-02-22T18:37:53.022Zrunning 0 tests
9212024-02-22T18:37:53.022Z
9222024-02-22T18:37:53.046Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
9232024-02-22T18:37:53.046Z
9242024-02-22T18:37:53.046Z+ for test_bin in tests/*
9252024-02-22T18:37:53.046Z+ ./tests/end_to_end_tests-25e7d53a8e782c84
9262024-02-22T18:37:53.188Z
9272024-02-22T18:37:53.188Zrunning 2 tests
9282024-02-22T18:37:53.448Ztest no_spoof::no_spoof_here ... ok
9292024-02-22T18:38:26.262Ztest instance_launch::instance_launch ... ok
9302024-02-22T18:38:26.262Z
9312024-02-22T18:38:26.262Ztest result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 33.07s
9322024-02-22T18:38:26.262Z
9332024-02-22T18:38:26.293Z+ _exit_trap
9342024-02-22T18:38:26.293Z+ local status=0
9352024-02-22T18:38:26.293Z+ [[ 0 -eq 0 ]]
9362024-02-22T18:38:26.293Z+ exit 0
9372024-02-22T18:38:26.324Zprocess exited: duration 803929 ms, exit code 0
 
9382024-02-22T18:38:26.445Zfound 97 output files
9392024-02-22T18:38:26.445Zuploading: /var/svc/log/oxide-sled-agent:default.log (1392024 bytes)
9402024-02-22T18:38:28.193Zuploaded: /var/svc/log/oxide-sled-agent:default.log
9412024-02-22T18:38:28.193Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log (2120839 bytes)
9422024-02-22T18:38:28.968Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log" changed size mid upload: 2120839 -> 2131626
9432024-02-22T18:38:30.039Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log
9442024-02-22T18:38:30.039Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log.0 (867196 bytes)
9452024-02-22T18:38:31.774Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log.0
9462024-02-22T18:38:31.774Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log (3427 bytes)
9472024-02-22T18:38:32.882Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log
9482024-02-22T18:38:32.882Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log.0 (4447 bytes)
9492024-02-22T18:38:33.976Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log.0
9502024-02-22T18:38:34.002Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log (15336 bytes)
9512024-02-22T18:38:34.067Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log" changed size mid upload: 15336 -> 16376
9522024-02-22T18:38:35.137Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log
9532024-02-22T18:38:35.138Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log.0 (8071 bytes)
9542024-02-22T18:38:36.255Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log.0
9552024-02-22T18:38:36.255Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log (1320911 bytes)
9562024-02-22T18:38:36.988Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log" changed size mid upload: 1320911 -> 1340082
9572024-02-22T18:38:38.046Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log
9582024-02-22T18:38:38.046Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log.0 (423203 bytes)
9592024-02-22T18:38:39.317Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log.0
9602024-02-22T18:38:39.317Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log (778932 bytes)
9612024-02-22T18:38:39.999Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log" changed size mid upload: 778932 -> 792796
9622024-02-22T18:38:41.069Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log
9632024-02-22T18:38:41.069Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log.0 (252903 bytes)
9642024-02-22T18:38:42.318Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log.0
9652024-02-22T18:38:42.318Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log (0 bytes)
9662024-02-22T18:38:43.390Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log
9672024-02-22T18:38:43.390Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log.0 (1388 bytes)
9682024-02-22T18:38:44.483Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log.0
9692024-02-22T18:38:44.483Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log (248806 bytes)
9702024-02-22T18:38:44.683Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log" changed size mid upload: 248806 -> 257885
9712024-02-22T18:38:45.754Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log
9722024-02-22T18:38:45.754Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log.0 (94750 bytes)
9732024-02-22T18:38:46.959Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log.0
9742024-02-22T18:38:46.959Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log (0 bytes)
9752024-02-22T18:38:48.032Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log
9762024-02-22T18:38:48.032Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log.0 (617 bytes)
9772024-02-22T18:38:49.124Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log.0
9782024-02-22T18:38:49.124Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log (0 bytes)
9792024-02-22T18:38:50.193Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log
9802024-02-22T18:38:50.221Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log.0 (5393 bytes)
9812024-02-22T18:38:51.287Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log.0
9822024-02-22T18:38:51.287Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_e47d17c7-5aec-4822-8f3e-34dce6fb230f/root/var/svc/log/oxide-clickhouse:default.log (1111989 bytes)
9832024-02-22T18:38:52.006Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_e47d17c7-5aec-4822-8f3e-34dce6fb230f/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 1111989 -> 1185009
9842024-02-22T18:38:53.070Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_e47d17c7-5aec-4822-8f3e-34dce6fb230f/root/var/svc/log/oxide-clickhouse:default.log
9852024-02-22T18:38:53.070Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_e47d17c7-5aec-4822-8f3e-34dce6fb230f/root/var/svc/log/oxide-zone-network-setup:default.log (1709 bytes)
9862024-02-22T18:38:54.163Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_e47d17c7-5aec-4822-8f3e-34dce6fb230f/root/var/svc/log/oxide-zone-network-setup:default.log
9872024-02-22T18:38:54.163Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_dd447aaf-fe87-4027-b7f7-c4967ee94cf8/root/var/svc/log/oxide-cockroachdb:default.log (4408 bytes)
9882024-02-22T18:38:55.257Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_dd447aaf-fe87-4027-b7f7-c4967ee94cf8/root/var/svc/log/oxide-cockroachdb:default.log
9892024-02-22T18:38:55.283Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_dd447aaf-fe87-4027-b7f7-c4967ee94cf8/root/var/svc/log/oxide-zone-network-setup:default.log (1711 bytes)
9902024-02-22T18:38:56.359Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_dd447aaf-fe87-4027-b7f7-c4967ee94cf8/root/var/svc/log/oxide-zone-network-setup:default.log
9912024-02-22T18:38:56.359Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_d64def60-1829-4c6d-9f91-15ca4b937ea0/root/var/svc/log/oxide-crucible-agent:default.log (12045 bytes)
9922024-02-22T18:38:57.475Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_d64def60-1829-4c6d-9f91-15ca4b937ea0/root/var/svc/log/oxide-crucible-agent:default.log
9932024-02-22T18:38:57.475Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_d64def60-1829-4c6d-9f91-15ca4b937ea0/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
9942024-02-22T18:38:58.568Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_d64def60-1829-4c6d-9f91-15ca4b937ea0/root/var/svc/log/oxide-crucible-downstairs:default.log
9952024-02-22T18:38:58.568Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_d64def60-1829-4c6d-9f91-15ca4b937ea0/root/var/svc/log/oxide-crucible-downstairs:downstairs-57bc9c39-a26f-43ac-9b01-f66dfc51aebb.log (5572 bytes)
9962024-02-22T18:38:59.665Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_d64def60-1829-4c6d-9f91-15ca4b937ea0/root/var/svc/log/oxide-crucible-downstairs:downstairs-57bc9c39-a26f-43ac-9b01-f66dfc51aebb.log
9972024-02-22T18:38:59.665Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_d64def60-1829-4c6d-9f91-15ca4b937ea0/root/var/svc/log/oxide-zone-network-setup:default.log (1699 bytes)
9982024-02-22T18:39:00.762Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_d64def60-1829-4c6d-9f91-15ca4b937ea0/root/var/svc/log/oxide-zone-network-setup:default.log
9992024-02-22T18:39:00.762Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_5405fcbf-06d0-4b1d-88ec-62cbf943a734/root/var/svc/log/oxide-crucible-pantry:default.log (780 bytes)
10002024-02-22T18:39:01.856Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_5405fcbf-06d0-4b1d-88ec-62cbf943a734/root/var/svc/log/oxide-crucible-pantry:default.log
10012024-02-22T18:39:01.882Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_5405fcbf-06d0-4b1d-88ec-62cbf943a734/root/var/svc/log/oxide-zone-network-setup:default.log (1739 bytes)
10022024-02-22T18:39:02.951Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_5405fcbf-06d0-4b1d-88ec-62cbf943a734/root/var/svc/log/oxide-zone-network-setup:default.log
10032024-02-22T18:39:02.951Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_70b0f066-1d10-4538-8efa-288a528930f5/root/var/svc/log/oxide-external_dns:default.log (26941 bytes)
10042024-02-22T18:39:03.065Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_70b0f066-1d10-4538-8efa-288a528930f5/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 26941 -> 30718
10052024-02-22T18:39:04.136Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_70b0f066-1d10-4538-8efa-288a528930f5/root/var/svc/log/oxide-external_dns:default.log
10062024-02-22T18:39:04.136Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_e60c8974-fc87-4dd2-a5c4-1a54c8a93761/root/var/svc/log/oxide-internal_dns:default.log (103166 bytes)
10072024-02-22T18:39:04.434Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_e60c8974-fc87-4dd2-a5c4-1a54c8a93761/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 103166 -> 106918
10082024-02-22T18:39:05.504Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_e60c8974-fc87-4dd2-a5c4-1a54c8a93761/root/var/svc/log/oxide-internal_dns:default.log
10092024-02-22T18:39:05.529Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_e60c8974-fc87-4dd2-a5c4-1a54c8a93761/root/var/svc/log/oxide-internal_dns:default.log.0 (6322 bytes)
10102024-02-22T18:39:06.600Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_e60c8974-fc87-4dd2-a5c4-1a54c8a93761/root/var/svc/log/oxide-internal_dns:default.log.0
10112024-02-22T18:39:06.600Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_12ff4efd-cccb-45ee-aa41-6436f8d6ec86/root/var/svc/log/oxide-nexus:default.log (1330478 bytes)
10122024-02-22T18:39:06.911Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_12ff4efd-cccb-45ee-aa41-6436f8d6ec86/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 1330478 -> 1442481
10132024-02-22T18:39:07.981Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_12ff4efd-cccb-45ee-aa41-6436f8d6ec86/root/var/svc/log/oxide-nexus:default.log
10142024-02-22T18:39:07.981Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_73923954-451c-445f-a817-40f310b2cc60/root/var/svc/log/oxide-cockroachdb:default.log (4407 bytes)
10152024-02-22T18:39:09.074Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_73923954-451c-445f-a817-40f310b2cc60/root/var/svc/log/oxide-cockroachdb:default.log
10162024-02-22T18:39:09.074Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_73923954-451c-445f-a817-40f310b2cc60/root/var/svc/log/oxide-zone-network-setup:default.log (1710 bytes)
10172024-02-22T18:39:10.168Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_73923954-451c-445f-a817-40f310b2cc60/root/var/svc/log/oxide-zone-network-setup:default.log
10182024-02-22T18:39:10.168Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_70a9b2fe-d892-4d91-8441-88f565073b2c/root/var/svc/log/oxide-crucible-agent:default.log (7682 bytes)
10192024-02-22T18:39:11.286Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_70a9b2fe-d892-4d91-8441-88f565073b2c/root/var/svc/log/oxide-crucible-agent:default.log
10202024-02-22T18:39:11.286Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_70a9b2fe-d892-4d91-8441-88f565073b2c/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10212024-02-22T18:39:12.382Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_70a9b2fe-d892-4d91-8441-88f565073b2c/root/var/svc/log/oxide-crucible-downstairs:default.log
10222024-02-22T18:39:12.382Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_70a9b2fe-d892-4d91-8441-88f565073b2c/root/var/svc/log/oxide-crucible-downstairs:downstairs-c308281a-ecfd-4086-9eb0-95ef86c2c736.log (2996 bytes)
10232024-02-22T18:39:13.473Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_70a9b2fe-d892-4d91-8441-88f565073b2c/root/var/svc/log/oxide-crucible-downstairs:downstairs-c308281a-ecfd-4086-9eb0-95ef86c2c736.log
10242024-02-22T18:39:13.498Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_70a9b2fe-d892-4d91-8441-88f565073b2c/root/var/svc/log/oxide-zone-network-setup:default.log (1699 bytes)
10252024-02-22T18:39:14.567Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_70a9b2fe-d892-4d91-8441-88f565073b2c/root/var/svc/log/oxide-zone-network-setup:default.log
10262024-02-22T18:39:14.567Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_09f88412-50cf-4570-869b-e162c58c2a7e/root/var/svc/log/oxide-external_dns:default.log (26936 bytes)
10272024-02-22T18:39:14.679Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_09f88412-50cf-4570-869b-e162c58c2a7e/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 26936 -> 30714
10282024-02-22T18:39:15.748Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_09f88412-50cf-4570-869b-e162c58c2a7e/root/var/svc/log/oxide-external_dns:default.log
10292024-02-22T18:39:15.748Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_bc493d1c-d5b6-4473-b2a2-9243eebfb063/root/var/svc/log/oxide-internal_dns:default.log (103153 bytes)
10302024-02-22T18:39:15.904Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_bc493d1c-d5b6-4473-b2a2-9243eebfb063/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 103153 -> 106908
10312024-02-22T18:39:16.973Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_bc493d1c-d5b6-4473-b2a2-9243eebfb063/root/var/svc/log/oxide-internal_dns:default.log
10322024-02-22T18:39:16.999Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_bc493d1c-d5b6-4473-b2a2-9243eebfb063/root/var/svc/log/oxide-internal_dns:default.log.0 (7528 bytes)
10332024-02-22T18:39:18.091Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_bc493d1c-d5b6-4473-b2a2-9243eebfb063/root/var/svc/log/oxide-internal_dns:default.log.0
10342024-02-22T18:39:18.091Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_2afcf605-31d2-4146-a47c-2f4ae0da41e6/root/var/svc/log/oxide-cockroachdb:default.log (4381 bytes)
10352024-02-22T18:39:19.185Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_2afcf605-31d2-4146-a47c-2f4ae0da41e6/root/var/svc/log/oxide-cockroachdb:default.log
10362024-02-22T18:39:19.185Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_2afcf605-31d2-4146-a47c-2f4ae0da41e6/root/var/svc/log/oxide-zone-network-setup:default.log (1711 bytes)
10372024-02-22T18:39:20.278Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_2afcf605-31d2-4146-a47c-2f4ae0da41e6/root/var/svc/log/oxide-zone-network-setup:default.log
10382024-02-22T18:39:20.278Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ef16a720-db26-4a30-846c-cf2465a5cd3d/root/var/svc/log/oxide-crucible-agent:default.log (12009 bytes)
10392024-02-22T18:39:21.393Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ef16a720-db26-4a30-846c-cf2465a5cd3d/root/var/svc/log/oxide-crucible-agent:default.log
10402024-02-22T18:39:21.393Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ef16a720-db26-4a30-846c-cf2465a5cd3d/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10412024-02-22T18:39:22.490Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ef16a720-db26-4a30-846c-cf2465a5cd3d/root/var/svc/log/oxide-crucible-downstairs:default.log
10422024-02-22T18:39:22.490Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ef16a720-db26-4a30-846c-cf2465a5cd3d/root/var/svc/log/oxide-crucible-downstairs:downstairs-6dda5d08-1811-468e-a105-4cbd5de23f66.log (5568 bytes)
10432024-02-22T18:39:23.583Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ef16a720-db26-4a30-846c-cf2465a5cd3d/root/var/svc/log/oxide-crucible-downstairs:downstairs-6dda5d08-1811-468e-a105-4cbd5de23f66.log
10442024-02-22T18:39:23.584Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ef16a720-db26-4a30-846c-cf2465a5cd3d/root/var/svc/log/oxide-zone-network-setup:default.log (1699 bytes)
10452024-02-22T18:39:24.677Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ef16a720-db26-4a30-846c-cf2465a5cd3d/root/var/svc/log/oxide-zone-network-setup:default.log
10462024-02-22T18:39:24.703Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_cefc02cf-3ecc-4cdc-9bf5-1804f77012a6/root/var/svc/log/oxide-internal_dns:default.log (103153 bytes)
10472024-02-22T18:39:24.834Zupload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_cefc02cf-3ecc-4cdc-9bf5-1804f77012a6/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 103153 -> 106909
10482024-02-22T18:39:25.906Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_cefc02cf-3ecc-4cdc-9bf5-1804f77012a6/root/var/svc/log/oxide-internal_dns:default.log
10492024-02-22T18:39:25.931Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_cefc02cf-3ecc-4cdc-9bf5-1804f77012a6/root/var/svc/log/oxide-internal_dns:default.log.0 (6001 bytes)
10502024-02-22T18:39:27.006Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_cefc02cf-3ecc-4cdc-9bf5-1804f77012a6/root/var/svc/log/oxide-internal_dns:default.log.0
10512024-02-22T18:39:27.006Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_def2d755-0762-4565-8b03-6a7e5f998ea0/root/var/svc/log/oxide-cockroachdb:default.log (4404 bytes)
10522024-02-22T18:39:28.095Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_def2d755-0762-4565-8b03-6a7e5f998ea0/root/var/svc/log/oxide-cockroachdb:default.log
10532024-02-22T18:39:28.095Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_def2d755-0762-4565-8b03-6a7e5f998ea0/root/var/svc/log/oxide-zone-network-setup:default.log (1711 bytes)
10542024-02-22T18:39:29.191Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_def2d755-0762-4565-8b03-6a7e5f998ea0/root/var/svc/log/oxide-zone-network-setup:default.log
10552024-02-22T18:39:29.191Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-crucible-agent:default.log (31194 bytes)
10562024-02-22T18:39:30.350Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-crucible-agent:default.log
10572024-02-22T18:39:30.350Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10582024-02-22T18:39:31.446Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-crucible-downstairs:default.log
10592024-02-22T18:39:31.470Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-crucible-downstairs:downstairs-3d7b769e-479a-4386-96ce-0f08d2587518.log (2996 bytes)
10602024-02-22T18:39:32.539Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-crucible-downstairs:downstairs-3d7b769e-479a-4386-96ce-0f08d2587518.log
10612024-02-22T18:39:32.539Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-crucible-downstairs:downstairs-8626f0ee-549f-4fad-9c87-482b61b90830.log (5697 bytes)
10622024-02-22T18:39:33.632Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-crucible-downstairs:downstairs-8626f0ee-549f-4fad-9c87-482b61b90830.log
10632024-02-22T18:39:33.633Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-crucible-downstairs:downstairs-c94843fe-a7b0-4b90-883d-d36016f76c26.log (5566 bytes)
10642024-02-22T18:39:34.724Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-crucible-downstairs:downstairs-c94843fe-a7b0-4b90-883d-d36016f76c26.log
10652024-02-22T18:39:34.725Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-crucible-downstairs:snapshot-8626f0ee-549f-4fad-9c87-482b61b90830-5d9df203-ce89-4f7f-9ce1-08b8316a4558.log (5264 bytes)
10662024-02-22T18:39:35.817Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-crucible-downstairs:snapshot-8626f0ee-549f-4fad-9c87-482b61b90830-5d9df203-ce89-4f7f-9ce1-08b8316a4558.log
10672024-02-22T18:39:35.842Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-zone-network-setup:default.log (1699 bytes)
10682024-02-22T18:39:36.910Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bb345b44-340b-4df2-aae1-2316d65631f9/root/var/svc/log/oxide-zone-network-setup:default.log
10692024-02-22T18:39:36.910Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_55e5dbbd-ab8e-48b2-8817-1fda42b9db23/root/var/svc/log/oxide-nexus:default.log (1187333 bytes)
10702024-02-22T18:39:37.638Zupload warning: file "/pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_55e5dbbd-ab8e-48b2-8817-1fda42b9db23/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 1187333 -> 1322808
10712024-02-22T18:39:38.710Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_55e5dbbd-ab8e-48b2-8817-1fda42b9db23/root/var/svc/log/oxide-nexus:default.log
10722024-02-22T18:39:38.710Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_88d71906-fd93-48ab-8fc5-4b0b36a9d1ff/root/var/svc/log/oxide-cockroachdb:default.log (4408 bytes)
10732024-02-22T18:39:39.807Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_88d71906-fd93-48ab-8fc5-4b0b36a9d1ff/root/var/svc/log/oxide-cockroachdb:default.log
10742024-02-22T18:39:39.807Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_88d71906-fd93-48ab-8fc5-4b0b36a9d1ff/root/var/svc/log/oxide-zone-network-setup:default.log (1711 bytes)
10752024-02-22T18:39:40.902Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_88d71906-fd93-48ab-8fc5-4b0b36a9d1ff/root/var/svc/log/oxide-zone-network-setup:default.log
10762024-02-22T18:39:40.902Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_eebe1b4f-0c34-4764-8d37-ddb62b8dc20f/root/var/svc/log/oxide-crucible-agent:default.log (18043 bytes)
10772024-02-22T18:39:42.040Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_eebe1b4f-0c34-4764-8d37-ddb62b8dc20f/root/var/svc/log/oxide-crucible-agent:default.log
10782024-02-22T18:39:42.040Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_eebe1b4f-0c34-4764-8d37-ddb62b8dc20f/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10792024-02-22T18:39:43.142Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_eebe1b4f-0c34-4764-8d37-ddb62b8dc20f/root/var/svc/log/oxide-crucible-downstairs:default.log
10802024-02-22T18:39:43.142Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_eebe1b4f-0c34-4764-8d37-ddb62b8dc20f/root/var/svc/log/oxide-crucible-downstairs:downstairs-9f852063-217e-476c-be0f-1cdbe52b6721.log (5695 bytes)
10812024-02-22T18:39:44.252Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_eebe1b4f-0c34-4764-8d37-ddb62b8dc20f/root/var/svc/log/oxide-crucible-downstairs:downstairs-9f852063-217e-476c-be0f-1cdbe52b6721.log
10822024-02-22T18:39:44.252Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_eebe1b4f-0c34-4764-8d37-ddb62b8dc20f/root/var/svc/log/oxide-crucible-downstairs:snapshot-9f852063-217e-476c-be0f-1cdbe52b6721-5d9df203-ce89-4f7f-9ce1-08b8316a4558.log (5262 bytes)
10832024-02-22T18:39:45.349Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_eebe1b4f-0c34-4764-8d37-ddb62b8dc20f/root/var/svc/log/oxide-crucible-downstairs:snapshot-9f852063-217e-476c-be0f-1cdbe52b6721-5d9df203-ce89-4f7f-9ce1-08b8316a4558.log
10842024-02-22T18:39:45.350Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_eebe1b4f-0c34-4764-8d37-ddb62b8dc20f/root/var/svc/log/oxide-zone-network-setup:default.log (1699 bytes)
10852024-02-22T18:39:46.448Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_eebe1b4f-0c34-4764-8d37-ddb62b8dc20f/root/var/svc/log/oxide-zone-network-setup:default.log
10862024-02-22T18:39:46.448Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_28a241a9-2ada-4e48-bacb-62ad53a2dd27/root/var/svc/log/oxide-crucible-agent:default.log (25264 bytes)
10872024-02-22T18:39:47.594Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_28a241a9-2ada-4e48-bacb-62ad53a2dd27/root/var/svc/log/oxide-crucible-agent:default.log
10882024-02-22T18:39:47.594Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_28a241a9-2ada-4e48-bacb-62ad53a2dd27/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10892024-02-22T18:39:48.693Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_28a241a9-2ada-4e48-bacb-62ad53a2dd27/root/var/svc/log/oxide-crucible-downstairs:default.log
10902024-02-22T18:39:48.693Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_28a241a9-2ada-4e48-bacb-62ad53a2dd27/root/var/svc/log/oxide-crucible-downstairs:downstairs-50289e55-97ca-4b80-9e1d-f7d128ecd0e1.log (3124 bytes)
10912024-02-22T18:39:49.792Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_28a241a9-2ada-4e48-bacb-62ad53a2dd27/root/var/svc/log/oxide-crucible-downstairs:downstairs-50289e55-97ca-4b80-9e1d-f7d128ecd0e1.log
10922024-02-22T18:39:49.792Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_28a241a9-2ada-4e48-bacb-62ad53a2dd27/root/var/svc/log/oxide-crucible-downstairs:downstairs-d125b52d-c7a4-4dfe-bd47-9d8cdc5137ac.log (5696 bytes)
10932024-02-22T18:39:50.894Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_28a241a9-2ada-4e48-bacb-62ad53a2dd27/root/var/svc/log/oxide-crucible-downstairs:downstairs-d125b52d-c7a4-4dfe-bd47-9d8cdc5137ac.log
10942024-02-22T18:39:50.918Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_28a241a9-2ada-4e48-bacb-62ad53a2dd27/root/var/svc/log/oxide-crucible-downstairs:snapshot-d125b52d-c7a4-4dfe-bd47-9d8cdc5137ac-5d9df203-ce89-4f7f-9ce1-08b8316a4558.log (5262 bytes)
10952024-02-22T18:39:52.005Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_28a241a9-2ada-4e48-bacb-62ad53a2dd27/root/var/svc/log/oxide-crucible-downstairs:snapshot-d125b52d-c7a4-4dfe-bd47-9d8cdc5137ac-5d9df203-ce89-4f7f-9ce1-08b8316a4558.log
10962024-02-22T18:39:52.005Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_28a241a9-2ada-4e48-bacb-62ad53a2dd27/root/var/svc/log/oxide-zone-network-setup:default.log (1699 bytes)
10972024-02-22T18:39:53.095Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_28a241a9-2ada-4e48-bacb-62ad53a2dd27/root/var/svc/log/oxide-zone-network-setup:default.log
10982024-02-22T18:39:53.120Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_pantry_0251fbc4-6983-484f-85e1-2a7a70335727/root/var/svc/log/oxide-crucible-pantry:default.log (1226176 bytes)
10992024-02-22T18:39:54.976Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_pantry_0251fbc4-6983-484f-85e1-2a7a70335727/root/var/svc/log/oxide-crucible-pantry:default.log
11002024-02-22T18:39:54.976Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_pantry_0251fbc4-6983-484f-85e1-2a7a70335727/root/var/svc/log/oxide-zone-network-setup:default.log (1741 bytes)
11012024-02-22T18:39:56.075Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_pantry_0251fbc4-6983-484f-85e1-2a7a70335727/root/var/svc/log/oxide-zone-network-setup:default.log
11022024-02-22T18:39:56.099Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d064259d-6a0d-4729-b5b6-b9843ea3ed15/root/var/svc/log/oxide-crucible-agent:default.log (7683 bytes)
11032024-02-22T18:39:57.196Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d064259d-6a0d-4729-b5b6-b9843ea3ed15/root/var/svc/log/oxide-crucible-agent:default.log
11042024-02-22T18:39:57.196Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d064259d-6a0d-4729-b5b6-b9843ea3ed15/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11052024-02-22T18:39:58.297Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d064259d-6a0d-4729-b5b6-b9843ea3ed15/root/var/svc/log/oxide-crucible-downstairs:default.log
11062024-02-22T18:39:58.297Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d064259d-6a0d-4729-b5b6-b9843ea3ed15/root/var/svc/log/oxide-crucible-downstairs:downstairs-b1c8f86a-8e52-4c2b-9c6f-8426e9101b71.log (2995 bytes)
11072024-02-22T18:39:59.396Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d064259d-6a0d-4729-b5b6-b9843ea3ed15/root/var/svc/log/oxide-crucible-downstairs:downstairs-b1c8f86a-8e52-4c2b-9c6f-8426e9101b71.log
11082024-02-22T18:39:59.396Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d064259d-6a0d-4729-b5b6-b9843ea3ed15/root/var/svc/log/oxide-zone-network-setup:default.log (1698 bytes)
11092024-02-22T18:40:00.496Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d064259d-6a0d-4729-b5b6-b9843ea3ed15/root/var/svc/log/oxide-zone-network-setup:default.log
11102024-02-22T18:40:00.520Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_nexus_0c45b733-3721-4bd0-83ed-d2d18dfc3be2/root/var/svc/log/oxide-nexus:default.log (38409433 bytes)
11112024-02-22T18:40:15.766Zupload warning: file "/pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_nexus_0c45b733-3721-4bd0-83ed-d2d18dfc3be2/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 38409433 -> 38983226
11122024-02-22T18:40:16.840Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_nexus_0c45b733-3721-4bd0-83ed-d2d18dfc3be2/root/var/svc/log/oxide-nexus:default.log
11132024-02-22T18:40:16.840Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8721d3e7-595c-4190-8708-03101668f10a/root/var/svc/log/oxide-crucible-agent:default.log (11576 bytes)
11142024-02-22T18:40:17.963Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8721d3e7-595c-4190-8708-03101668f10a/root/var/svc/log/oxide-crucible-agent:default.log
11152024-02-22T18:40:17.963Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8721d3e7-595c-4190-8708-03101668f10a/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11162024-02-22T18:40:19.063Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8721d3e7-595c-4190-8708-03101668f10a/root/var/svc/log/oxide-crucible-downstairs:default.log
11172024-02-22T18:40:19.087Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8721d3e7-595c-4190-8708-03101668f10a/root/var/svc/log/oxide-crucible-downstairs:downstairs-705152e8-611d-4413-aeb0-b99956b77768.log (3125 bytes)
11182024-02-22T18:40:20.163Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8721d3e7-595c-4190-8708-03101668f10a/root/var/svc/log/oxide-crucible-downstairs:downstairs-705152e8-611d-4413-aeb0-b99956b77768.log
11192024-02-22T18:40:20.187Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8721d3e7-595c-4190-8708-03101668f10a/root/var/svc/log/oxide-zone-network-setup:default.log (1699 bytes)
11202024-02-22T18:40:21.265Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8721d3e7-595c-4190-8708-03101668f10a/root/var/svc/log/oxide-zone-network-setup:default.log
11212024-02-22T18:40:21.289Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f33222b2-bf73-4c12-839a-0ea712d6d279/root/var/svc/log/oxide-crucible-agent:default.log (12045 bytes)
11222024-02-22T18:40:22.390Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f33222b2-bf73-4c12-839a-0ea712d6d279/root/var/svc/log/oxide-crucible-agent:default.log
11232024-02-22T18:40:22.390Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f33222b2-bf73-4c12-839a-0ea712d6d279/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11242024-02-22T18:40:23.493Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f33222b2-bf73-4c12-839a-0ea712d6d279/root/var/svc/log/oxide-crucible-downstairs:default.log
11252024-02-22T18:40:23.493Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f33222b2-bf73-4c12-839a-0ea712d6d279/root/var/svc/log/oxide-crucible-downstairs:downstairs-4ce8d28c-ab81-4546-a39a-2193abe09ad3.log (3126 bytes)
11262024-02-22T18:40:24.594Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f33222b2-bf73-4c12-839a-0ea712d6d279/root/var/svc/log/oxide-crucible-downstairs:downstairs-4ce8d28c-ab81-4546-a39a-2193abe09ad3.log
11272024-02-22T18:40:24.619Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f33222b2-bf73-4c12-839a-0ea712d6d279/root/var/svc/log/oxide-zone-network-setup:default.log (1699 bytes)
11282024-02-22T18:40:25.694Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f33222b2-bf73-4c12-839a-0ea712d6d279/root/var/svc/log/oxide-zone-network-setup:default.log
11292024-02-22T18:40:25.718Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_7d81ce0b-ada1-4345-b061-8c636701eabd/root/var/svc/log/oxide-crucible-pantry:default.log (783 bytes)
11302024-02-22T18:40:26.793Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_7d81ce0b-ada1-4345-b061-8c636701eabd/root/var/svc/log/oxide-crucible-pantry:default.log
11312024-02-22T18:40:26.793Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_7d81ce0b-ada1-4345-b061-8c636701eabd/root/var/svc/log/oxide-zone-network-setup:default.log (1740 bytes)
11322024-02-22T18:40:27.892Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_7d81ce0b-ada1-4345-b061-8c636701eabd/root/var/svc/log/oxide-zone-network-setup:default.log
11332024-02-22T18:40:27.892Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/svc/log/oxide-ntp:default.log (67 bytes)
11342024-02-22T18:40:28.991Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/svc/log/oxide-ntp:default.log
11352024-02-22T18:40:28.991Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/svc/log/oxide-ntp:default.log.0 (1253 bytes)
11362024-02-22T18:40:30.091Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/svc/log/oxide-ntp:default.log.0
11372024-02-22T18:40:30.091Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_770af29a-66f6-4fd2-8b97-f605d3efcacc/root/var/svc/log/oxide-oximeter:default.log (300976 bytes)
11382024-02-22T18:40:30.552Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_770af29a-66f6-4fd2-8b97-f605d3efcacc/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 300976 -> 398233
11392024-02-22T18:40:31.630Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_770af29a-66f6-4fd2-8b97-f605d3efcacc/root/var/svc/log/oxide-oximeter:default.log
11402024-02-22T18:40:31.630Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_770af29a-66f6-4fd2-8b97-f605d3efcacc/root/var/svc/log/oxide-zone-network-setup:default.log (1697 bytes)
11412024-02-22T18:40:32.730Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_770af29a-66f6-4fd2-8b97-f605d3efcacc/root/var/svc/log/oxide-zone-network-setup:default.log
11422024-02-22T18:40:32.730Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/log/chrony/measurements.log (25071 bytes)
11432024-02-22T18:40:32.848Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/log/chrony/measurements.log" changed size mid upload: 25071 -> 29592
11442024-02-22T18:40:33.922Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/log/chrony/measurements.log
11452024-02-22T18:40:33.922Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/log/chrony/statistics.log (17316 bytes)
11462024-02-22T18:40:34.020Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/log/chrony/statistics.log" changed size mid upload: 17316 -> 20943
11472024-02-22T18:40:35.093Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/log/chrony/statistics.log
11482024-02-22T18:40:35.117Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/log/chrony/tracking.log (2381 bytes)
11492024-02-22T18:40:35.142Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/log/chrony/tracking.log" changed size mid upload: 2381 -> 2645
11502024-02-22T18:40:36.216Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_df46de24-e0b9-400e-9c93-9dc82a56d584/root/var/log/chrony/tracking.log