01HNZK5M4YS0T8TMJKCG3Q7VXE: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log*",
        "%/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.28",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01HNZK641CQ9F9MPYKBSQ0B5C3

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12024-02-06T16:50:01.570Zjob dependencies complete; ready to run (waiting for 35 m 32 s)
22024-02-06T16:53:10.955Zjob assigned to worker 01HNZN7C6F6R1EXA2ARSWMGA4A (queued for 3 m 9 s)
32024-02-06T16:53:10.981Zhttp://172.20.0.5:9969/ipxe/buskin... ok
42024-02-06T16:53:11.100Zxcvsh0tfvd xcvsh0tfvd xcvsh0tfvd xcvsh0tfvd xcvsh0tfvd xcvsh0tfvd xcvsh0tfvd xcvsh0tfvd
52024-02-06T16:53:11.155Zhttp://172.20.0.5:9969/ipxe/buskin... ok
62024-02-06T16:53:11.192Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/kernel/amd64/unix... ok
72024-02-06T16:53:11.229Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive... ok
82024-02-06T16:53:11.267Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive.hash... ok
92024-02-06T16:53:11.304Zhttp://172.20.0.5:9969/postboot/buskin... ok
102024-02-06T16:53:11.338ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
112024-02-06T16:53:11.371Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d1
122024-02-06T16:53:11.405ZOxide Helios Version helios-2.0.22451 64-bit
132024-02-06T16:53:11.438ZConfiguring devices.
142024-02-06T16:53:11.472Z##################################################
152024-02-06T16:53:11.506Z#### ############# ##
162024-02-06T16:53:11.539Z### ### ############ ##
172024-02-06T16:53:11.574Z## ### # ## ### ## Oxide Computer Company ##
182024-02-06T16:53:11.606Z## ## ## ### # ### ##
192024-02-06T16:53:11.640Z## # ### #### #### This Station Under ##
202024-02-06T16:53:11.676Z### ### #### # ### Computer Control ##
212024-02-06T16:53:11.707Z#### #### ### ## ##
222024-02-06T16:53:11.752Z##################################################
232024-02-06T16:53:11.785ZHostname: buskin
242024-02-06T16:53:11.818ZLoading smf(7) service descriptions:
252024-02-06T16:53:12.503Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap-8c3c2066493dbc13.gz
262024-02-06T16:53:14.401Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap-8c3c2066493dbc13.gz
272024-02-06T16:53:14.401Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
282024-02-06T16:53:33.420Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
292024-02-06T16:53:33.444Zdownloading input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-17ee7da3f7e5501f.gz
302024-02-06T16:53:52.736Zdownloaded input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-17ee7da3f7e5501f.gz
312024-02-06T16:53:52.736Zdownloading input: /input/ci-tools/work/caboose-util.gz
322024-02-06T16:53:53.545Zdownloaded input: /input/ci-tools/work/caboose-util.gz
332024-02-06T16:53:53.545Zdownloading input: /input/ci-tools/work/tufaceous.gz
342024-02-06T16:53:56.428Zdownloaded input: /input/ci-tools/work/tufaceous.gz
352024-02-06T16:53:56.452Zdownloading input: /input/package/work/version.txt
362024-02-06T16:53:56.488Zdownloaded input: /input/package/work/version.txt
372024-02-06T16:53:56.488Zdownloading input: /input/package/work/package.tar.gz
382024-02-06T16:54:11.201Zdownloaded input: /input/package/work/package.tar.gz
392024-02-06T16:54:11.201Zdownloading input: /input/package/work/global-zone-packages.tar.gz
402024-02-06T16:54:32.701Zdownloaded input: /input/package/work/global-zone-packages.tar.gz
412024-02-06T16:54:32.701Zdownloading input: /input/package/work/trampoline-global-zone-packages.tar.gz
422024-02-06T16:54:39.116Zdownloaded input: /input/package/work/trampoline-global-zone-packages.tar.gz
432024-02-06T16:54:39.140Zdownloading input: /input/package/work/zones/clickhouse.tar.gz
442024-02-06T16:55:12.423Zdownloaded input: /input/package/work/zones/clickhouse.tar.gz
452024-02-06T16:55:12.423Zdownloading input: /input/package/work/zones/clickhouse_keeper.tar.gz
462024-02-06T16:55:45.859Zdownloaded input: /input/package/work/zones/clickhouse_keeper.tar.gz
472024-02-06T16:55:45.859Zdownloading input: /input/package/work/zones/cockroachdb.tar.gz
482024-02-06T16:56:08.510Zdownloaded input: /input/package/work/zones/cockroachdb.tar.gz
492024-02-06T16:56:08.510Zdownloading input: /input/package/work/zones/crucible-pantry.tar.gz
502024-02-06T16:56:11.654Zdownloaded input: /input/package/work/zones/crucible-pantry.tar.gz
512024-02-06T16:56:11.654Zdownloading input: /input/package/work/zones/crucible.tar.gz
522024-02-06T16:56:17.591Zdownloaded input: /input/package/work/zones/crucible.tar.gz
532024-02-06T16:56:17.615Zdownloading input: /input/package/work/zones/external-dns.tar.gz
542024-02-06T16:56:22.020Zdownloaded input: /input/package/work/zones/external-dns.tar.gz
552024-02-06T16:56:22.020Zdownloading input: /input/package/work/zones/internal-dns.tar.gz
562024-02-06T16:56:26.173Zdownloaded input: /input/package/work/zones/internal-dns.tar.gz
572024-02-06T16:56:26.173Zdownloading input: /input/package/work/zones/ntp.tar.gz
582024-02-06T16:56:26.212Zdownloaded input: /input/package/work/zones/ntp.tar.gz
592024-02-06T16:56:26.212Zdownloading input: /input/package/work/zones/omicron-gateway-asic.tar.gz
602024-02-06T16:56:29.653Zdownloaded input: /input/package/work/zones/omicron-gateway-asic.tar.gz
612024-02-06T16:56:29.653Zdownloading input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
622024-02-06T16:56:33.100Zdownloaded input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
632024-02-06T16:56:33.100Zdownloading input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
642024-02-06T16:56:48.426Zdownloaded input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
652024-02-06T16:56:48.450Zdownloading input: /input/package/work/zones/omicron-nexus.tar.gz
662024-02-06T16:57:03.686Zdownloaded input: /input/package/work/zones/omicron-nexus.tar.gz
672024-02-06T16:57:03.686Zdownloading input: /input/package/work/zones/overlay.tar.gz
682024-02-06T16:57:03.752Zdownloaded input: /input/package/work/zones/overlay.tar.gz
692024-02-06T16:57:03.777Zdownloading input: /input/package/work/zones/oximeter.tar.gz
702024-02-06T16:57:13.583Zdownloaded input: /input/package/work/zones/oximeter.tar.gz
712024-02-06T16:57:13.583Zdownloading input: /input/package/work/zones/propolis-server.tar.gz
722024-02-06T16:57:25.354Zdownloaded input: /input/package/work/zones/propolis-server.tar.gz
732024-02-06T16:57:25.354Zdownloading input: /input/package/work/zones/switch-asic.tar.gz
742024-02-06T16:58:13.875Zdownloaded input: /input/package/work/zones/switch-asic.tar.gz
752024-02-06T16:58:13.876Zdownloading input: /input/package/work/zones/switch-softnpu.tar.gz
762024-02-06T16:58:49.821Zdownloaded input: /input/package/work/zones/switch-softnpu.tar.gz
 
772024-02-06T16:58:49.821Zstarting task 0: "setup"
782024-02-06T16:58:49.846Z++ uname -s
792024-02-06T16:58:49.846Z+ kern=SunOS
802024-02-06T16:58:49.846Z+ case "$kern" in
812024-02-06T16:58:49.846Z+ groupadd -g 12345 build
822024-02-06T16:58:49.846Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
832024-02-06T16:58:51.838Z+ zfs create -o mountpoint=/work rpool/work
842024-02-06T16:58:51.863Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
852024-02-06T16:58:51.863Z+ home_fs=zfs
862024-02-06T16:58:51.863Z+ [[ zfs == autofs ]]
872024-02-06T16:58:51.863Z+ mkdir -p /home/build
882024-02-06T16:58:51.863Z+ chown build:build /home/build /work
892024-02-06T16:58:53.866Z+ chmod 0700 /home/build /work
902024-02-06T16:58:53.890Zprocess exited: duration 4047 ms, exit code 0
 
912024-02-06T16:58:53.939Zstarting task 1: "authentication"
922024-02-06T16:58:53.981Zprocess exited: duration 42 ms, exit code 0
 
932024-02-06T16:58:54.028Zstarting task 2: "build"
942024-02-06T16:58:54.052Z+ trap _exit_trap EXIT
952024-02-06T16:58:54.052Z+ set +x
962024-02-06T16:59:21.594Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
972024-02-06T16:59:21.836Zsecpolicy_fs_linkdir: 0x55 = 0x55
982024-02-06T16:59:21.836Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
992024-02-06T16:59:21.836Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
1002024-02-06T16:59:21.836Zsecpolicy_fs_linkdir+3: 0 = 0xe5
1012024-02-06T16:59:21.860Zsecpolicy_fs_linkdir+4: 0 = 0xb8
1022024-02-06T16:59:21.860Zsecpolicy_fs_linkdir+5: 0 = 0x1
1032024-02-06T16:59:21.860Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
1042024-02-06T16:59:21.860Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
1052024-02-06T16:59:21.861Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
1062024-02-06T16:59:21.861Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
1072024-02-06T16:59:21.861Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
1082024-02-06T16:59:21.861Z+ [[ -d /opt/oxide ]]
1092024-02-06T16:59:21.861Z+ pfexec mv /opt/oxide /opt/oxide-underneath
1102024-02-06T16:59:21.861Z+ pfexec mkdir /opt/oxide
1112024-02-06T16:59:21.861Z+ pfexec mount -F tmpfs -O swap /opt/oxide
1122024-02-06T16:59:21.861Z+ [[ -d /opt/oxide-underneath ]]
1132024-02-06T16:59:21.861Z+ cd /opt/oxide-underneath
1142024-02-06T16:59:21.861Z+ pfexec tar ceEp@/f - .
1152024-02-06T16:59:21.861Z+ cd /opt/oxide
1162024-02-06T16:59:21.861Z+ pfexec tar xveEp@/f -
1172024-02-06T16:59:21.899Zx ., 0 bytes, 0 tape blocks
1182024-02-06T16:59:21.899Zx ./opte, 0 bytes, 0 tape blocks
1192024-02-06T16:59:21.900Zx ./opte/bin, 0 bytes, 0 tape blocks
1202024-02-06T16:59:21.900Zx ./opte/bin/opteadm, 51799224 bytes, 101171 tape blocks
1212024-02-06T16:59:21.923Z+ pfexec rm -rf /opt/oxide-underneath
1222024-02-06T16:59:21.923Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
1232024-02-06T16:59:21.948Z+ pfexec mkdir /opt/oxide/work
1242024-02-06T16:59:21.948Z+ pfexec chown build:build /opt/oxide/work
1252024-02-06T16:59:21.948Z+ cd /opt/oxide/work
1262024-02-06T16:59:21.948Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
1272024-02-06T16:59:23.621ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
1282024-02-06T16:59:23.621Zx out/mg-ddm-gz.tar, 63076864 bytes, 123197 tape blocks
1292024-02-06T16:59:23.621Zx out/omicron-sled-agent.tar, 122532864 bytes, 239322 tape blocks
1302024-02-06T16:59:23.621Zx out/oxlog.tar, 8382464 bytes, 16372 tape blocks
1312024-02-06T16:59:23.621Zx out/target/test, 75 bytes, 1 tape blocks
1322024-02-06T16:59:23.621Zx out/npuzone/npuzone, 37184104 bytes, 72626 tape blocks
1332024-02-06T16:59:23.621Zx package-manifest.toml, 23732 bytes, 47 tape blocks
1342024-02-06T16:59:23.621Zx smf/sled-agent/non-gimlet/config.toml, 3448 bytes, 7 tape blocks
1352024-02-06T16:59:23.621Zx target/release/omicron-package, 45711600 bytes, 89281 tape blocks
1362024-02-06T16:59:23.622Zx tools/create_virtual_hardware.sh, 3184 bytes, 7 tape blocks
1372024-02-06T16:59:23.622Zx tools/virtual_hardware.sh, 3416 bytes, 7 tape blocks
1382024-02-06T16:59:23.622Zx tools/scrimlet/create-softnpu-zone.sh, 627 bytes, 2 tape blocks
1392024-02-06T16:59:23.622Zx tools/scrimlet/destroy-softnpu-zone.sh, 203 bytes, 1 tape blocks
1402024-02-06T16:59:23.622Zx tools/scrimlet/softnpu-init.sh, 2157 bytes, 5 tape blocks
1412024-02-06T16:59:23.622Zx tools/scrimlet/softnpu-zone.txt, 330 bytes, 1 tape blocks
1422024-02-06T16:59:23.622Zx tools/scrimlet/softnpu.toml, 171 bytes, 1 tape blocks
1432024-02-06T16:59:23.646Z
1442024-02-06T16:59:23.646Zreal 1.679904165
1452024-02-06T16:59:23.646Zuser 1.505685290
1462024-02-06T16:59:23.646Zsys 0.304501059
1472024-02-06T16:59:23.646Ztrap 0.000064622
1482024-02-06T16:59:23.646Ztflt 0.000000000
1492024-02-06T16:59:23.646Zdflt 0.000000000
1502024-02-06T16:59:23.646Zkflt 0.000000000
1512024-02-06T16:59:23.646Zlock 0.000000000
1522024-02-06T16:59:23.646Zslp 1.506538408
1532024-02-06T16:59:23.646Zlat 0.041068875
1542024-02-06T16:59:23.646Zstop 0.000127603
1552024-02-06T16:59:23.646Z+ cp /input/package/work/zones/clickhouse_keeper.tar.gz /input/package/work/zones/clickhouse.tar.gz /input/package/work/zones/cockroachdb.tar.gz /input/package/work/zones/crucible-pantry.tar.gz /input/package/work/zones/crucible.tar.gz /input/package/work/zones/external-dns.tar.gz /input/package/work/zones/internal-dns.tar.gz /input/package/work/zones/ntp.tar.gz /input/package/work/zones/omicron-gateway-asic.tar.gz /input/package/work/zones/omicron-gateway-softnpu.tar.gz /input/package/work/zones/omicron-nexus-single-sled.tar.gz /input/package/work/zones/omicron-nexus.tar.gz /input/package/work/zones/overlay.tar.gz /input/package/work/zones/oximeter.tar.gz /input/package/work/zones/propolis-server.tar.gz /input/package/work/zones/switch-asic.tar.gz /input/package/work/zones/switch-softnpu.tar.gz out/
1562024-02-06T16:59:24.851Z+ mv out/omicron-nexus-single-sled.tar.gz out/omicron-nexus.tar.gz
1572024-02-06T16:59:24.875Z+ mkdir tests
1582024-02-06T16:59:24.875Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1592024-02-06T16:59:24.875Z+ ptime -m gunzip
1602024-02-06T16:59:24.875Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-8c3c2066493dbc13
1612024-02-06T16:59:25.107Z
1622024-02-06T16:59:25.131Zreal 0.231438719
1632024-02-06T16:59:25.132Zuser 0.213858282
1642024-02-06T16:59:25.132Zsys 0.017335740
1652024-02-06T16:59:25.132Ztrap 0.000017300
1662024-02-06T16:59:25.132Ztflt 0.000000000
1672024-02-06T16:59:25.132Zdflt 0.000000000
1682024-02-06T16:59:25.132Zkflt 0.000000000
1692024-02-06T16:59:25.132Zlock 0.000000000
1702024-02-06T16:59:25.132Zslp 0.000023190
1712024-02-06T16:59:25.132Zlat 0.000017030
1722024-02-06T16:59:25.132Zstop 0.000125143
1732024-02-06T16:59:25.132Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-8c3c2066493dbc13
1742024-02-06T16:59:25.132Z+ chmod a+x tests/bootstrap-8c3c2066493dbc13
1752024-02-06T16:59:25.132Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1762024-02-06T16:59:25.132Z+ ptime -m gunzip
1772024-02-06T16:59:25.132Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1782024-02-06T16:59:27.593Z
1792024-02-06T16:59:27.617Zreal 2.475019169
1802024-02-06T16:59:27.617Zuser 2.333675466
1812024-02-06T16:59:27.617Zsys 0.140883816
1822024-02-06T16:59:27.617Ztrap 0.000216900
1832024-02-06T16:59:27.617Ztflt 0.000000000
1842024-02-06T16:59:27.617Zdflt 0.000000000
1852024-02-06T16:59:27.617Zkflt 0.000000000
1862024-02-06T16:59:27.617Zlock 0.000000000
1872024-02-06T16:59:27.617Zslp 0.000023110
1882024-02-06T16:59:27.617Zlat 0.000034391
1892024-02-06T16:59:27.618Zstop 0.000129793
1902024-02-06T16:59:27.618Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1912024-02-06T16:59:27.618Z+ chmod a+x tests/bootstrap
1922024-02-06T16:59:27.618Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1932024-02-06T16:59:27.618Z+ ptime -m gunzip
1942024-02-06T16:59:27.618Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-17ee7da3f7e5501f
1952024-02-06T16:59:30.172Z
1962024-02-06T16:59:30.196Zreal 2.566765755
1972024-02-06T16:59:30.196Zuser 2.399731661
1982024-02-06T16:59:30.196Zsys 0.166615433
1992024-02-06T16:59:30.196Ztrap 0.000175744
2002024-02-06T16:59:30.196Ztflt 0.000000000
2012024-02-06T16:59:30.196Zdflt 0.000000000
2022024-02-06T16:59:30.196Zkflt 0.000000000
2032024-02-06T16:59:30.196Zlock 0.000000000
2042024-02-06T16:59:30.196Zslp 0.000019730
2052024-02-06T16:59:30.196Zlat 0.000017030
2062024-02-06T16:59:30.196Zstop 0.000119273
2072024-02-06T16:59:30.196Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-17ee7da3f7e5501f
2082024-02-06T16:59:30.196Z+ chmod a+x tests/end_to_end_tests-17ee7da3f7e5501f
2092024-02-06T16:59:30.196Z+ bmat address ls
2102024-02-06T16:59:30.196ZNAME CIDR FIRST LAST COUNT
2112024-02-06T16:59:30.196Zextra 10.151.1.0/24 10.151.1.164 10.151.1.227 64
2122024-02-06T16:59:30.196Z++ bmat address ls -f extra -Ho first,last,count
2132024-02-06T16:59:30.220Z+ set -- 10.151.1.164 10.151.1.227 64
2142024-02-06T16:59:30.221Z+ EXTRA_IP_START=10.151.1.164
2152024-02-06T16:59:30.221Z+ EXTRA_IP_END=10.151.1.227
2162024-02-06T16:59:30.221Z+ EXTRA_IP_COUNT=64
2172024-02-06T16:59:30.221Z+ (( EXTRA_IP_COUNT >= 32 ))
2182024-02-06T16:59:30.221Z+ EXTRA_IP_BASE=10.151.1
2192024-02-06T16:59:30.221Z+ EXTRA_IP_FOCTET=164
2202024-02-06T16:59:30.221Z+ EXTRA_IP_LOCTET=227
2212024-02-06T16:59:30.221Z+ SERVICE_IP_POOL_START=10.151.1.164
2222024-02-06T16:59:30.221Z+ SERVICE_IP_POOL_END=10.151.1.173
2232024-02-06T16:59:30.221Z+ DNS_IP1=10.151.1.164
2242024-02-06T16:59:30.221Z+ DNS_IP2=10.151.1.165
2252024-02-06T16:59:30.221Z+ UPLINK_IP=10.151.1.174
2262024-02-06T16:59:30.221Z+ IPPOOL_START=10.151.1.175
2272024-02-06T16:59:30.221Z+ IPPOOL_END=10.151.1.227
2282024-02-06T16:59:30.221Z++ ipadm show-addr -po type,addr
2292024-02-06T16:59:30.221Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
2302024-02-06T16:59:30.221Z+ GATEWAY_IP=10.151.1.100
2312024-02-06T16:59:30.221Z+ [[ -n 10.151.1.100 ]]
2322024-02-06T16:59:30.221Z+ ping -s 10.151.1.100 56 1
2332024-02-06T16:59:30.221ZPING 10.151.1.100: 56 data bytes
2342024-02-06T16:59:30.221Z64 bytes from 10.151.1.100: icmp_seq=0. time=0.010 ms
2352024-02-06T16:59:30.221Z
2362024-02-06T16:59:30.221Z----10.151.1.100 PING Statistics----
2372024-02-06T16:59:30.222Z1 packets transmitted, 1 packets received, 0% packet loss
2382024-02-06T16:59:30.222Zround-trip (ms) min/avg/max/stddev = 0.010/0.010/0.010/-nan
2392024-02-06T16:59:30.222Z++ arp -an
2402024-02-06T16:59:30.222Z++ awk -vgw=10.151.1.100 '$2 == gw {print $NF}'
2412024-02-06T16:59:30.249Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
2422024-02-06T16:59:30.249Z+ routeadm -e ipv4-forwarding -u
2432024-02-06T16:59:30.335Z+ PXA_START=10.151.1.164
2442024-02-06T16:59:30.335Z+ PXA_END=10.151.1.227
2452024-02-06T16:59:30.359Z+ export GATEWAY_IP GATEWAY_MAC PXA_START PXA_END
2462024-02-06T16:59:30.359Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
2472024-02-06T16:59:30.522Z+ ZPOOL_VDEV_DIR=/scratch
2482024-02-06T16:59:30.522Z+ ptime -m pfexec ./tools/create_virtual_hardware.sh
2492024-02-06T16:59:30.546Z+++ dirname ./tools/create_virtual_hardware.sh
2502024-02-06T16:59:30.546Z++ cd ./tools
2512024-02-06T16:59:30.546Z++ pwd
2522024-02-06T16:59:30.546Z+ SOURCE_DIR=/opt/oxide/work/tools
2532024-02-06T16:59:30.546Z+ OMICRON_TOP=/opt/oxide/work/tools/..
2542024-02-06T16:59:30.546Z+ SOFTNPU_MODE=zone
2552024-02-06T16:59:30.546Z+ . /opt/oxide/work/tools/virtual_hardware.sh
2562024-02-06T16:59:30.546Z++ MARKER=/etc/opt/oxide/NO_INSTALL
2572024-02-06T16:59:30.546Z++ [[ -f /etc/opt/oxide/NO_INSTALL ]]
2582024-02-06T16:59:30.546Z++ dladm show-phys -p -o LINK
2592024-02-06T16:59:30.546Z++ head -1
2602024-02-06T16:59:30.571Z+ PHYSICAL_LINK=igb0
2612024-02-06T16:59:30.571Z+ echo 'Using igb0 as physical link'
2622024-02-06T16:59:30.571ZUsing igb0 as physical link
2632024-02-06T16:59:30.571Z+ ensure_run_as_root
2642024-02-06T16:59:30.571Z++ id -u
2652024-02-06T16:59:30.571Z+ [[ 0 -ne 0 ]]
2662024-02-06T16:59:30.571Z+ ensure_zpools
2672024-02-06T16:59:30.571Z+ ZPOOL_TYPES=('oxp_' 'oxi_')
2682024-02-06T16:59:30.571Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
2692024-02-06T16:59:30.571Z+ readarray -t ZPOOLS
2702024-02-06T16:59:30.571Z++ grep '"oxp_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
2712024-02-06T16:59:30.571Z++ sed 's/[ ",]//g'
2722024-02-06T16:59:30.571Z+ for ZPOOL in "${ZPOOLS[@]}"
2732024-02-06T16:59:30.571Z+ echo 'Zpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]'
2742024-02-06T16:59:30.571ZZpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]
2752024-02-06T16:59:30.571Z+ VDEV_PATH=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2762024-02-06T16:59:30.571Z+ [[ -f /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
2772024-02-06T16:59:30.571Z+ dd if=/dev/zero of=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
2782024-02-06T16:59:30.571Z0+0 records in
2792024-02-06T16:59:30.571Z0+0 records out
2802024-02-06T16:59:30.572Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
2812024-02-06T16:59:30.572Z+ success 'ZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
2822024-02-06T16:59:30.572Z+ set +x
2832024-02-06T16:59:30.572ZZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
2842024-02-06T16:59:30.572Z++ zpool list -o name
2852024-02-06T16:59:30.572Z++ grep oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
2862024-02-06T16:59:30.572Z+ [[ -z '' ]]
2872024-02-06T16:59:30.572Z+ zpool create -f oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2882024-02-06T16:59:30.602Z+ success 'ZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
2892024-02-06T16:59:30.626Z+ set +x
2902024-02-06T16:59:30.626ZZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists
2912024-02-06T16:59:30.626ZZpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]
2922024-02-06T16:59:30.626Z+ for ZPOOL in "${ZPOOLS[@]}"
2932024-02-06T16:59:30.626Z+ echo 'Zpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
2942024-02-06T16:59:30.626Z+ VDEV_PATH=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
2952024-02-06T16:59:30.626Z+ [[ -f /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
2962024-02-06T16:59:30.626Z+ dd if=/dev/zero of=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
2972024-02-06T16:59:30.626Z0+0 records in
2982024-02-06T16:59:30.626Z0+0 records out
2992024-02-06T16:59:30.626Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3002024-02-06T16:59:30.626Z+ success 'ZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3012024-02-06T16:59:30.626Z+ set +x
3022024-02-06T16:59:30.626ZZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3032024-02-06T16:59:30.626Z++ zpool list -o name
3042024-02-06T16:59:30.626Z++ grep oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
3052024-02-06T16:59:30.627Z+ [[ -z '' ]]
3062024-02-06T16:59:30.627Z+ zpool create -f oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3072024-02-06T16:59:30.651Z+ success 'ZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3082024-02-06T16:59:30.651Z+ set +x
3092024-02-06T16:59:30.651ZZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3102024-02-06T16:59:30.651Z+ for ZPOOL in "${ZPOOLS[@]}"
3112024-02-06T16:59:30.652Z+ echo 'Zpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3122024-02-06T16:59:30.652ZZpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]
3132024-02-06T16:59:30.652Z+ VDEV_PATH=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3142024-02-06T16:59:30.652Z+ [[ -f /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3152024-02-06T16:59:30.652Z+ dd if=/dev/zero of=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3162024-02-06T16:59:30.652Z0+0 records in
3172024-02-06T16:59:30.652Z0+0 records out
3182024-02-06T16:59:30.652Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3192024-02-06T16:59:30.652Z+ success 'ZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3202024-02-06T16:59:30.652Z+ set +x
3212024-02-06T16:59:30.652ZZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3222024-02-06T16:59:30.652Z++ zpool list -o name
3232024-02-06T16:59:30.652Z++ grep oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
3242024-02-06T16:59:30.676Z+ [[ -z '' ]]
3252024-02-06T16:59:30.676Z+ zpool create -f oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3262024-02-06T16:59:30.700Z+ success 'ZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3272024-02-06T16:59:30.700Z+ set +x
3282024-02-06T16:59:30.700ZZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3292024-02-06T16:59:30.700Z+ for ZPOOL in "${ZPOOLS[@]}"
3302024-02-06T16:59:30.700ZZpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]
3312024-02-06T16:59:30.700Z+ echo 'Zpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3322024-02-06T16:59:30.700Z+ VDEV_PATH=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3332024-02-06T16:59:30.700Z+ [[ -f /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3342024-02-06T16:59:30.700Z+ dd if=/dev/zero of=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3352024-02-06T16:59:30.701Z0+0 records in
3362024-02-06T16:59:30.701Z0+0 records out
3372024-02-06T16:59:30.701Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3382024-02-06T16:59:30.701Z+ success 'ZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3392024-02-06T16:59:30.701Z+ set +x
3402024-02-06T16:59:30.701ZZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3412024-02-06T16:59:30.701Z++ zpool list -o name
3422024-02-06T16:59:30.701Z++ grep oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03
3432024-02-06T16:59:30.701Z+ [[ -z '' ]]
3442024-02-06T16:59:30.701Z+ zpool create -f oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3452024-02-06T16:59:30.725Z+ success 'ZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3462024-02-06T16:59:30.725Z+ set +x
3472024-02-06T16:59:30.725ZZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3482024-02-06T16:59:30.725Z+ for ZPOOL in "${ZPOOLS[@]}"
3492024-02-06T16:59:30.725Z+ echo 'Zpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3502024-02-06T16:59:30.725ZZpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]
3512024-02-06T16:59:30.725Z+ VDEV_PATH=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3522024-02-06T16:59:30.725Z+ [[ -f /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3532024-02-06T16:59:30.725Z+ dd if=/dev/zero of=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3542024-02-06T16:59:30.750Z0+0 records in
3552024-02-06T16:59:30.750Z0+0 records out
3562024-02-06T16:59:30.751Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3572024-02-06T16:59:30.751Z+ success 'ZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3582024-02-06T16:59:30.751Z+ set +x
3592024-02-06T16:59:30.751ZZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3602024-02-06T16:59:30.751Z++ zpool list -o name
3612024-02-06T16:59:30.751Z++ grep oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03
3622024-02-06T16:59:30.751Z+ [[ -z '' ]]
3632024-02-06T16:59:30.751Z+ zpool create -f oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3642024-02-06T16:59:30.776Z+ success 'ZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3652024-02-06T16:59:30.776Z+ set +x
3662024-02-06T16:59:30.776ZZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3672024-02-06T16:59:30.776Z+ for ZPOOL in "${ZPOOLS[@]}"
3682024-02-06T16:59:30.776Z+ echo 'Zpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]'
3692024-02-06T16:59:30.776ZZpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]
3702024-02-06T16:59:30.776Z+ VDEV_PATH=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3712024-02-06T16:59:30.776Z+ [[ -f /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev ]]
3722024-02-06T16:59:30.776Z+ dd if=/dev/zero of=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev bs=1 count=0 seek=20G
3732024-02-06T16:59:30.776Z0+0 records in
3742024-02-06T16:59:30.776Z0+0 records out
3752024-02-06T16:59:30.776Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3762024-02-06T16:59:30.776Z+ success 'ZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists'
3772024-02-06T16:59:30.776Z+ set +x
3782024-02-06T16:59:30.776ZZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists
3792024-02-06T16:59:30.776Z++ zpool list -o name
3802024-02-06T16:59:30.801Z++ grep oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930
3812024-02-06T16:59:30.801Z+ [[ -z '' ]]
3822024-02-06T16:59:30.801Z+ zpool create -f oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3832024-02-06T16:59:30.825Z+ success 'ZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists'
3842024-02-06T16:59:30.825Z+ set +x
3852024-02-06T16:59:30.825ZZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists
3862024-02-06T16:59:30.825Z+ for ZPOOL in "${ZPOOLS[@]}"
3872024-02-06T16:59:30.825Z+ echo 'Zpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]'
3882024-02-06T16:59:30.825ZZpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]
3892024-02-06T16:59:30.825Z+ VDEV_PATH=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
3902024-02-06T16:59:30.825Z+ [[ -f /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev ]]
3912024-02-06T16:59:30.825Z+ dd if=/dev/zero of=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev bs=1 count=0 seek=20G
3922024-02-06T16:59:30.825Z0+0 records in
3932024-02-06T16:59:30.825Z0+0 records out
3942024-02-06T16:59:30.825Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3952024-02-06T16:59:30.825Z+ success 'ZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists'
3962024-02-06T16:59:30.825Z+ set +x
3972024-02-06T16:59:30.825ZZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists
3982024-02-06T16:59:30.825Z++ zpool list -o name
3992024-02-06T16:59:30.826Z++ grep oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60
4002024-02-06T16:59:30.853Z+ [[ -z '' ]]
4012024-02-06T16:59:30.853Z+ zpool create -f oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
4022024-02-06T16:59:30.878Z+ success 'ZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists'
4032024-02-06T16:59:30.879Z+ set +x
4042024-02-06T16:59:30.879ZZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists
4052024-02-06T16:59:30.879Z+ for ZPOOL in "${ZPOOLS[@]}"
4062024-02-06T16:59:30.879Z+ echo 'Zpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]'
4072024-02-06T16:59:30.879ZZpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]
4082024-02-06T16:59:30.879Z+ VDEV_PATH=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4092024-02-06T16:59:30.879Z+ [[ -f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev ]]
4102024-02-06T16:59:30.879Z+ dd if=/dev/zero of=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev bs=1 count=0 seek=20G
4112024-02-06T16:59:30.879Z0+0 records in
4122024-02-06T16:59:30.879Z0+0 records out
4132024-02-06T16:59:30.879Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4142024-02-06T16:59:30.879Z+ success 'ZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists'
4152024-02-06T16:59:30.879Z+ set +x
4162024-02-06T16:59:30.879ZZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists
4172024-02-06T16:59:30.879Z++ zpool list -o name
4182024-02-06T16:59:30.879Z++ grep oxp_31bd71cd-4736-4a12-a387-9b74b050396f
4192024-02-06T16:59:30.879Z+ [[ -z '' ]]
4202024-02-06T16:59:30.879Z+ zpool create -f oxp_31bd71cd-4736-4a12-a387-9b74b050396f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4212024-02-06T16:59:30.905Z+ success 'ZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists'
4222024-02-06T16:59:30.905Z+ set +x
4232024-02-06T16:59:30.905ZZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists
4242024-02-06T16:59:30.905Z+ for ZPOOL in "${ZPOOLS[@]}"
4252024-02-06T16:59:30.905Z+ echo 'Zpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]'
4262024-02-06T16:59:30.905ZZpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]
4272024-02-06T16:59:30.905Z+ VDEV_PATH=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4282024-02-06T16:59:30.905Z+ [[ -f /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev ]]
4292024-02-06T16:59:30.905Z+ dd if=/dev/zero of=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev bs=1 count=0 seek=20G
4302024-02-06T16:59:30.929Z0+0 records in
4312024-02-06T16:59:30.929Z0+0 records out
4322024-02-06T16:59:30.930Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4332024-02-06T16:59:30.930Z+ success 'ZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists'
4342024-02-06T16:59:30.930Z+ set +x
4352024-02-06T16:59:30.930ZZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists
4362024-02-06T16:59:30.930Z++ zpool list -o name
4372024-02-06T16:59:30.930Z++ grep oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7
4382024-02-06T16:59:30.930Z+ [[ -z '' ]]
4392024-02-06T16:59:30.930Z+ zpool create -f oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4402024-02-06T16:59:30.955Z+ success 'ZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists'
4412024-02-06T16:59:30.955Z+ set +x
4422024-02-06T16:59:30.955ZZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists
4432024-02-06T16:59:30.955Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
4442024-02-06T16:59:30.955Z+ readarray -t ZPOOLS
4452024-02-06T16:59:30.955Z++ grep '"oxi_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
4462024-02-06T16:59:30.955Z++ sed 's/[ ",]//g'
4472024-02-06T16:59:30.955Z+ for ZPOOL in "${ZPOOLS[@]}"
4482024-02-06T16:59:30.955Z+ echo 'Zpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4492024-02-06T16:59:30.955ZZpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]
4502024-02-06T16:59:30.955Z+ VDEV_PATH=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4512024-02-06T16:59:30.955Z+ [[ -f /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4522024-02-06T16:59:30.955Z+ dd if=/dev/zero of=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4532024-02-06T16:59:30.955Z0+0 records in
4542024-02-06T16:59:30.955Z0+0 records out
4552024-02-06T16:59:30.956Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4562024-02-06T16:59:30.956Z+ success 'ZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4572024-02-06T16:59:30.956Z+ set +x
4582024-02-06T16:59:30.956ZZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4592024-02-06T16:59:30.981Z++ zpool list -o name
4602024-02-06T16:59:30.981Z++ grep oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b
4612024-02-06T16:59:30.981Z+ [[ -z '' ]]
4622024-02-06T16:59:30.981Z+ zpool create -f oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4632024-02-06T16:59:31.006Z+ success 'ZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4642024-02-06T16:59:31.006Z+ set +x
4652024-02-06T16:59:31.006ZZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4662024-02-06T16:59:31.006Z+ for ZPOOL in "${ZPOOLS[@]}"
4672024-02-06T16:59:31.006Z+ echo 'Zpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4682024-02-06T16:59:31.006ZZpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]
4692024-02-06T16:59:31.006Z+ VDEV_PATH=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4702024-02-06T16:59:31.006Z+ [[ -f /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4712024-02-06T16:59:31.006Z+ dd if=/dev/zero of=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4722024-02-06T16:59:31.006Z0+0 records in
4732024-02-06T16:59:31.006Z0+0 records out
4742024-02-06T16:59:31.006Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4752024-02-06T16:59:31.007Z+ success 'ZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4762024-02-06T16:59:31.007Z+ set +x
4772024-02-06T16:59:31.007ZZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4782024-02-06T16:59:31.007Z++ zpool list -o name
4792024-02-06T16:59:31.007Z++ grep oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b
4802024-02-06T16:59:31.031Z+ [[ -z '' ]]
4812024-02-06T16:59:31.031Z+ zpool create -f oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4822024-02-06T16:59:31.055Z+ success 'ZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4832024-02-06T16:59:31.055Z+ set +x
4842024-02-06T16:59:31.055ZZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4852024-02-06T16:59:31.055Z+ [[ zone == \z\o\n\e ]]
4862024-02-06T16:59:31.055Z+ ensure_simulated_links igb0
4872024-02-06T16:59:31.055Z+ local PHYSICAL_LINK=igb0
4882024-02-06T16:59:31.055Z+ INDICES=("0" "1")
4892024-02-06T16:59:31.055Z+ for I in "${INDICES[@]}"
4902024-02-06T16:59:31.055Z++ get_simnet_name_if_exists net0
4912024-02-06T16:59:31.055Z++ dladm show-simnet -p -o LINK net0
4922024-02-06T16:59:31.055Z++ echo ''
4932024-02-06T16:59:31.055Z+ [[ -z '' ]]
4942024-02-06T16:59:31.055Z+ dladm create-simnet -t net0
4952024-02-06T16:59:31.055Z+ dladm create-simnet -t sc0_0
4962024-02-06T16:59:31.055Z+ dladm modify-simnet -t -p net0 sc0_0
4972024-02-06T16:59:31.079Z+ dladm set-linkprop -p mtu=9000 sc0_0
4982024-02-06T16:59:31.079Z+ success 'Simnet net0/sc0_0 exists'
4992024-02-06T16:59:31.079Z+ set +x
5002024-02-06T16:59:31.079ZSimnet net0/sc0_0 exists
5012024-02-06T16:59:31.079Z+ for I in "${INDICES[@]}"
5022024-02-06T16:59:31.080Z++ get_simnet_name_if_exists net1
5032024-02-06T16:59:31.080Z++ dladm show-simnet -p -o LINK net1
5042024-02-06T16:59:31.080Z++ echo ''
5052024-02-06T16:59:31.080Z+ [[ -z '' ]]
5062024-02-06T16:59:31.080Z+ dladm create-simnet -t net1
5072024-02-06T16:59:31.080Z+ dladm create-simnet -t sc1_0
5082024-02-06T16:59:31.080Z+ dladm modify-simnet -t -p net1 sc1_0
5092024-02-06T16:59:31.080Z+ dladm set-linkprop -p mtu=9000 sc1_0
5102024-02-06T16:59:31.104Z+ success 'Simnet net1/sc1_0 exists'
5112024-02-06T16:59:31.104Z+ set +x
5122024-02-06T16:59:31.104ZSimnet net1/sc1_0 exists
5132024-02-06T16:59:31.104Z++ get_vnic_name_if_exists sc0_1
5142024-02-06T16:59:31.104Z++ dladm show-vnic -p -o LINK sc0_1
5152024-02-06T16:59:31.104Z++ echo ''
5162024-02-06T16:59:31.105Z+ [[ -z '' ]]
5172024-02-06T16:59:31.105Z+ dladm create-vnic -t sc0_1 -l igb0 -m a8:e1:de:01:70:1d
5182024-02-06T16:59:31.105Z+ [[ -v PROMISC_FILT_OFF ]]
5192024-02-06T16:59:31.105Z+ success 'Vnic sc0_1 exists'
5202024-02-06T16:59:31.105Z+ set +x
5212024-02-06T16:59:31.105ZVnic sc0_1 exists
5222024-02-06T16:59:31.105Z+ warn_if_no_proxy_arp
5232024-02-06T16:59:31.105Z+ [[ -v PXA_START ]]
5242024-02-06T16:59:31.105Z+ [[ -v PXA_END ]]
5252024-02-06T16:59:31.105Z+ ensure_softnpu_zone
5262024-02-06T16:59:31.105Z+ zoneadm list
5272024-02-06T16:59:31.105Z+ grep -q sidecar_softnpu
5282024-02-06T16:59:31.105Z+ [[ -f out/npuzone/npuzone ]]
5292024-02-06T16:59:31.105Z+ out/npuzone/npuzone create sidecar --omicron-zone --ports sc0_0,tfportrear0_0 --ports sc0_1,tfportqsfp0_0 --sidecar-lite-commit e3ea4b495ba0a71801ded0776ae4bbd31df57e26 --softnpu-commit dbab082dfa89da5db5ca2325c257089d2f130092
5302024-02-06T16:59:31.129Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/softnpu/image/dbab082dfa89da5db5ca2325c257089d2f130092/softnpu.sha256.txt
5312024-02-06T16:59:36.124Zdownloading softnpu ...done
5322024-02-06T16:59:36.148Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/libsidecar_lite.so.sha256.txt
5332024-02-06T16:59:40.134Zdownloading asic_program.so ...done
5342024-02-06T16:59:40.158Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/scadm.sha256.txt
5352024-02-06T16:59:44.879Zdownloading scadm ...done
5362024-02-06T16:59:44.903Zcreating port tfportrear0_0 for sc0_0
5372024-02-06T16:59:44.903Zcreating port tfportqsfp0_0 for sc0_1
5382024-02-06T16:59:44.927Zsoftnpu zone setup
5392024-02-06T16:59:44.927Zconfigure zone
5402024-02-06T16:59:44.927Zinstall zone
5412024-02-06T16:59:45.952Zboot zone
5422024-02-06T16:59:49.062Z[sidecar_softnpu] svcs milestone/network
5432024-02-06T16:59:49.089ZSTATE STIME FMRI
5442024-02-06T16:59:49.089Zonline 16:59:46 svc:/milestone/network:default
5452024-02-06T16:59:49.089Z[sidecar_softnpu] /softnpu/npu start
5462024-02-06T16:59:49.114Z
5472024-02-06T16:59:49.138Z+ /opt/oxide/work/tools/scrimlet/softnpu-init.sh
5482024-02-06T16:59:49.138Z+ GATEWAY_IP=10.151.1.100
5492024-02-06T16:59:49.138ZUsing 10.151.1.100 as gateway ip
5502024-02-06T16:59:49.138Z+ echo 'Using 10.151.1.100 as gateway ip'
5512024-02-06T16:59:49.138Z+ [[ ! -v GATEWAY_MAC ]]
5522024-02-06T16:59:49.138Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
5532024-02-06T16:59:49.138Z++ grep -c -E '^([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}$' /dev/fd/63
5542024-02-06T16:59:49.138Z+++ echo 18:c0:4d:0d:9f:b2
5552024-02-06T16:59:49.138Z+ COUNT=1
5562024-02-06T16:59:49.139Z+ [[ 1 -ne 1 ]]
5572024-02-06T16:59:49.139Z+ echo 'Using 18:c0:4d:0d:9f:b2 as gateway mac'
5582024-02-06T16:59:49.139ZUsing 18:c0:4d:0d:9f:b2 as gateway mac
5592024-02-06T16:59:49.139Z+ z_scadm add-arp-entry 10.151.1.100 18:c0:4d:0d:9f:b2
5602024-02-06T16:59:49.139Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-arp-entry 10.151.1.100 18:c0:4d:0d:9f:b2
5612024-02-06T16:59:49.163Z+ PXA_MAC=a8:e1:de:01:70:1d
5622024-02-06T16:59:49.163Z+ [[ -v PXA_START ]]
5632024-02-06T16:59:49.163Z+ z_scadm add-proxy-arp 10.151.1.164 10.151.1.227 a8:e1:de:01:70:1d
5642024-02-06T16:59:49.163Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-proxy-arp 10.151.1.164 10.151.1.227 a8:e1:de:01:70:1d
5652024-02-06T16:59:49.187Z+ z_scadm dump-state
5662024-02-06T16:59:49.187Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
5672024-02-06T16:59:49.211Zlocal v6:
5682024-02-06T16:59:49.211Zlocal v4:
5692024-02-06T16:59:49.211Zrouter v6:
5702024-02-06T16:59:49.211Zrouter v4_idx:
5712024-02-06T16:59:49.211Zrouter v4_routes:
5722024-02-06T16:59:49.211Zresolver v4:
5732024-02-06T16:59:49.211Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
5742024-02-06T16:59:49.211Zresolver v6:
5752024-02-06T16:59:49.211Znat v4:
5762024-02-06T16:59:49.211Znat_v6:
5772024-02-06T16:59:49.211Zport_mac:
5782024-02-06T16:59:49.211Zproxy arp:
5792024-02-06T16:59:49.211Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
5802024-02-06T16:59:49.235Z+ success 'softnpu zone exists'
5812024-02-06T16:59:49.235Z+ set +x
5822024-02-06T16:59:49.235Zsoftnpu zone exists
5832024-02-06T16:59:49.235Z
5842024-02-06T16:59:49.236Zreal 18.688985471
5852024-02-06T16:59:49.236Zuser 0.726597724
5862024-02-06T16:59:49.236Zsys 1.570048978
5872024-02-06T16:59:49.236Ztrap 0.000090012
5882024-02-06T16:59:49.236Ztflt 0.000167976
5892024-02-06T16:59:49.236Zdflt 0.001224002
5902024-02-06T16:59:49.236Zkflt 0.000000000
5912024-02-06T16:59:49.236Zlock 14:21.396742130
5922024-02-06T16:59:49.236Zslp 55.687165991
5932024-02-06T16:59:49.236Zlat 0.089369556
5942024-02-06T16:59:49.236Zstop 0.372386537
5952024-02-06T16:59:49.236Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml
5962024-02-06T16:59:49.260Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
5972024-02-06T16:59:49.260Z+ SILO_NAME=recovery
5982024-02-06T16:59:49.260Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
5992024-02-06T16:59:49.260Z+ EXTERNAL_DNS_DOMAIN=oxide.test
6002024-02-06T16:59:49.260Z+ sed -i~ '
6012024-02-06T16:59:49.260Z /^external_dns_ips/c\
6022024-02-06T16:59:49.260Zexternal_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
6032024-02-06T16:59:49.260Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
6042024-02-06T16:59:49.260Z /^first/c\
6052024-02-06T16:59:49.260Zfirst = "10.151.1.164"
6062024-02-06T16:59:49.261Z /^last/c\
6072024-02-06T16:59:49.261Zlast = "10.151.1.173"
6082024-02-06T16:59:49.261Z }
6092024-02-06T16:59:49.261Z /^\[rack_network_config/,/^$/ {
6102024-02-06T16:59:49.261Z /^infra_ip_first/c\
6112024-02-06T16:59:49.261Zinfra_ip_first = "10.151.1.174"
6122024-02-06T16:59:49.261Z /^infra_ip_last/c\
6132024-02-06T16:59:49.261Zinfra_ip_last = "10.151.1.174"
6142024-02-06T16:59:49.261Z }
6152024-02-06T16:59:49.261Z /^\[\[rack_network_config.ports/,/^$/ {
6162024-02-06T16:59:49.261Z /^routes/c\
6172024-02-06T16:59:49.261Zroutes = \[{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}\]
6182024-02-06T16:59:49.261Z /^addresses/c\
6192024-02-06T16:59:49.261Zaddresses = \["10.151.1.174/24"\]
6202024-02-06T16:59:49.261Z }
6212024-02-06T16:59:49.261Z' pkg/config-rss.toml
6222024-02-06T16:59:49.261Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
6232024-02-06T16:59:49.286Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
6242024-02-06T16:59:49.286Z+++ pkg/config-rss.toml Tue Feb 6 16:59:48 2024
6252024-02-06T16:59:49.286Z@@ -41,7 +41,7 @@
6262024-02-06T16:59:49.286Z # the DNS domain delegated to the rack by the customer. Each of these addresses
6272024-02-06T16:59:49.286Z # must be contained in one of the "internal services" IP Pool ranges listed
6282024-02-06T16:59:49.286Z # below.
6292024-02-06T16:59:49.286Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
6302024-02-06T16:59:49.286Z+external_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
6312024-02-06T16:59:49.286Z
6322024-02-06T16:59:49.286Z # Initial TLS certificates for the external API
6332024-02-06T16:59:49.287Z #
6342024-02-06T16:59:49.287Z@@ -77,8 +77,8 @@
6352024-02-06T16:59:49.287Z #
6362024-02-06T16:59:49.287Z # For more on this and what to put here, see docs/how-to-run.adoc.
6372024-02-06T16:59:49.287Z [[internal_services_ip_pool_ranges]]
6382024-02-06T16:59:49.287Z-first = "192.168.1.20"
6392024-02-06T16:59:49.287Z-last = "192.168.1.29"
6402024-02-06T16:59:49.287Z+first = "10.151.1.164"
6412024-02-06T16:59:49.287Z+last = "10.151.1.173"
6422024-02-06T16:59:49.287Z
6432024-02-06T16:59:49.287Z # TODO - this configuration is subject to change going forward. Ultimately these
6442024-02-06T16:59:49.287Z # parameters should be provided to the control plane via wicket, but we need to
6452024-02-06T16:59:49.287Z@@ -92,8 +92,8 @@
6462024-02-06T16:59:49.287Z # A range of IP addresses used by Boundary Services on the external network. In
6472024-02-06T16:59:49.287Z # a real system, these would be addresses of the uplink ports on the Sidecar.
6482024-02-06T16:59:49.287Z # With softnpu, only one address is used.
6492024-02-06T16:59:49.287Z-infra_ip_first = "192.168.1.30"
6502024-02-06T16:59:49.287Z-infra_ip_last = "192.168.1.30"
6512024-02-06T16:59:49.287Z+infra_ip_first = "10.151.1.174"
6522024-02-06T16:59:49.287Z+infra_ip_last = "10.151.1.174"
6532024-02-06T16:59:49.287Z
6542024-02-06T16:59:49.287Z # Configurations for BGP routers to run on the scrimlets.
6552024-02-06T16:59:49.287Z bgp = []
6562024-02-06T16:59:49.287Z@@ -101,9 +101,9 @@
6572024-02-06T16:59:49.288Z # You can configure multiple uplinks by repeating the following stanza
6582024-02-06T16:59:49.288Z [[rack_network_config.ports]]
6592024-02-06T16:59:49.288Z # Routes associated with this port.
6602024-02-06T16:59:49.288Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
6612024-02-06T16:59:49.288Z+routes = [{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}]
6622024-02-06T16:59:49.288Z # Addresses associated with this port.
6632024-02-06T16:59:49.288Z-addresses = ["192.168.1.30/24"]
6642024-02-06T16:59:49.288Z+addresses = ["10.151.1.174/24"]
6652024-02-06T16:59:49.288Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
6662024-02-06T16:59:49.288Z port = "qsfp0"
6672024-02-06T16:59:49.288Z # The speed of this port.
6682024-02-06T16:59:49.288Z+ true
6692024-02-06T16:59:49.288Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml
6702024-02-06T16:59:49.288Za pkg/config-rss.toml 7K
6712024-02-06T16:59:49.288Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~
6722024-02-06T16:59:49.288Z+ TLS_NAME=recovery.sys.oxide.test
6732024-02-06T16:59:49.288Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
6742024-02-06T16:59:49.785Z....+...+.......+..............+.+..+.+.........+..+.+..+...+....+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+......+..+.+.....+.+...+..+.+..+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+...+......+........+.+..+...+.........+.+.........+..+...+................+..+..........+........+....+..+......+.......+..+....+..+....+.....................+..+.+...........+....+............+........+.+......+.....+.......+................................+...+.+.........+.....+............................+..............+......+..................+.......+.....+.........+......+....+........+.+.....+......+...............+...+.......+........+...+..........+.....+.......+............+...+..+..........+......+.....+...................+.........+...........+.+.....+.........+......+...+....+...+..+..........+..+....+.....+................+.......................+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6752024-02-06T16:59:50.465Z...........+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+....+...+......+.........+..+....+..+............+.+..........................+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+.........+...............+..........................+.+...+..+....+......+.........+...........+...+...................+..+...+...+..........+.................+.+............+...+...........+...............+................+............+......+............+...+......+.....+..................+.......+.....+.+...............+..................+..+....+...........+...+........................+.+.....+.......+..............+...+.............+...+..............+............+...+.+.....+.........+.+......+...+..+.+.........+........+.......+..............+.+...+......+..+......................+..................+..+...+.........+..........+.....+.............+..................+...+..............+...+....+.........+......+........+...+...+.+...+..+.+..............+......+.........+...+.....................+.........+............+......+.......+.....+.+.....+......+.+...............+............+.....+............+.+..............+....+..+....+...........+..........+.........+...+............+.....+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6762024-02-06T16:59:50.491Z-----
6772024-02-06T16:59:50.491Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6782024-02-06T16:59:50.515Za pkg/initial-tls-cert.pem 2K
6792024-02-06T16:59:50.515Za pkg/initial-tls-key.pem 4K
6802024-02-06T16:59:50.515Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6812024-02-06T16:59:50.515Z+ rmdir pkg
6822024-02-06T16:59:50.515Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
6832024-02-06T16:59:50.515Z+ pfexec mkdir -p /usr/oxide
6842024-02-06T16:59:50.515Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0
6852024-02-06T16:59:51.250Z+ pfexec chmod +x /usr/oxide/oxide
6862024-02-06T16:59:51.274Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
6872024-02-06T17:00:09.575Z+ OMICRON_NO_UNINSTALL=1
6882024-02-06T17:00:09.575Z+ ptime -m pfexec ./target/release/omicron-package -t test install
6892024-02-06T17:00:09.599ZLogging to: /opt/oxide/work/out/LOG
6902024-02-06T17:00:10.083Z
6912024-02-06T17:00:10.107Zreal 0.503147595
6922024-02-06T17:00:10.107Zuser 0.108410581
6932024-02-06T17:00:10.107Zsys 1.982863604
6942024-02-06T17:00:10.107Ztrap 0.000000250
6952024-02-06T17:00:10.108Ztflt 0.000029198
6962024-02-06T17:00:10.108Zdflt 0.000633886
6972024-02-06T17:00:10.108Zkflt 0.000000000
6982024-02-06T17:00:10.108Zlock 45.707074790
6992024-02-06T17:00:10.108Zslp 0.707830458
7002024-02-06T17:00:10.108Zlat 0.003135090
7012024-02-06T17:00:10.108Zstop 0.195728872
7022024-02-06T17:00:10.108Z+ retry=0
7032024-02-06T17:00:10.108Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7042024-02-06T17:00:10.108Z+ [[ 0 -gt 30 ]]
7052024-02-06T17:00:10.108Z+ sleep 1
7062024-02-06T17:00:11.100Z+ retry=1
7072024-02-06T17:00:11.101Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7082024-02-06T17:00:11.125Z+ [[ 1 -gt 30 ]]
7092024-02-06T17:00:11.125Z+ sleep 1
7102024-02-06T17:00:12.112Z+ retry=2
7112024-02-06T17:00:12.112Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7122024-02-06T17:00:12.137Z+ [[ 2 -gt 30 ]]
7132024-02-06T17:00:12.137Z+ sleep 1
7142024-02-06T17:00:13.126Z+ retry=3
7152024-02-06T17:00:13.126Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7162024-02-06T17:00:13.151Z+ [[ 3 -gt 30 ]]
7172024-02-06T17:00:13.151Z+ sleep 1
7182024-02-06T17:00:14.139Z+ retry=4
7192024-02-06T17:00:14.139Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7202024-02-06T17:00:39.312Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
7212024-02-06T17:00:39.338Zlocal v6:
7222024-02-06T17:00:39.338Zfd7a:a711:ff32:fc13::1
7232024-02-06T17:00:39.338Zfe80::f014:a6ff:fe63:f908
7242024-02-06T17:00:39.338Zlocal v4:
7252024-02-06T17:00:39.338Z10.151.1.174
7262024-02-06T17:00:39.338Zrouter v6:
7272024-02-06T17:00:39.338Zfd00:1122:3344:1::/64 -> fe80::dca8:ffff:fe12:8864 (1)
7282024-02-06T17:00:39.338Zfd00:1122:3344:3::/64 -> fe80::dca8:ffff:fe12:8864 (1)
7292024-02-06T17:00:39.338Zfd00:1122:3344:101::/64 -> fe80::dca8:ffff:fe12:8864 (1)
7302024-02-06T17:00:39.338Zfd00:1122:3344:2::/64 -> fe80::dca8:ffff:fe12:8864 (1)
7312024-02-06T17:00:39.339Zfdb0:18c0:4d0d:9fb2::/64 -> fe80::dca8:ffff:fe12:8864 (1)
7322024-02-06T17:00:39.339Zrouter v4_idx:
7332024-02-06T17:00:39.339Z0.0.0.0/0 -> 0
7342024-02-06T17:00:39.339Zrouter v4_routes:
7352024-02-06T17:00:39.339Z0 -> 10.151.1.100 (2)
7362024-02-06T17:00:39.339Zresolver v4:
7372024-02-06T17:00:39.363Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
7382024-02-06T17:00:39.363Z10.151.1.174 -> e2:51:68:18:79:d9
7392024-02-06T17:00:39.363Zresolver v6:
7402024-02-06T17:00:39.363Zfe80::f014:a6ff:fe63:f908 -> f2:14:a6:63:f9:08
7412024-02-06T17:00:39.363Zfe80::dca8:ffff:fe12:8864 -> de:a8:ff:12:88:64
7422024-02-06T17:00:39.363Znat v4:
7432024-02-06T17:00:39.363Znat_v6:
7442024-02-06T17:00:39.363Zport_mac:
7452024-02-06T17:00:39.363Z1000: a8:40:25:59:18:96
7462024-02-06T17:00:39.363Z1: a8:40:25:59:18:97
7472024-02-06T17:00:39.363Zproxy arp:
7482024-02-06T17:00:39.363Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
7492024-02-06T17:00:39.363Z+ retry=0
7502024-02-06T17:00:39.363Z+++ zoneadm list -n
7512024-02-06T17:00:39.363Z+++ grep oxz_ntp
7522024-02-06T17:00:39.363Z++ pfexec svcs -z -Hostate oxide/ntp
7532024-02-06T17:00:39.363Zsvcs: zone '-Hostate': entity not found
7542024-02-06T17:00:39.363Z++ true
7552024-02-06T17:00:39.364Z+ [[ '' != online ]]
7562024-02-06T17:00:39.364Z+ [[ 0 -gt 60 ]]
7572024-02-06T17:00:39.364Z+ sleep 1
7582024-02-06T17:00:40.363Z+ retry=1
7592024-02-06T17:00:40.388Z+++ zoneadm list -n
7602024-02-06T17:00:40.388Z+++ grep oxz_ntp
7612024-02-06T17:00:40.388Z++ pfexec svcs -z -Hostate oxide/ntp
7622024-02-06T17:00:40.388Zsvcs: zone '-Hostate': entity not found
7632024-02-06T17:00:40.388Z++ true
7642024-02-06T17:00:40.388Z+ [[ '' != online ]]
7652024-02-06T17:00:40.388Z+ [[ 1 -gt 60 ]]
7662024-02-06T17:00:40.388Z+ sleep 1
7672024-02-06T17:00:41.387Z+ retry=2
7682024-02-06T17:00:41.411Z+++ zoneadm list -n
7692024-02-06T17:00:41.411Z+++ grep oxz_ntp
7702024-02-06T17:00:41.411Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
7712024-02-06T17:00:41.449Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7722024-02-06T17:00:41.474Z++ true
7732024-02-06T17:00:41.474Z+ [[ '' != online ]]
7742024-02-06T17:00:41.474Z+ [[ 2 -gt 60 ]]
7752024-02-06T17:00:41.474Z+ sleep 1
7762024-02-06T17:00:42.460Z+ retry=3
7772024-02-06T17:00:42.460Z+++ zoneadm list -n
7782024-02-06T17:00:42.460Z+++ grep oxz_ntp
7792024-02-06T17:00:42.484Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
7802024-02-06T17:00:42.484Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7812024-02-06T17:00:42.484Z++ true
7822024-02-06T17:00:42.484Z+ [[ '' != online ]]
7832024-02-06T17:00:42.484Z+ [[ 3 -gt 60 ]]
7842024-02-06T17:00:42.484Z+ sleep 1
7852024-02-06T17:00:43.483Z+ retry=4
7862024-02-06T17:00:43.507Z+++ zoneadm list -n
7872024-02-06T17:00:43.507Z+++ grep oxz_ntp
7882024-02-06T17:00:43.507Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
7892024-02-06T17:00:43.531Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7902024-02-06T17:00:43.531Z++ true
7912024-02-06T17:00:43.531Z+ [[ '' != online ]]
7922024-02-06T17:00:43.531Z+ [[ 4 -gt 60 ]]
7932024-02-06T17:00:43.531Z+ sleep 1
7942024-02-06T17:00:44.513Z+ retry=5
7952024-02-06T17:00:44.538Z+++ zoneadm list -n
7962024-02-06T17:00:44.538Z+++ grep oxz_ntp
7972024-02-06T17:00:44.538Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
7982024-02-06T17:00:44.562Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7992024-02-06T17:00:44.562Z++ true
8002024-02-06T17:00:44.562Z+ [[ '' != online ]]
8012024-02-06T17:00:44.562Z+ [[ 5 -gt 60 ]]
8022024-02-06T17:00:44.562Z+ sleep 1
8032024-02-06T17:00:45.545Z+ retry=6
8042024-02-06T17:00:45.569Z+++ zoneadm list -n
8052024-02-06T17:00:45.569Z+++ grep oxz_ntp
8062024-02-06T17:00:45.569Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
8072024-02-06T17:00:45.569Z+ [[ uninitialized != online ]]
8082024-02-06T17:00:45.569Z+ [[ 6 -gt 60 ]]
8092024-02-06T17:00:45.569Z+ sleep 1
8102024-02-06T17:00:46.568Z+ retry=7
8112024-02-06T17:00:46.592Z+++ zoneadm list -n
8122024-02-06T17:00:46.592Z+++ grep oxz_ntp
8132024-02-06T17:00:46.592Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
8142024-02-06T17:00:46.592Z+ [[ disabled != online ]]
8152024-02-06T17:00:46.592Z+ [[ 7 -gt 60 ]]
8162024-02-06T17:00:46.592Z+ sleep 1
8172024-02-06T17:00:47.590Z+ retry=8
8182024-02-06T17:00:47.614Z+++ zoneadm list -n
8192024-02-06T17:00:47.614Z+++ grep oxz_ntp
8202024-02-06T17:00:47.614Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
8212024-02-06T17:00:47.614Z+ [[ disabled != online ]]
8222024-02-06T17:00:47.614Z+ [[ 8 -gt 60 ]]
8232024-02-06T17:00:47.614Z+ sleep 1
8242024-02-06T17:00:48.613Z+ retry=9
8252024-02-06T17:00:48.637Z+++ zoneadm list -n
8262024-02-06T17:00:48.637Z+++ grep oxz_ntp
8272024-02-06T17:00:48.637Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
8282024-02-06T17:00:48.637Z+ [[ disabled != online ]]
8292024-02-06T17:00:48.637Z+ [[ 9 -gt 60 ]]
8302024-02-06T17:00:48.637Z+ sleep 1
8312024-02-06T17:00:49.637Z+ retry=10
8322024-02-06T17:00:49.662Z+++ zoneadm list -n
8332024-02-06T17:00:49.662Z+++ grep oxz_ntp
8342024-02-06T17:00:49.662Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
8352024-02-06T17:00:49.662Z+ [[ disabled != online ]]
8362024-02-06T17:00:49.662Z+ [[ 10 -gt 60 ]]
8372024-02-06T17:00:49.662Z+ sleep 1
8382024-02-06T17:00:50.662Z+ retry=11
8392024-02-06T17:00:50.688Z+++ zoneadm list -n
8402024-02-06T17:00:50.688Z+++ grep oxz_ntp
8412024-02-06T17:00:50.688Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
8422024-02-06T17:00:50.688Z+ [[ disabled != online ]]
8432024-02-06T17:00:50.688Z+ [[ 11 -gt 60 ]]
8442024-02-06T17:00:50.688Z+ sleep 1
8452024-02-06T17:00:51.685Z+ retry=12
8462024-02-06T17:00:51.709Z+++ zoneadm list -n
8472024-02-06T17:00:51.709Z+++ grep oxz_ntp
8482024-02-06T17:00:51.710Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
8492024-02-06T17:00:51.710Z+ [[ disabled != online ]]
8502024-02-06T17:00:51.710Z+ [[ 12 -gt 60 ]]
8512024-02-06T17:00:51.710Z+ sleep 1
8522024-02-06T17:00:52.709Z+ retry=13
8532024-02-06T17:00:52.733Z+++ zoneadm list -n
8542024-02-06T17:00:52.733Z+++ grep oxz_ntp
8552024-02-06T17:00:52.733Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
8562024-02-06T17:00:52.733Z+ [[ disabled != online ]]
8572024-02-06T17:00:52.733Z+ [[ 13 -gt 60 ]]
8582024-02-06T17:00:52.733Z+ sleep 1
 
8592024-02-06T17:00:53.633Zbuskin console login: Feb 6 17:00:52 buskin ip: ip: joining multicasts failed (4) on vopte0 - will use link layer broadcasts for multicast
 
8602024-02-06T17:00:53.731Z+ retry=14
8612024-02-06T17:00:53.755Z+++ zoneadm list -n
8622024-02-06T17:00:53.755Z+++ grep oxz_ntp
8632024-02-06T17:00:53.755Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
8642024-02-06T17:00:53.755Z+ [[ offline* != online ]]
8652024-02-06T17:00:53.755Z+ [[ 14 -gt 60 ]]
8662024-02-06T17:00:53.755Z+ sleep 1
8672024-02-06T17:00:54.754Z+ retry=15
8682024-02-06T17:00:54.778Z+++ zoneadm list -n
8692024-02-06T17:00:54.778Z+++ grep oxz_ntp
8702024-02-06T17:00:54.778Z++ pfexec svcs -z oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86 -Hostate oxide/ntp
8712024-02-06T17:00:54.778Z+ [[ online != online ]]
8722024-02-06T17:00:54.778Z+ echo 'Waited for chrony: 15s'
8732024-02-06T17:00:54.778ZWaited for chrony: 15s
8742024-02-06T17:00:54.779Z+ export RUST_BACKTRACE=1
8752024-02-06T17:00:54.779Z+ RUST_BACKTRACE=1
8762024-02-06T17:00:54.779Z+ export E2E_TLS_CERT IPPOOL_START IPPOOL_END
8772024-02-06T17:00:54.779Z++ ./tests/bootstrap
8782024-02-06T17:00:54.806ZUsing external DNS server at 10.151.1.164:53
8792024-02-06T17:00:54.806Z2024-02-06 17:00:54.322777481 UTC: attempting to log into API
8802024-02-06T17:01:09.841Z2024-02-06 17:01:10.171103532 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8812024-02-06T17:01:10.842Z2024-02-06 17:01:11.170074510 UTC: attempting to log into API
8822024-02-06T17:01:25.874Z2024-02-06 17:01:26.161299659 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8832024-02-06T17:01:26.875Z2024-02-06 17:01:27.159338711 UTC: attempting to log into API
8842024-02-06T17:01:41.907Z2024-02-06 17:01:42.150567431 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8852024-02-06T17:01:42.907Z2024-02-06 17:01:43.148522411 UTC: attempting to log into API
8862024-02-06T17:01:57.938Z2024-02-06 17:01:58.139070564 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8872024-02-06T17:01:58.939Z2024-02-06 17:01:59.137134227 UTC: attempting to log into API
8882024-02-06T17:02:13.970Z2024-02-06 17:02:14.127510594 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8892024-02-06T17:02:14.971Z2024-02-06 17:02:15.125658499 UTC: attempting to log into API
8902024-02-06T17:02:30.002Z2024-02-06 17:02:30.116262703 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8912024-02-06T17:02:31.003Z2024-02-06 17:02:31.114439309 UTC: attempting to log into API
8922024-02-06T17:02:46.035Z2024-02-06 17:02:46.103917035 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8932024-02-06T17:02:47.036Z2024-02-06 17:02:47.102312859 UTC: attempting to log into API
8942024-02-06T17:03:02.068Z2024-02-06 17:03:02.092179206 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8952024-02-06T17:03:03.069Z2024-02-06 17:03:03.090237031 UTC: attempting to log into API
8962024-02-06T17:03:18.101Z2024-02-06 17:03:18.083675429 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8972024-02-06T17:03:19.103Z2024-02-06 17:03:19.085359780 UTC: attempting to log into API
8982024-02-06T17:03:34.137Z2024-02-06 17:03:34.115649890 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8992024-02-06T17:03:35.135Z2024-02-06 17:03:35.116549548 UTC: attempting to log into API
9002024-02-06T17:03:50.167Z2024-02-06 17:03:50.148500906 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9012024-02-06T17:03:51.168Z2024-02-06 17:03:51.149320978 UTC: attempting to log into API
9022024-02-06T17:04:06.200Z2024-02-06 17:04:06.182491837 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9032024-02-06T17:04:07.202Z2024-02-06 17:04:07.184261336 UTC: attempting to log into API
9042024-02-06T17:04:22.236Z2024-02-06 17:04:22.219182294 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9052024-02-06T17:04:23.238Z2024-02-06 17:04:23.220882762 UTC: attempting to log into API
9062024-02-06T17:04:38.270Z2024-02-06 17:04:38.253300200 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9072024-02-06T17:04:39.270Z2024-02-06 17:04:39.254462893 UTC: attempting to log into API
 
9082024-02-06T17:04:53.968ZFeb 6 17:04:53 buskin ip: ip: joining multicasts failed (4) on vopte3 - will use link layer broadcasts for multicast
 
9092024-02-06T17:04:54.305Z2024-02-06 17:04:54.289818617 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9102024-02-06T17:04:55.307Z2024-02-06 17:04:55.291055270 UTC: attempting to log into API
 
9112024-02-06T17:05:01.238ZFeb 6 17:05:00 buskin ip: ip: joining multicasts failed (4) on vopte2 - will use link layer broadcasts for multicast
9122024-02-06T17:05:08.241ZFeb 6 17:05:07 buskin ip: ip: joining multicasts failed (4) on vopte4 - will use link layer broadcasts for multicast
 
9132024-02-06T17:05:10.336Z2024-02-06 17:05:10.323193664 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9142024-02-06T17:05:11.338Z2024-02-06 17:05:11.324438917 UTC: attempting to log into API
 
9152024-02-06T17:05:14.356ZFeb 6 17:05:13 buskin ip: ip: joining multicasts failed (4) on vopte5 - will use link layer broadcasts for multicast
9162024-02-06T17:05:21.616ZFeb 6 17:05:20 buskin ip: ip: joining multicasts failed (4) on vopte1 - will use link layer broadcasts for multicast
 
9172024-02-06T17:05:26.370Z2024-02-06 17:05:26.358161393 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9182024-02-06T17:05:27.370Z2024-02-06 17:05:27.359330694 UTC: attempting to log into API
9192024-02-06T17:05:38.168Z2024-02-06 17:05:38.158612242 UTC: login succeeded
9202024-02-06T17:05:38.200Zwaiting for nexus to come up...
9212024-02-06T17:05:38.271Zcreating IP pool... 10.151.1.175 - 10.151.1.227
9222024-02-06T17:05:38.444Zsetting up quotas...
9232024-02-06T17:05:38.484Zensuring datasets are ready...
9242024-02-06T17:05:41.130Zlet's roll.
9252024-02-06T17:05:41.154Z+ eval 'OXIDE_HOST=https://recovery.sys.oxide.test
9262024-02-06T17:05:41.155ZOXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
9272024-02-06T17:05:41.155ZOXIDE_TOKEN=oxide-token-9475981be69700875d773b72dae3fbb9bdf2f412'
9282024-02-06T17:05:41.155Z++ OXIDE_HOST=https://recovery.sys.oxide.test
9292024-02-06T17:05:41.155Z++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
9302024-02-06T17:05:41.155Z++ OXIDE_TOKEN=oxide-token-9475981be69700875d773b72dae3fbb9bdf2f412
9312024-02-06T17:05:41.155Z+ export OXIDE_HOST OXIDE_TOKEN
9322024-02-06T17:05:41.155Z+ retry=0
9332024-02-06T17:05:41.155Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9342024-02-06T17:05:41.179Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9352024-02-06T17:05:41.179Z+ [[ 0 -gt 60 ]]
9362024-02-06T17:05:41.179Z+ sleep 1
9372024-02-06T17:05:42.161Z+ retry=1
9382024-02-06T17:05:42.161Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9392024-02-06T17:05:42.185Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9402024-02-06T17:05:42.185Z+ [[ 1 -gt 60 ]]
9412024-02-06T17:05:42.185Z+ sleep 1
9422024-02-06T17:05:43.177Z+ retry=2
9432024-02-06T17:05:43.177Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9442024-02-06T17:05:43.201Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9452024-02-06T17:05:43.201Z+ [[ 2 -gt 60 ]]
9462024-02-06T17:05:43.201Z+ sleep 1
9472024-02-06T17:05:44.195Z+ retry=3
9482024-02-06T17:05:44.195Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9492024-02-06T17:05:44.219Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9502024-02-06T17:05:44.219Z+ [[ 3 -gt 60 ]]
9512024-02-06T17:05:44.219Z+ sleep 1
9522024-02-06T17:05:45.209Z+ retry=4
9532024-02-06T17:05:45.209Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9542024-02-06T17:05:45.233Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9552024-02-06T17:05:45.233Z+ [[ 4 -gt 60 ]]
9562024-02-06T17:05:45.233Z+ sleep 1
9572024-02-06T17:05:46.226Z+ retry=5
9582024-02-06T17:05:46.226Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9592024-02-06T17:05:46.250Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9602024-02-06T17:05:46.250Z+ [[ 5 -gt 60 ]]
9612024-02-06T17:05:46.250Z+ sleep 1
9622024-02-06T17:05:47.242Z+ retry=6
9632024-02-06T17:05:47.242Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9642024-02-06T17:05:47.267Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9652024-02-06T17:05:47.267Z+ [[ 6 -gt 60 ]]
9662024-02-06T17:05:47.267Z+ sleep 1
9672024-02-06T17:05:48.258Z+ retry=7
9682024-02-06T17:05:48.258Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9692024-02-06T17:05:48.282Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9702024-02-06T17:05:48.282Z+ [[ 7 -gt 60 ]]
9712024-02-06T17:05:48.282Z+ sleep 1
9722024-02-06T17:05:49.275Z+ retry=8
9732024-02-06T17:05:49.275Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9742024-02-06T17:05:49.301Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9752024-02-06T17:05:49.301Z+ [[ 8 -gt 60 ]]
9762024-02-06T17:05:49.301Z+ sleep 1
9772024-02-06T17:05:50.292Z+ retry=9
9782024-02-06T17:05:50.292Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9792024-02-06T17:05:50.316Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9802024-02-06T17:05:50.317Z+ [[ 9 -gt 60 ]]
9812024-02-06T17:05:50.317Z+ sleep 1
9822024-02-06T17:05:51.307Z+ retry=10
9832024-02-06T17:05:51.307Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9842024-02-06T17:05:51.331Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9852024-02-06T17:05:51.331Z+ [[ 10 -gt 60 ]]
9862024-02-06T17:05:51.331Z+ sleep 1
9872024-02-06T17:05:52.322Z+ retry=11
9882024-02-06T17:05:52.323Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9892024-02-06T17:05:52.346Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9902024-02-06T17:05:52.346Z+ [[ 11 -gt 60 ]]
9912024-02-06T17:05:52.346Z+ sleep 1
9922024-02-06T17:05:53.338Z+ retry=12
9932024-02-06T17:05:53.339Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9942024-02-06T17:05:53.362Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9952024-02-06T17:05:53.362Z+ [[ 12 -gt 60 ]]
9962024-02-06T17:05:53.362Z+ sleep 1
9972024-02-06T17:05:54.354Z+ retry=13
9982024-02-06T17:05:54.354Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9992024-02-06T17:05:54.381Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10002024-02-06T17:05:54.381Z+ [[ 13 -gt 60 ]]
10012024-02-06T17:05:54.381Z+ sleep 1
10022024-02-06T17:05:55.370Z+ retry=14
10032024-02-06T17:05:55.370Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10042024-02-06T17:05:55.394Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10052024-02-06T17:05:55.394Z+ [[ 14 -gt 60 ]]
10062024-02-06T17:05:55.394Z+ sleep 1
10072024-02-06T17:05:56.386Z+ retry=15
10082024-02-06T17:05:56.387Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10092024-02-06T17:05:56.411Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10102024-02-06T17:05:56.411Z+ [[ 15 -gt 60 ]]
10112024-02-06T17:05:56.411Z+ sleep 1
10122024-02-06T17:05:57.403Z+ retry=16
10132024-02-06T17:05:57.403Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10142024-02-06T17:05:57.430Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10152024-02-06T17:05:57.430Z+ [[ 16 -gt 60 ]]
10162024-02-06T17:05:57.430Z+ sleep 1
10172024-02-06T17:05:58.420Z+ retry=17
10182024-02-06T17:05:58.420Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10192024-02-06T17:05:58.445Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10202024-02-06T17:05:58.445Z+ [[ 17 -gt 60 ]]
10212024-02-06T17:05:58.445Z+ sleep 1
10222024-02-06T17:05:59.437Z+ retry=18
10232024-02-06T17:05:59.437Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10242024-02-06T17:05:59.462Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10252024-02-06T17:05:59.462Z+ [[ 18 -gt 60 ]]
10262024-02-06T17:05:59.462Z+ sleep 1
10272024-02-06T17:06:00.454Z+ retry=19
10282024-02-06T17:06:00.454Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10292024-02-06T17:06:00.478Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10302024-02-06T17:06:00.479Z+ [[ 19 -gt 60 ]]
10312024-02-06T17:06:00.479Z+ sleep 1
10322024-02-06T17:06:01.474Z+ retry=20
10332024-02-06T17:06:01.474Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10342024-02-06T17:06:01.501Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10352024-02-06T17:06:01.501Z+ [[ 20 -gt 60 ]]
10362024-02-06T17:06:01.501Z+ sleep 1
10372024-02-06T17:06:02.490Z+ retry=21
10382024-02-06T17:06:02.490Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10392024-02-06T17:06:02.516Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10402024-02-06T17:06:02.516Z+ [[ 21 -gt 60 ]]
10412024-02-06T17:06:02.516Z+ sleep 1
10422024-02-06T17:06:03.504Z+ retry=22
10432024-02-06T17:06:03.504Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10442024-02-06T17:06:03.529Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10452024-02-06T17:06:03.530Z+ [[ 22 -gt 60 ]]
10462024-02-06T17:06:03.530Z+ sleep 1
10472024-02-06T17:06:04.520Z+ retry=23
10482024-02-06T17:06:04.520Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10492024-02-06T17:06:04.545Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10502024-02-06T17:06:04.545Z+ [[ 23 -gt 60 ]]
10512024-02-06T17:06:04.545Z+ sleep 1
10522024-02-06T17:06:05.537Z+ retry=24
10532024-02-06T17:06:05.537Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10542024-02-06T17:06:05.562Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10552024-02-06T17:06:05.562Z+ [[ 24 -gt 60 ]]
10562024-02-06T17:06:05.562Z+ sleep 1
10572024-02-06T17:06:06.553Z+ retry=25
10582024-02-06T17:06:06.553Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10592024-02-06T17:06:06.691Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10602024-02-06T17:06:06.691Z+ [[ 25 -gt 60 ]]
10612024-02-06T17:06:06.691Z+ sleep 1
10622024-02-06T17:06:07.569Z+ retry=26
10632024-02-06T17:06:07.569Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10642024-02-06T17:06:07.599Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10652024-02-06T17:06:07.599Z+ [[ 26 -gt 60 ]]
10662024-02-06T17:06:07.599Z+ sleep 1
10672024-02-06T17:06:08.587Z+ retry=27
10682024-02-06T17:06:08.587Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10692024-02-06T17:06:08.611Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10702024-02-06T17:06:08.611Z+ [[ 27 -gt 60 ]]
10712024-02-06T17:06:08.611Z+ sleep 1
10722024-02-06T17:06:09.603Z+ retry=28
10732024-02-06T17:06:09.603Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10742024-02-06T17:06:09.628Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10752024-02-06T17:06:09.628Z+ [[ 28 -gt 60 ]]
10762024-02-06T17:06:09.628Z+ sleep 1
10772024-02-06T17:06:10.620Z+ retry=29
10782024-02-06T17:06:10.620Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10792024-02-06T17:06:10.644Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10802024-02-06T17:06:10.644Z+ [[ 29 -gt 60 ]]
10812024-02-06T17:06:10.644Z+ sleep 1
10822024-02-06T17:06:11.635Z+ retry=30
10832024-02-06T17:06:11.635Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10842024-02-06T17:06:11.660Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10852024-02-06T17:06:11.660Z+ [[ 30 -gt 60 ]]
10862024-02-06T17:06:11.660Z+ sleep 1
10872024-02-06T17:06:12.652Z+ retry=31
10882024-02-06T17:06:12.652Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10892024-02-06T17:06:12.677Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10902024-02-06T17:06:12.677Z+ [[ 31 -gt 60 ]]
10912024-02-06T17:06:12.677Z+ sleep 1
10922024-02-06T17:06:13.669Z+ retry=32
10932024-02-06T17:06:13.669Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10942024-02-06T17:06:13.693Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10952024-02-06T17:06:13.693Z+ [[ 32 -gt 60 ]]
10962024-02-06T17:06:13.694Z+ sleep 1
10972024-02-06T17:06:14.685Z+ retry=33
10982024-02-06T17:06:14.685Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10992024-02-06T17:06:14.709Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11002024-02-06T17:06:14.709Z+ [[ 33 -gt 60 ]]
11012024-02-06T17:06:14.709Z+ sleep 1
11022024-02-06T17:06:15.699Z+ retry=34
11032024-02-06T17:06:15.700Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11042024-02-06T17:06:15.725Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11052024-02-06T17:06:15.725Z+ [[ 34 -gt 60 ]]
11062024-02-06T17:06:15.725Z+ sleep 1
11072024-02-06T17:06:16.717Z+ retry=35
11082024-02-06T17:06:16.717Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11092024-02-06T17:06:16.742Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11102024-02-06T17:06:16.742Z+ [[ 35 -gt 60 ]]
11112024-02-06T17:06:16.742Z+ sleep 1
11122024-02-06T17:06:17.733Z+ retry=36
11132024-02-06T17:06:17.734Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11142024-02-06T17:06:17.758Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11152024-02-06T17:06:17.758Z+ [[ 36 -gt 60 ]]
11162024-02-06T17:06:17.758Z+ sleep 1
11172024-02-06T17:06:18.750Z+ retry=37
11182024-02-06T17:06:18.750Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11192024-02-06T17:06:18.774Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11202024-02-06T17:06:18.774Z+ [[ 37 -gt 60 ]]
11212024-02-06T17:06:18.774Z+ sleep 1
11222024-02-06T17:06:19.766Z+ retry=38
11232024-02-06T17:06:19.766Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11242024-02-06T17:06:19.791Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11252024-02-06T17:06:19.791Z+ [[ 38 -gt 60 ]]
11262024-02-06T17:06:19.791Z+ sleep 1
11272024-02-06T17:06:20.784Z+ retry=39
11282024-02-06T17:06:20.784Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11292024-02-06T17:06:20.809Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11302024-02-06T17:06:20.809Z+ [[ 39 -gt 60 ]]
11312024-02-06T17:06:20.809Z+ sleep 1
11322024-02-06T17:06:21.818Z+ retry=40
11332024-02-06T17:06:21.818Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11342024-02-06T17:06:21.842Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11352024-02-06T17:06:21.842Z+ [[ 40 -gt 60 ]]
11362024-02-06T17:06:21.842Z+ sleep 1
11372024-02-06T17:06:22.816Z+ retry=41
11382024-02-06T17:06:22.816Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11392024-02-06T17:06:22.901Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images'
11402024-02-06T17:06:23.336Z{"status":"ok"}success
11412024-02-06T17:06:23.336ZProject {
11422024-02-06T17:06:23.336Z description: "some images",
11432024-02-06T17:06:23.336Z id: 0569f896-92e6-4421-a507-e4ba388c40ef,
11442024-02-06T17:06:23.337Z name: Name(
11452024-02-06T17:06:23.337Z "images",
11462024-02-06T17:06:23.337Z ),
11472024-02-06T17:06:23.337Z time_created: 2024-02-06T17:06:22.993187Z,
11482024-02-06T17:06:23.337Z time_modified: 2024-02-06T17:06:22.993187Z,
11492024-02-06T17:06:23.337Z}
11502024-02-06T17:06:23.361Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11
11512024-02-06T17:12:07.037Zdone!
11522024-02-06T17:12:07.137Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11
11532024-02-06T17:12:07.252Zsuccess
11542024-02-06T17:12:07.252ZImage {
11552024-02-06T17:12:07.252Z block_size: ByteCount(
11562024-02-06T17:12:07.252Z 512,
11572024-02-06T17:12:07.252Z ),
11582024-02-06T17:12:07.276Z description: "debian 11 original base image",
11592024-02-06T17:12:07.276Z digest: None,
11602024-02-06T17:12:07.276Z id: bc23340c-42d2-414c-a96e-b3a2e6f36f8f,
11612024-02-06T17:12:07.276Z name: Name(
11622024-02-06T17:12:07.276Z "debian11",
11632024-02-06T17:12:07.276Z ),
11642024-02-06T17:12:07.276Z os: "debian",
11652024-02-06T17:12:07.276Z project_id: None,
11662024-02-06T17:12:07.277Z size: ByteCount(
11672024-02-06T17:12:07.277Z 2147483648,
11682024-02-06T17:12:07.277Z ),
11692024-02-06T17:12:07.277Z time_created: 2024-02-06T17:12:06.995931Z,
11702024-02-06T17:12:07.277Z time_modified: 2024-02-06T17:12:07.229166Z,
11712024-02-06T17:12:07.277Z url: None,
11722024-02-06T17:12:07.277Z version: "11",
11732024-02-06T17:12:07.277Z}
11742024-02-06T17:12:07.277Z+ rm ./tests/bootstrap
11752024-02-06T17:12:07.338Z+ for test_bin in tests/*
11762024-02-06T17:12:07.338Z+ ./tests/bootstrap-8c3c2066493dbc13
11772024-02-06T17:12:07.362Z
11782024-02-06T17:12:07.362Zrunning 0 tests
11792024-02-06T17:12:07.363Z
11802024-02-06T17:12:07.363Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
11812024-02-06T17:12:07.363Z
11822024-02-06T17:12:07.363Z+ for test_bin in tests/*
11832024-02-06T17:12:07.363Z+ ./tests/end_to_end_tests-17ee7da3f7e5501f
11842024-02-06T17:12:07.392Z
11852024-02-06T17:12:07.392Zrunning 2 tests
11862024-02-06T17:12:07.621Ztest no_spoof::no_spoof_here ... ok
11872024-02-06T17:12:41.231Ztest instance_launch::instance_launch ... ok
11882024-02-06T17:12:41.231Z
11892024-02-06T17:12:41.231Ztest result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 33.84s
11902024-02-06T17:12:41.231Z
11912024-02-06T17:12:41.256Z+ _exit_trap
11922024-02-06T17:12:41.256Z+ local status=0
11932024-02-06T17:12:41.256Z+ [[ 0 -eq 0 ]]
11942024-02-06T17:12:41.256Z+ exit 0
11952024-02-06T17:12:41.256Zprocess exited: duration 827211 ms, exit code 0
 
11962024-02-06T17:12:41.371Zfound 72 output files
11972024-02-06T17:12:41.371Zuploading: /var/svc/log/oxide-sled-agent:default.log (803505 bytes)
11982024-02-06T17:12:42.666Zuploaded: /var/svc/log/oxide-sled-agent:default.log
11992024-02-06T17:12:42.691Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log (3114186 bytes)
12002024-02-06T17:12:43.875Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log" changed size mid upload: 3114186 -> 3124970
12012024-02-06T17:12:44.947Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log
12022024-02-06T17:12:44.947Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log (120525 bytes)
12032024-02-06T17:12:45.109Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log" changed size mid upload: 120525 -> 121081
12042024-02-06T17:12:46.180Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log
12052024-02-06T17:12:46.181Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log (133063 bytes)
12062024-02-06T17:12:46.364Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log" changed size mid upload: 133063 -> 133808
12072024-02-06T17:12:47.481Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log
12082024-02-06T17:12:47.481Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log (1542708 bytes)
12092024-02-06T17:12:48.154Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log" changed size mid upload: 1542708 -> 1554690
12102024-02-06T17:12:49.226Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log
12112024-02-06T17:12:49.227Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log (952063 bytes)
12122024-02-06T17:12:50.020Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log" changed size mid upload: 952063 -> 958995
12132024-02-06T17:12:51.095Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log
12142024-02-06T17:12:51.095Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log (1388 bytes)
12152024-02-06T17:12:52.191Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log
12162024-02-06T17:12:52.191Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log (349624 bytes)
12172024-02-06T17:12:52.435Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log" changed size mid upload: 349624 -> 355174
12182024-02-06T17:12:53.504Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log
12192024-02-06T17:12:53.504Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log (617 bytes)
12202024-02-06T17:12:54.601Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log
12212024-02-06T17:12:54.626Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log (5393 bytes)
12222024-02-06T17:12:55.696Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log
12232024-02-06T17:12:55.721Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_865ad47c-d43e-4181-8068-852ba46c4a1c/root/var/svc/log/oxide-clickhouse:default.log (1202550 bytes)
12242024-02-06T17:12:56.423Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_865ad47c-d43e-4181-8068-852ba46c4a1c/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 1202550 -> 1258968
12252024-02-06T17:12:57.496Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_865ad47c-d43e-4181-8068-852ba46c4a1c/root/var/svc/log/oxide-clickhouse:default.log
12262024-02-06T17:12:57.496Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_865ad47c-d43e-4181-8068-852ba46c4a1c/root/var/svc/log/oxide-zone-network-setup:default.log (1709 bytes)
12272024-02-06T17:12:58.590Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_865ad47c-d43e-4181-8068-852ba46c4a1c/root/var/svc/log/oxide-zone-network-setup:default.log
12282024-02-06T17:12:58.590Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_22c95d52-fc0e-47d4-a693-df2a73cfe760/root/var/svc/log/oxide-cockroachdb:default.log (4382 bytes)
12292024-02-06T17:12:59.686Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_22c95d52-fc0e-47d4-a693-df2a73cfe760/root/var/svc/log/oxide-cockroachdb:default.log
12302024-02-06T17:12:59.686Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_22c95d52-fc0e-47d4-a693-df2a73cfe760/root/var/svc/log/oxide-zone-network-setup:default.log (1710 bytes)
12312024-02-06T17:13:00.780Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_22c95d52-fc0e-47d4-a693-df2a73cfe760/root/var/svc/log/oxide-zone-network-setup:default.log
12322024-02-06T17:13:00.780Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c2971d9f-ab9e-4e5d-ab15-2cb5cd7e5008/root/var/svc/log/oxide-crucible-agent:default.log (9790 bytes)
12332024-02-06T17:13:01.900Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c2971d9f-ab9e-4e5d-ab15-2cb5cd7e5008/root/var/svc/log/oxide-crucible-agent:default.log
12342024-02-06T17:13:01.925Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c2971d9f-ab9e-4e5d-ab15-2cb5cd7e5008/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12352024-02-06T17:13:02.996Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c2971d9f-ab9e-4e5d-ab15-2cb5cd7e5008/root/var/svc/log/oxide-crucible-downstairs:default.log
12362024-02-06T17:13:03.021Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c2971d9f-ab9e-4e5d-ab15-2cb5cd7e5008/root/var/svc/log/oxide-crucible-downstairs:downstairs-4afa10dc-d0b4-47c5-8f2e-375dbeb9fd8b.log (2945 bytes)
12372024-02-06T17:13:04.096Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c2971d9f-ab9e-4e5d-ab15-2cb5cd7e5008/root/var/svc/log/oxide-crucible-downstairs:downstairs-4afa10dc-d0b4-47c5-8f2e-375dbeb9fd8b.log
12382024-02-06T17:13:04.121Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_702f8f33-dc76-4f2a-96b9-74d42e1a35ac/root/var/svc/log/oxide-external_dns:default.log (30722 bytes)
12392024-02-06T17:13:05.262Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_702f8f33-dc76-4f2a-96b9-74d42e1a35ac/root/var/svc/log/oxide-external_dns:default.log
12402024-02-06T17:13:05.287Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_5ecc702f-32cc-4181-81b0-4e7597159232/root/var/svc/log/oxide-internal_dns:default.log (118503 bytes)
12412024-02-06T17:13:06.473Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_5ecc702f-32cc-4181-81b0-4e7597159232/root/var/svc/log/oxide-internal_dns:default.log
12422024-02-06T17:13:06.473Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_c24038a4-2e23-467a-a59f-38eaaba94f29/root/var/svc/log/oxide-nexus:default.log (34555316 bytes)
12432024-02-06T17:13:16.937Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_c24038a4-2e23-467a-a59f-38eaaba94f29/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 34555316 -> 34586546
12442024-02-06T17:13:18.010Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_c24038a4-2e23-467a-a59f-38eaaba94f29/root/var/svc/log/oxide-nexus:default.log
12452024-02-06T17:13:18.010Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_e4dda04a-92d8-42d5-b527-3ae93efe2c01/root/var/svc/log/oxide-nexus:default.log (1286643 bytes)
12462024-02-06T17:13:18.737Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_e4dda04a-92d8-42d5-b527-3ae93efe2c01/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 1286643 -> 1317873
12472024-02-06T17:13:19.812Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_e4dda04a-92d8-42d5-b527-3ae93efe2c01/root/var/svc/log/oxide-nexus:default.log
12482024-02-06T17:13:19.812Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_f7912df7-560c-4bed-8c66-7219a2123c6b/root/var/svc/log/oxide-cockroachdb:default.log (4406 bytes)
12492024-02-06T17:13:20.914Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_f7912df7-560c-4bed-8c66-7219a2123c6b/root/var/svc/log/oxide-cockroachdb:default.log
12502024-02-06T17:13:20.914Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_f7912df7-560c-4bed-8c66-7219a2123c6b/root/var/svc/log/oxide-zone-network-setup:default.log (1710 bytes)
12512024-02-06T17:13:22.009Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_f7912df7-560c-4bed-8c66-7219a2123c6b/root/var/svc/log/oxide-zone-network-setup:default.log
12522024-02-06T17:13:22.034Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_69715454-b064-461a-aa2b-38a31e381da2/root/var/svc/log/oxide-crucible-agent:default.log (3436 bytes)
12532024-02-06T17:13:23.104Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_69715454-b064-461a-aa2b-38a31e381da2/root/var/svc/log/oxide-crucible-agent:default.log
12542024-02-06T17:13:23.129Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_69715454-b064-461a-aa2b-38a31e381da2/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12552024-02-06T17:13:24.200Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_69715454-b064-461a-aa2b-38a31e381da2/root/var/svc/log/oxide-crucible-downstairs:default.log
12562024-02-06T17:13:24.201Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_bd9a3d00-2b74-46f2-92b8-005d2b7dc930/root/var/svc/log/oxide-external_dns:default.log (30708 bytes)
12572024-02-06T17:13:24.314Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_bd9a3d00-2b74-46f2-92b8-005d2b7dc930/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 30708 -> 34481
12582024-02-06T17:13:25.387Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_bd9a3d00-2b74-46f2-92b8-005d2b7dc930/root/var/svc/log/oxide-external_dns:default.log
12592024-02-06T17:13:25.387Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_39be50a7-eb44-4518-8d44-a38e2789bb9f/root/var/svc/log/oxide-internal_dns:default.log (112914 bytes)
12602024-02-06T17:13:25.547Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_39be50a7-eb44-4518-8d44-a38e2789bb9f/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 112914 -> 116669
12612024-02-06T17:13:26.617Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_39be50a7-eb44-4518-8d44-a38e2789bb9f/root/var/svc/log/oxide-internal_dns:default.log
12622024-02-06T17:13:26.642Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_f005a4c8-8b7a-4dfd-8c42-508957b405fe/root/var/svc/log/oxide-cockroachdb:default.log (4407 bytes)
12632024-02-06T17:13:27.713Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_f005a4c8-8b7a-4dfd-8c42-508957b405fe/root/var/svc/log/oxide-cockroachdb:default.log
12642024-02-06T17:13:27.713Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_f005a4c8-8b7a-4dfd-8c42-508957b405fe/root/var/svc/log/oxide-zone-network-setup:default.log (1710 bytes)
12652024-02-06T17:13:28.808Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_f005a4c8-8b7a-4dfd-8c42-508957b405fe/root/var/svc/log/oxide-zone-network-setup:default.log
12662024-02-06T17:13:28.808Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_c83296b4-dec2-4c81-85c9-f295fcc69098/root/var/svc/log/oxide-crucible-agent:default.log (35171 bytes)
12672024-02-06T17:13:29.969Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_c83296b4-dec2-4c81-85c9-f295fcc69098/root/var/svc/log/oxide-crucible-agent:default.log
12682024-02-06T17:13:29.970Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_c83296b4-dec2-4c81-85c9-f295fcc69098/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12692024-02-06T17:13:31.068Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_c83296b4-dec2-4c81-85c9-f295fcc69098/root/var/svc/log/oxide-crucible-downstairs:default.log
12702024-02-06T17:13:31.092Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_c83296b4-dec2-4c81-85c9-f295fcc69098/root/var/svc/log/oxide-crucible-downstairs:downstairs-8c0d83a5-d472-4072-bac6-8e44a67c7b15.log (5614 bytes)
12712024-02-06T17:13:32.162Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_c83296b4-dec2-4c81-85c9-f295fcc69098/root/var/svc/log/oxide-crucible-downstairs:downstairs-8c0d83a5-d472-4072-bac6-8e44a67c7b15.log
12722024-02-06T17:13:32.162Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_c83296b4-dec2-4c81-85c9-f295fcc69098/root/var/svc/log/oxide-crucible-downstairs:downstairs-c81ac63b-d945-4bb9-8b4e-6a263e8758ca.log (5538 bytes)
12732024-02-06T17:13:33.257Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_c83296b4-dec2-4c81-85c9-f295fcc69098/root/var/svc/log/oxide-crucible-downstairs:downstairs-c81ac63b-d945-4bb9-8b4e-6a263e8758ca.log
12742024-02-06T17:13:33.283Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_c83296b4-dec2-4c81-85c9-f295fcc69098/root/var/svc/log/oxide-crucible-downstairs:snapshot-8c0d83a5-d472-4072-bac6-8e44a67c7b15-7b23de1a-c7ea-4c85-9c25-9375bdee1c71.log (5177 bytes)
12752024-02-06T17:13:34.354Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_c83296b4-dec2-4c81-85c9-f295fcc69098/root/var/svc/log/oxide-crucible-downstairs:snapshot-8c0d83a5-d472-4072-bac6-8e44a67c7b15-7b23de1a-c7ea-4c85-9c25-9375bdee1c71.log
12762024-02-06T17:13:34.355Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_8f2bb917-5a5c-44bc-a0c4-326eae81a6f2/root/var/svc/log/oxide-internal_dns:default.log (117375 bytes)
12772024-02-06T17:13:34.513Zupload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_8f2bb917-5a5c-44bc-a0c4-326eae81a6f2/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 117375 -> 121129
12782024-02-06T17:13:35.583Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_8f2bb917-5a5c-44bc-a0c4-326eae81a6f2/root/var/svc/log/oxide-internal_dns:default.log
12792024-02-06T17:13:35.583Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_77c12814-b3ed-49de-ba9e-07265f527890/root/var/svc/log/oxide-cockroachdb:default.log (4406 bytes)
12802024-02-06T17:13:36.678Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_77c12814-b3ed-49de-ba9e-07265f527890/root/var/svc/log/oxide-cockroachdb:default.log
12812024-02-06T17:13:36.678Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_77c12814-b3ed-49de-ba9e-07265f527890/root/var/svc/log/oxide-zone-network-setup:default.log (1711 bytes)
12822024-02-06T17:13:37.773Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_77c12814-b3ed-49de-ba9e-07265f527890/root/var/svc/log/oxide-zone-network-setup:default.log
12832024-02-06T17:13:37.773Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_e0a755a5-55e8-45c2-a3f3-f08f3f3c8714/root/var/svc/log/oxide-crucible-agent:default.log (27371 bytes)
12842024-02-06T17:13:38.915Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_e0a755a5-55e8-45c2-a3f3-f08f3f3c8714/root/var/svc/log/oxide-crucible-agent:default.log
12852024-02-06T17:13:38.915Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_e0a755a5-55e8-45c2-a3f3-f08f3f3c8714/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12862024-02-06T17:13:40.009Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_e0a755a5-55e8-45c2-a3f3-f08f3f3c8714/root/var/svc/log/oxide-crucible-downstairs:default.log
12872024-02-06T17:13:40.009Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_e0a755a5-55e8-45c2-a3f3-f08f3f3c8714/root/var/svc/log/oxide-crucible-downstairs:downstairs-f7883d29-84da-464f-a66a-a94f5c7a0e5d.log (5614 bytes)
12882024-02-06T17:13:41.103Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_e0a755a5-55e8-45c2-a3f3-f08f3f3c8714/root/var/svc/log/oxide-crucible-downstairs:downstairs-f7883d29-84da-464f-a66a-a94f5c7a0e5d.log
12892024-02-06T17:13:41.103Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_e0a755a5-55e8-45c2-a3f3-f08f3f3c8714/root/var/svc/log/oxide-crucible-downstairs:downstairs-fb0a0998-fc76-4022-91bd-f79baf20e90a.log (3051 bytes)
12902024-02-06T17:13:42.201Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_e0a755a5-55e8-45c2-a3f3-f08f3f3c8714/root/var/svc/log/oxide-crucible-downstairs:downstairs-fb0a0998-fc76-4022-91bd-f79baf20e90a.log
12912024-02-06T17:13:42.201Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_e0a755a5-55e8-45c2-a3f3-f08f3f3c8714/root/var/svc/log/oxide-crucible-downstairs:snapshot-f7883d29-84da-464f-a66a-a94f5c7a0e5d-7b23de1a-c7ea-4c85-9c25-9375bdee1c71.log (5177 bytes)
12922024-02-06T17:13:43.293Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_e0a755a5-55e8-45c2-a3f3-f08f3f3c8714/root/var/svc/log/oxide-crucible-downstairs:snapshot-f7883d29-84da-464f-a66a-a94f5c7a0e5d-7b23de1a-c7ea-4c85-9c25-9375bdee1c71.log
12932024-02-06T17:13:43.293Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_3c746123-2da3-4816-b2ab-0bd685c5cae7/root/var/svc/log/oxide-cockroachdb:default.log (4408 bytes)
12942024-02-06T17:13:44.393Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_3c746123-2da3-4816-b2ab-0bd685c5cae7/root/var/svc/log/oxide-cockroachdb:default.log
12952024-02-06T17:13:44.393Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_3c746123-2da3-4816-b2ab-0bd685c5cae7/root/var/svc/log/oxide-zone-network-setup:default.log (1711 bytes)
12962024-02-06T17:13:45.486Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_3c746123-2da3-4816-b2ab-0bd685c5cae7/root/var/svc/log/oxide-zone-network-setup:default.log
12972024-02-06T17:13:45.486Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_00acbfc1-f1c4-40b9-ab1c-a41fcddab7cb/root/var/svc/log/oxide-crucible-agent:default.log (14116 bytes)
12982024-02-06T17:13:46.606Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_00acbfc1-f1c4-40b9-ab1c-a41fcddab7cb/root/var/svc/log/oxide-crucible-agent:default.log
12992024-02-06T17:13:46.607Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_00acbfc1-f1c4-40b9-ab1c-a41fcddab7cb/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
13002024-02-06T17:13:47.702Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_00acbfc1-f1c4-40b9-ab1c-a41fcddab7cb/root/var/svc/log/oxide-crucible-downstairs:default.log
13012024-02-06T17:13:47.702Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_00acbfc1-f1c4-40b9-ab1c-a41fcddab7cb/root/var/svc/log/oxide-crucible-downstairs:downstairs-6dbf0a5b-30cf-4f09-969b-c0a622771f22.log (3051 bytes)
13022024-02-06T17:13:48.796Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_00acbfc1-f1c4-40b9-ab1c-a41fcddab7cb/root/var/svc/log/oxide-crucible-downstairs:downstairs-6dbf0a5b-30cf-4f09-969b-c0a622771f22.log
13032024-02-06T17:13:48.824Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_4619fe59-8e66-4635-952f-cc296a2cb048/root/var/svc/log/oxide-crucible-pantry:default.log (2338 bytes)
13042024-02-06T17:13:49.891Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_4619fe59-8e66-4635-952f-cc296a2cb048/root/var/svc/log/oxide-crucible-pantry:default.log
13052024-02-06T17:13:49.891Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_e1300cd0-e49f-420c-855b-c6c25dd084ba/root/var/svc/log/oxide-crucible-pantry:default.log (2338 bytes)
13062024-02-06T17:13:50.989Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_e1300cd0-e49f-420c-855b-c6c25dd084ba/root/var/svc/log/oxide-crucible-pantry:default.log
13072024-02-06T17:13:50.989Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86/root/var/svc/log/oxide-ntp:default.log (1328 bytes)
13082024-02-06T17:13:52.084Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86/root/var/svc/log/oxide-ntp:default.log
13092024-02-06T17:13:52.084Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_7bc744b5-bba0-44fd-a747-9dc30c122313/root/var/svc/log/oxide-crucible-agent:default.log (33727 bytes)
13102024-02-06T17:13:53.246Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_7bc744b5-bba0-44fd-a747-9dc30c122313/root/var/svc/log/oxide-crucible-agent:default.log
13112024-02-06T17:13:53.246Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_7bc744b5-bba0-44fd-a747-9dc30c122313/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
13122024-02-06T17:13:54.344Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_7bc744b5-bba0-44fd-a747-9dc30c122313/root/var/svc/log/oxide-crucible-downstairs:default.log
13132024-02-06T17:13:54.344Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_7bc744b5-bba0-44fd-a747-9dc30c122313/root/var/svc/log/oxide-crucible-downstairs:downstairs-0ab9dede-0c68-4524-9e76-2e3701462ef5.log (2944 bytes)
13142024-02-06T17:13:55.439Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_7bc744b5-bba0-44fd-a747-9dc30c122313/root/var/svc/log/oxide-crucible-downstairs:downstairs-0ab9dede-0c68-4524-9e76-2e3701462ef5.log
13152024-02-06T17:13:55.439Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_7bc744b5-bba0-44fd-a747-9dc30c122313/root/var/svc/log/oxide-crucible-downstairs:downstairs-77e9e512-49d3-4444-b783-b0d57fdea786.log (5618 bytes)
13162024-02-06T17:13:56.534Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_7bc744b5-bba0-44fd-a747-9dc30c122313/root/var/svc/log/oxide-crucible-downstairs:downstairs-77e9e512-49d3-4444-b783-b0d57fdea786.log
13172024-02-06T17:13:56.558Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_7bc744b5-bba0-44fd-a747-9dc30c122313/root/var/svc/log/oxide-crucible-downstairs:downstairs-eac16d54-fe3c-46c4-b725-2d5a382f3169.log (5539 bytes)
13182024-02-06T17:13:57.629Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_7bc744b5-bba0-44fd-a747-9dc30c122313/root/var/svc/log/oxide-crucible-downstairs:downstairs-eac16d54-fe3c-46c4-b725-2d5a382f3169.log
13192024-02-06T17:13:57.654Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_7bc744b5-bba0-44fd-a747-9dc30c122313/root/var/svc/log/oxide-crucible-downstairs:snapshot-77e9e512-49d3-4444-b783-b0d57fdea786-7b23de1a-c7ea-4c85-9c25-9375bdee1c71.log (5179 bytes)
13202024-02-06T17:13:58.725Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_7bc744b5-bba0-44fd-a747-9dc30c122313/root/var/svc/log/oxide-crucible-downstairs:snapshot-77e9e512-49d3-4444-b783-b0d57fdea786-7b23de1a-c7ea-4c85-9c25-9375bdee1c71.log
13212024-02-06T17:13:58.725Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_nexus_b605e718-9fa8-46ff-acf7-d329ef9a8f4e/root/var/svc/log/oxide-nexus:default.log (2807816 bytes)
13222024-02-06T17:13:59.546Zupload warning: file "/pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_nexus_b605e718-9fa8-46ff-acf7-d329ef9a8f4e/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 2807816 -> 2948767
13232024-02-06T17:14:00.615Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_nexus_b605e718-9fa8-46ff-acf7-d329ef9a8f4e/root/var/svc/log/oxide-nexus:default.log
13242024-02-06T17:14:00.615Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_oximeter_469a4112-59e3-4841-a4d5-31337dc69571/root/var/svc/log/oxide-oximeter:default.log (337010 bytes)
13252024-02-06T17:14:00.846Zupload warning: file "/pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_oximeter_469a4112-59e3-4841-a4d5-31337dc69571/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 337010 -> 397804
13262024-02-06T17:14:01.917Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_oximeter_469a4112-59e3-4841-a4d5-31337dc69571/root/var/svc/log/oxide-oximeter:default.log
13272024-02-06T17:14:01.942Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_oximeter_469a4112-59e3-4841-a4d5-31337dc69571/root/var/svc/log/oxide-zone-network-setup:default.log (1697 bytes)
13282024-02-06T17:14:03.018Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_oximeter_469a4112-59e3-4841-a4d5-31337dc69571/root/var/svc/log/oxide-zone-network-setup:default.log
13292024-02-06T17:14:03.018Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_8d36b261-d278-4bc0-b3b9-ce89d601953e/root/var/svc/log/oxide-crucible-agent:default.log (3436 bytes)
13302024-02-06T17:14:04.117Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_8d36b261-d278-4bc0-b3b9-ce89d601953e/root/var/svc/log/oxide-crucible-agent:default.log
13312024-02-06T17:14:04.118Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_8d36b261-d278-4bc0-b3b9-ce89d601953e/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
13322024-02-06T17:14:05.220Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_8d36b261-d278-4bc0-b3b9-ce89d601953e/root/var/svc/log/oxide-crucible-downstairs:default.log
13332024-02-06T17:14:05.220Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_ab57af46-e251-41f7-af1f-92509123f5a4/root/var/svc/log/oxide-crucible-agent:default.log (9791 bytes)
13342024-02-06T17:14:06.334Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_ab57af46-e251-41f7-af1f-92509123f5a4/root/var/svc/log/oxide-crucible-agent:default.log
13352024-02-06T17:14:06.334Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_ab57af46-e251-41f7-af1f-92509123f5a4/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
13362024-02-06T17:14:07.460Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_ab57af46-e251-41f7-af1f-92509123f5a4/root/var/svc/log/oxide-crucible-downstairs:default.log
13372024-02-06T17:14:07.460Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_ab57af46-e251-41f7-af1f-92509123f5a4/root/var/svc/log/oxide-crucible-downstairs:downstairs-976ac57c-569e-4228-ad30-9ecca972a22b.log (2945 bytes)
13382024-02-06T17:14:08.543Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_ab57af46-e251-41f7-af1f-92509123f5a4/root/var/svc/log/oxide-crucible-downstairs:downstairs-976ac57c-569e-4228-ad30-9ecca972a22b.log
13392024-02-06T17:14:08.543Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_bbe6ba5e-7ec6-41c7-8364-e12f704196a6/root/var/svc/log/oxide-crucible-agent:default.log (24360 bytes)
13402024-02-06T17:14:09.686Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_bbe6ba5e-7ec6-41c7-8364-e12f704196a6/root/var/svc/log/oxide-crucible-agent:default.log
13412024-02-06T17:14:09.686Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_bbe6ba5e-7ec6-41c7-8364-e12f704196a6/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
13422024-02-06T17:14:10.783Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_bbe6ba5e-7ec6-41c7-8364-e12f704196a6/root/var/svc/log/oxide-crucible-downstairs:default.log
13432024-02-06T17:14:10.807Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_bbe6ba5e-7ec6-41c7-8364-e12f704196a6/root/var/svc/log/oxide-crucible-downstairs:downstairs-587b3df2-c187-49b0-a689-b6546548ad29.log (3051 bytes)
13442024-02-06T17:14:11.882Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_bbe6ba5e-7ec6-41c7-8364-e12f704196a6/root/var/svc/log/oxide-crucible-downstairs:downstairs-587b3df2-c187-49b0-a689-b6546548ad29.log
13452024-02-06T17:14:11.882Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_bbe6ba5e-7ec6-41c7-8364-e12f704196a6/root/var/svc/log/oxide-crucible-downstairs:downstairs-cc6ff073-ec07-456c-8b84-c7e9a960b0c8.log (5537 bytes)
13462024-02-06T17:14:12.982Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_bbe6ba5e-7ec6-41c7-8364-e12f704196a6/root/var/svc/log/oxide-crucible-downstairs:downstairs-cc6ff073-ec07-456c-8b84-c7e9a960b0c8.log
13472024-02-06T17:14:12.982Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_262c177d-468e-426b-85f1-508619427eda/root/var/svc/log/oxide-crucible-pantry:default.log (1224057 bytes)
13482024-02-06T17:14:14.348Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_262c177d-468e-426b-85f1-508619427eda/root/var/svc/log/oxide-crucible-pantry:default.log
13492024-02-06T17:14:14.379Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86/root/var/log/chrony/measurements.log (25893 bytes)
13502024-02-06T17:14:14.463Zupload warning: file "/pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86/root/var/log/chrony/measurements.log" changed size mid upload: 25893 -> 29455
13512024-02-06T17:14:15.536Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86/root/var/log/chrony/measurements.log
13522024-02-06T17:14:15.536Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86/root/var/log/chrony/statistics.log (14976 bytes)
13532024-02-06T17:14:15.631Zupload warning: file "/pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86/root/var/log/chrony/statistics.log" changed size mid upload: 14976 -> 17550
13542024-02-06T17:14:16.704Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86/root/var/log/chrony/statistics.log
13552024-02-06T17:14:16.704Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86/root/var/log/chrony/tracking.log (2647 bytes)
13562024-02-06T17:14:16.753Zupload warning: file "/pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86/root/var/log/chrony/tracking.log" changed size mid upload: 2647 -> 2911
13572024-02-06T17:14:17.825Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_4e2a180a-8193-4375-9f3e-66c70cd25d86/root/var/log/chrony/tracking.log