01HKTW8P65KW25VHEBPG8AZQ4J: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.27",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01HKTW9QYWQX75J6RVGQNZB6ZD

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12024-01-11T00:38:38.731Zjob dependencies complete; ready to run (waiting for 52 m 29 s)
22024-01-11T00:44:29.359Zjob assigned to worker 01HKTZB4245KC26MPYD0D8WKAJ (queued for 5 m 50 s)
32024-01-11T00:44:29.508Zhttp://172.20.0.5:9969/ipxe/buskin... ok
42024-01-11T00:44:29.548Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/kernel/amd64/unix... ok
52024-01-11T00:44:29.581Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive... ok
62024-01-11T00:44:29.615Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive.hash... ok
72024-01-11T00:44:29.888Zhttp://172.20.0.5:9969/postboot/buskin... ok
82024-01-11T00:44:29.951ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
92024-01-11T00:44:29.994Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d1
102024-01-11T00:44:30.034ZOxide Helios Version helios-2.0.22333 64-bit
112024-01-11T00:44:30.077ZConfiguring devices.
122024-01-11T00:44:30.117Z##################################################
132024-01-11T00:44:30.223Z#### ############# ##
142024-01-11T00:44:30.261Z### ### ############ ##
152024-01-11T00:44:30.294Z## ### # ## ### ## Oxide Computer Company ##
162024-01-11T00:44:30.335Z## ## ## ### # ### ##
172024-01-11T00:44:30.381Z## # ### #### #### This Station Under ##
182024-01-11T00:44:30.417Z### ### #### # ### Computer Control ##
192024-01-11T00:44:30.453Z#### #### ### ## ##
202024-01-11T00:44:30.484Z##################################################
212024-01-11T00:44:30.526ZHostname: buskin
222024-01-11T00:44:30.562ZLoading smf(7) service descriptions:
232024-01-11T00:44:31.189Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap-6bdde9b1284fa368.gz
242024-01-11T00:44:33.186Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap-6bdde9b1284fa368.gz
252024-01-11T00:44:33.214Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
262024-01-11T00:44:55.671Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
272024-01-11T00:44:55.671Zdownloading input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-826dc01617fce641.gz
282024-01-11T00:45:22.791Zdownloaded input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-826dc01617fce641.gz
292024-01-11T00:45:22.791Zdownloading input: /input/ci-tools/work/caboose-util.gz
302024-01-11T00:45:23.860Zdownloaded input: /input/ci-tools/work/caboose-util.gz
312024-01-11T00:45:23.886Zdownloading input: /input/ci-tools/work/tufaceous.gz
322024-01-11T00:45:26.866Zdownloaded input: /input/ci-tools/work/tufaceous.gz
332024-01-11T00:45:26.892Zdownloading input: /input/package/work/version.txt
342024-01-11T00:45:27.329Zdownloaded input: /input/package/work/version.txt
352024-01-11T00:45:27.329Zdownloading input: /input/package/work/package.tar.gz
362024-01-11T00:45:44.809Zdownloaded input: /input/package/work/package.tar.gz
372024-01-11T00:45:44.809Zdownloading input: /input/package/work/global-zone-packages.tar.gz
382024-01-11T00:46:28.617Zdownloaded input: /input/package/work/global-zone-packages.tar.gz
392024-01-11T00:46:28.618Zdownloading input: /input/package/work/trampoline-global-zone-packages.tar.gz
402024-01-11T00:46:36.487Zdownloaded input: /input/package/work/trampoline-global-zone-packages.tar.gz
412024-01-11T00:46:36.517Zdownloading input: /input/package/work/zones/clickhouse.tar.gz
422024-01-11T00:47:32.672Zdownloaded input: /input/package/work/zones/clickhouse.tar.gz
432024-01-11T00:47:32.672Zdownloading input: /input/package/work/zones/clickhouse_keeper.tar.gz
442024-01-11T00:48:17.464Zdownloaded input: /input/package/work/zones/clickhouse_keeper.tar.gz
452024-01-11T00:48:17.464Zdownloading input: /input/package/work/zones/cockroachdb.tar.gz
462024-01-11T00:48:48.052Zdownloaded input: /input/package/work/zones/cockroachdb.tar.gz
472024-01-11T00:48:48.053Zdownloading input: /input/package/work/zones/crucible-pantry.tar.gz
482024-01-11T00:48:53.353Zdownloaded input: /input/package/work/zones/crucible-pantry.tar.gz
492024-01-11T00:48:53.353Zdownloading input: /input/package/work/zones/crucible.tar.gz
502024-01-11T00:48:59.452Zdownloaded input: /input/package/work/zones/crucible.tar.gz
512024-01-11T00:48:59.452Zdownloading input: /input/package/work/zones/external-dns.tar.gz
522024-01-11T00:49:04.634Zdownloaded input: /input/package/work/zones/external-dns.tar.gz
532024-01-11T00:49:04.634Zdownloading input: /input/package/work/zones/internal-dns.tar.gz
542024-01-11T00:49:09.079Zdownloaded input: /input/package/work/zones/internal-dns.tar.gz
552024-01-11T00:49:09.080Zdownloading input: /input/package/work/zones/ntp.tar.gz
562024-01-11T00:49:09.116Zdownloaded input: /input/package/work/zones/ntp.tar.gz
572024-01-11T00:49:09.116Zdownloading input: /input/package/work/zones/omicron-gateway-asic.tar.gz
582024-01-11T00:49:13.821Zdownloaded input: /input/package/work/zones/omicron-gateway-asic.tar.gz
592024-01-11T00:49:13.821Zdownloading input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
602024-01-11T00:49:17.416Zdownloaded input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
612024-01-11T00:49:17.416Zdownloading input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
622024-01-11T00:49:44.520Zdownloaded input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
632024-01-11T00:49:44.520Zdownloading input: /input/package/work/zones/omicron-nexus.tar.gz
642024-01-11T00:50:02.928Zdownloaded input: /input/package/work/zones/omicron-nexus.tar.gz
652024-01-11T00:50:02.929Zdownloading input: /input/package/work/zones/overlay.tar.gz
662024-01-11T00:50:03.022Zdownloaded input: /input/package/work/zones/overlay.tar.gz
672024-01-11T00:50:03.022Zdownloading input: /input/package/work/zones/oximeter-collector.tar.gz
682024-01-11T00:50:11.146Zdownloaded input: /input/package/work/zones/oximeter-collector.tar.gz
692024-01-11T00:50:11.146Zdownloading input: /input/package/work/zones/propolis-server.tar.gz
702024-01-11T00:50:34.324Zdownloaded input: /input/package/work/zones/propolis-server.tar.gz
712024-01-11T00:50:34.324Zdownloading input: /input/package/work/zones/switch-asic.tar.gz
722024-01-11T00:51:31.312Zdownloaded input: /input/package/work/zones/switch-asic.tar.gz
732024-01-11T00:51:31.312Zdownloading input: /input/package/work/zones/switch-softnpu.tar.gz
742024-01-11T00:52:15.378Zdownloaded input: /input/package/work/zones/switch-softnpu.tar.gz
 
752024-01-11T00:52:15.445Zstarting task 0: "setup"
762024-01-11T00:52:15.446Z++ uname -s
772024-01-11T00:52:15.446Z+ kern=SunOS
782024-01-11T00:52:15.446Z+ case "$kern" in
792024-01-11T00:52:15.446Z+ groupadd -g 12345 build
802024-01-11T00:52:15.446Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
812024-01-11T00:52:15.529Z+ zfs create -o mountpoint=/work rpool/work
822024-01-11T00:52:15.573Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
832024-01-11T00:52:15.573Z+ home_fs=zfs
842024-01-11T00:52:15.574Z+ [[ zfs == autofs ]]
852024-01-11T00:52:15.574Z+ mkdir -p /home/build
862024-01-11T00:52:15.574Z+ chown build:build /home/build /work
872024-01-11T00:52:16.609Z+ chmod 0700 /home/build /work
882024-01-11T00:52:16.650Zprocess exited: duration 3047 ms, exit code 0
 
892024-01-11T00:52:16.717Zstarting task 1: "authentication"
902024-01-11T00:52:16.756Zprocess exited: duration 45 ms, exit code 0
 
912024-01-11T00:52:16.818Zstarting task 2: "build"
922024-01-11T00:52:16.844Z+ trap _exit_trap EXIT
932024-01-11T00:52:16.844Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
942024-01-11T00:52:17.026Zsecpolicy_fs_linkdir: 0x55 = 0x55
952024-01-11T00:52:17.026Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
962024-01-11T00:52:17.026Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
972024-01-11T00:52:17.026Zsecpolicy_fs_linkdir+3: 0 = 0xe5
982024-01-11T00:52:17.059Zsecpolicy_fs_linkdir+4: 0 = 0xb8
992024-01-11T00:52:17.059Zsecpolicy_fs_linkdir+5: 0 = 0x1
1002024-01-11T00:52:17.059Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
1012024-01-11T00:52:17.059Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
1022024-01-11T00:52:17.059Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
1032024-01-11T00:52:17.059Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
1042024-01-11T00:52:17.060Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
1052024-01-11T00:52:17.060Z+ [[ -d /opt/oxide ]]
1062024-01-11T00:52:17.060Z+ pfexec mv /opt/oxide /opt/oxide-underneath
1072024-01-11T00:52:17.060Z+ pfexec mkdir /opt/oxide
1082024-01-11T00:52:17.060Z+ pfexec mount -F tmpfs -O swap /opt/oxide
1092024-01-11T00:52:17.060Z+ [[ -d /opt/oxide-underneath ]]
1102024-01-11T00:52:17.060Z+ cd /opt/oxide-underneath
1112024-01-11T00:52:17.060Z+ pfexec tar ceEp@/f - .
1122024-01-11T00:52:17.060Z+ cd /opt/oxide
1132024-01-11T00:52:17.060Z+ pfexec tar xveEp@/f -
1142024-01-11T00:52:17.146Zx ., 0 bytes, 0 tape blocks
1152024-01-11T00:52:17.147Zx ./opte, 0 bytes, 0 tape blocks
1162024-01-11T00:52:17.147Zx ./opte/bin, 0 bytes, 0 tape blocks
1172024-01-11T00:52:17.147Zx ./opte/bin/opteadm, 52567720 bytes, 102672 tape blocks
1182024-01-11T00:52:17.177Z+ pfexec rm -rf /opt/oxide-underneath
1192024-01-11T00:52:17.177Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
1202024-01-11T00:52:17.177Z+ pfexec mkdir /opt/oxide/work
1212024-01-11T00:52:17.206Z+ pfexec chown build:build /opt/oxide/work
1222024-01-11T00:52:17.206Z+ cd /opt/oxide/work
1232024-01-11T00:52:17.206Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
1242024-01-11T00:52:18.733ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
1252024-01-11T00:52:18.734Zx out/mg-ddm-gz.tar, 62723072 bytes, 122506 tape blocks
1262024-01-11T00:52:18.734Zx out/omicron-sled-agent.tar, 114147840 bytes, 222945 tape blocks
1272024-01-11T00:52:18.734Zx out/target/test, 75 bytes, 1 tape blocks
1282024-01-11T00:52:18.734Zx out/npuzone/npuzone, 37184104 bytes, 72626 tape blocks
1292024-01-11T00:52:18.734Zx package-manifest.toml, 22445 bytes, 44 tape blocks
1302024-01-11T00:52:18.734Zx smf/sled-agent/non-gimlet/config.toml, 3448 bytes, 7 tape blocks
1312024-01-11T00:52:18.735Zx target/release/omicron-package, 42860992 bytes, 83713 tape blocks
1322024-01-11T00:52:18.735Zx tools/create_virtual_hardware.sh, 3176 bytes, 7 tape blocks
1332024-01-11T00:52:18.735Zx tools/virtual_hardware.sh, 3408 bytes, 7 tape blocks
1342024-01-11T00:52:18.735Zx tools/scrimlet/create-softnpu-zone.sh, 619 bytes, 2 tape blocks
1352024-01-11T00:52:18.735Zx tools/scrimlet/destroy-softnpu-zone.sh, 195 bytes, 1 tape blocks
1362024-01-11T00:52:18.735Zx tools/scrimlet/softnpu-init.sh, 2149 bytes, 5 tape blocks
1372024-01-11T00:52:18.735Zx tools/scrimlet/softnpu-zone.txt, 330 bytes, 1 tape blocks
1382024-01-11T00:52:18.735Zx tools/scrimlet/softnpu.toml, 171 bytes, 1 tape blocks
1392024-01-11T00:52:18.760Z
1402024-01-11T00:52:18.760Zreal 1.549752471
1412024-01-11T00:52:18.760Zuser 1.394625988
1422024-01-11T00:52:18.760Zsys 0.274814091
1432024-01-11T00:52:18.760Ztrap 0.000054332
1442024-01-11T00:52:18.760Ztflt 0.000000000
1452024-01-11T00:52:18.760Zdflt 0.000000000
1462024-01-11T00:52:18.760Zkflt 0.000000000
1472024-01-11T00:52:18.760Zlock 0.000000000
1482024-01-11T00:52:18.760Zslp 1.392780144
1492024-01-11T00:52:18.760Zlat 0.034972535
1502024-01-11T00:52:18.760Zstop 0.000125473
1512024-01-11T00:52:18.760Z+ cp /input/package/work/zones/clickhouse_keeper.tar.gz /input/package/work/zones/clickhouse.tar.gz /input/package/work/zones/cockroachdb.tar.gz /input/package/work/zones/crucible-pantry.tar.gz /input/package/work/zones/crucible.tar.gz /input/package/work/zones/external-dns.tar.gz /input/package/work/zones/internal-dns.tar.gz /input/package/work/zones/ntp.tar.gz /input/package/work/zones/omicron-gateway-asic.tar.gz /input/package/work/zones/omicron-gateway-softnpu.tar.gz /input/package/work/zones/omicron-nexus-single-sled.tar.gz /input/package/work/zones/omicron-nexus.tar.gz /input/package/work/zones/overlay.tar.gz /input/package/work/zones/oximeter-collector.tar.gz /input/package/work/zones/propolis-server.tar.gz /input/package/work/zones/switch-asic.tar.gz /input/package/work/zones/switch-softnpu.tar.gz out/
1522024-01-11T00:52:19.820Z+ mv out/omicron-nexus-single-sled.tar.gz out/omicron-nexus.tar.gz
1532024-01-11T00:52:20.259Z+ mkdir tests
1542024-01-11T00:52:20.259Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1552024-01-11T00:52:20.259Z+ ptime -m gunzip
1562024-01-11T00:52:20.259Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-6bdde9b1284fa368
1572024-01-11T00:52:20.291Z
1582024-01-11T00:52:20.293Zreal 0.141068012
1592024-01-11T00:52:20.293Zuser 0.127433194
1602024-01-11T00:52:20.293Zsys 0.013405122
1612024-01-11T00:52:20.293Ztrap 0.000007220
1622024-01-11T00:52:20.293Ztflt 0.000000000
1632024-01-11T00:52:20.293Zdflt 0.000000000
1642024-01-11T00:52:20.294Zkflt 0.000000000
1652024-01-11T00:52:20.294Zlock 0.000000000
1662024-01-11T00:52:20.294Zslp 0.000020970
1672024-01-11T00:52:20.294Zlat 0.000023339
1682024-01-11T00:52:20.294Zstop 0.000119213
1692024-01-11T00:52:20.294Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-6bdde9b1284fa368
1702024-01-11T00:52:20.294Z+ chmod a+x tests/bootstrap-6bdde9b1284fa368
1712024-01-11T00:52:20.294Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1722024-01-11T00:52:20.294Z+ ptime -m gunzip
1732024-01-11T00:52:20.295Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1742024-01-11T00:52:22.234Z
1752024-01-11T00:52:22.266Zreal 2.245527070
1762024-01-11T00:52:22.267Zuser 2.116181525
1772024-01-11T00:52:22.267Zsys 0.128950185
1782024-01-11T00:52:22.267Ztrap 0.000183234
1792024-01-11T00:52:22.268Ztflt 0.000000000
1802024-01-11T00:52:22.268Zdflt 0.000000000
1812024-01-11T00:52:22.268Zkflt 0.000000000
1822024-01-11T00:52:22.269Zlock 0.000000000
1832024-01-11T00:52:22.269Zslp 0.000020550
1842024-01-11T00:52:22.269Zlat 0.000016610
1852024-01-11T00:52:22.269Zstop 0.000108412
1862024-01-11T00:52:22.269Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1872024-01-11T00:52:22.269Z+ chmod a+x tests/bootstrap
1882024-01-11T00:52:22.269Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1892024-01-11T00:52:22.269Z+ ptime -m gunzip
1902024-01-11T00:52:22.269Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-826dc01617fce641
1912024-01-11T00:52:24.564Z
1922024-01-11T00:52:24.590Zreal 2.317609518
1932024-01-11T00:52:24.590Zuser 2.182633705
1942024-01-11T00:52:24.590Zsys 0.134586675
1952024-01-11T00:52:24.590Ztrap 0.000167494
1962024-01-11T00:52:24.590Ztflt 0.000000000
1972024-01-11T00:52:24.590Zdflt 0.000000000
1982024-01-11T00:52:24.590Zkflt 0.000000000
1992024-01-11T00:52:24.590Zlock 0.000000000
2002024-01-11T00:52:24.590Zslp 0.000024220
2012024-01-11T00:52:24.590Zlat 0.000025380
2022024-01-11T00:52:24.590Zstop 0.000108622
2032024-01-11T00:52:24.590Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-826dc01617fce641
2042024-01-11T00:52:24.590Z+ chmod a+x tests/end_to_end_tests-826dc01617fce641
2052024-01-11T00:52:24.590Z+ bmat address ls
2062024-01-11T00:52:24.590ZNAME CIDR FIRST LAST COUNT
2072024-01-11T00:52:24.591Zextra 10.151.1.0/24 10.151.1.164 10.151.1.227 64
2082024-01-11T00:52:24.591Z++ bmat address ls -f extra -Ho first,last,count
2092024-01-11T00:52:24.615Z+ set -- 10.151.1.164 10.151.1.227 64
2102024-01-11T00:52:24.615Z+ EXTRA_IP_START=10.151.1.164
2112024-01-11T00:52:24.615Z+ EXTRA_IP_END=10.151.1.227
2122024-01-11T00:52:24.615Z+ EXTRA_IP_COUNT=64
2132024-01-11T00:52:24.616Z+ (( EXTRA_IP_COUNT >= 32 ))
2142024-01-11T00:52:24.616Z+ EXTRA_IP_BASE=10.151.1
2152024-01-11T00:52:24.616Z+ EXTRA_IP_FOCTET=164
2162024-01-11T00:52:24.616Z+ EXTRA_IP_LOCTET=227
2172024-01-11T00:52:24.616Z+ SERVICE_IP_POOL_START=10.151.1.164
2182024-01-11T00:52:24.616Z+ SERVICE_IP_POOL_END=10.151.1.173
2192024-01-11T00:52:24.616Z+ DNS_IP1=10.151.1.164
2202024-01-11T00:52:24.616Z+ DNS_IP2=10.151.1.165
2212024-01-11T00:52:24.616Z+ UPLINK_IP=10.151.1.174
2222024-01-11T00:52:24.616Z+ IPPOOL_START=10.151.1.175
2232024-01-11T00:52:24.616Z+ IPPOOL_END=10.151.1.227
2242024-01-11T00:52:24.616Z++ ipadm show-addr -po type,addr
2252024-01-11T00:52:24.616Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
2262024-01-11T00:52:24.616Z+ GATEWAY_IP=10.151.1.100
2272024-01-11T00:52:24.616Z+ [[ -n 10.151.1.100 ]]
2282024-01-11T00:52:24.616Z+ ping -s 10.151.1.100 56 1
2292024-01-11T00:52:24.616ZPING 10.151.1.100: 56 data bytes
2302024-01-11T00:52:24.641Z64 bytes from 10.151.1.100: icmp_seq=0. time=0.009 ms
2312024-01-11T00:52:24.642Z
2322024-01-11T00:52:24.642Z----10.151.1.100 PING Statistics----
2332024-01-11T00:52:24.642Z1 packets transmitted, 1 packets received, 0% packet loss
2342024-01-11T00:52:24.642Zround-trip (ms) min/avg/max/stddev = 0.009/0.009/0.009/-nan
2352024-01-11T00:52:24.642Z++ arp -an
2362024-01-11T00:52:24.642Z++ awk -vgw=10.151.1.100 '$2 == gw {print $NF}'
2372024-01-11T00:52:24.642Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
2382024-01-11T00:52:24.642Z+ routeadm -e ipv4-forwarding -u
2392024-01-11T00:52:24.745Z+ PXA_START=10.151.1.164
2402024-01-11T00:52:24.745Z+ PXA_END=10.151.1.227
2412024-01-11T00:52:24.774Z+ export GATEWAY_IP GATEWAY_MAC PXA_START PXA_END
2422024-01-11T00:52:24.774Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
2432024-01-11T00:52:24.883Z+ ZPOOL_VDEV_DIR=/scratch
2442024-01-11T00:52:24.883Z+ ptime -m pfexec ./tools/create_virtual_hardware.sh
2452024-01-11T00:52:24.908Z+++ dirname ./tools/create_virtual_hardware.sh
2462024-01-11T00:52:24.908Z++ cd ./tools
2472024-01-11T00:52:24.908Z++ pwd
2482024-01-11T00:52:24.908Z+ SOURCE_DIR=/opt/oxide/work/tools
2492024-01-11T00:52:24.908Z+ OMICRON_TOP=/opt/oxide/work/tools/..
2502024-01-11T00:52:24.908Z+ SOFTNPU_MODE=zone
2512024-01-11T00:52:24.908Z+ . /opt/oxide/work/tools/virtual_hardware.sh
2522024-01-11T00:52:24.908Z++ MARKER=/etc/opt/oxide/NO_INSTALL
2532024-01-11T00:52:24.908Z++ [[ -f /etc/opt/oxide/NO_INSTALL ]]
2542024-01-11T00:52:24.908Z++ dladm show-phys -p -o LINK
2552024-01-11T00:52:24.908Z++ head -1
2562024-01-11T00:52:24.966Z+ PHYSICAL_LINK=igb0
2572024-01-11T00:52:24.966Z+ echo 'Using igb0 as physical link'
2582024-01-11T00:52:25.406ZUsing igb0 as physical link
2592024-01-11T00:52:25.406Z+ ensure_run_as_root
2602024-01-11T00:52:25.406Z++ id -u
2612024-01-11T00:52:25.407Z+ [[ 0 -ne 0 ]]
2622024-01-11T00:52:25.407Z+ ensure_zpools
2632024-01-11T00:52:25.407Z+ ZPOOL_TYPES=('oxp_' 'oxi_')
2642024-01-11T00:52:25.407Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
2652024-01-11T00:52:25.407Z+ readarray -t ZPOOLS
2662024-01-11T00:52:25.407Z++ grep '"oxp_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
2672024-01-11T00:52:25.407Z++ sed 's/[ ",]//g'
2682024-01-11T00:52:25.407Z+ for ZPOOL in "${ZPOOLS[@]}"
2692024-01-11T00:52:25.407Z+ echo 'Zpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]'
2702024-01-11T00:52:25.407ZZpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]
2712024-01-11T00:52:25.407Z+ VDEV_PATH=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2722024-01-11T00:52:25.407Z+ [[ -f /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
2732024-01-11T00:52:25.408Z+ dd if=/dev/zero of=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
2742024-01-11T00:52:25.433Z0+0 records in
2752024-01-11T00:52:25.434Z0+0 records out
2762024-01-11T00:52:25.434Z0 bytes (0 iB) transferred in 0.000005 secs (0 iB/sec)
2772024-01-11T00:52:25.434Z+ success 'ZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
2782024-01-11T00:52:25.434Z+ set +x
2792024-01-11T00:52:25.434ZZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
2802024-01-11T00:52:25.434Z++ zpool list -o name
2812024-01-11T00:52:25.434Z++ grep oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
2822024-01-11T00:52:25.434Z+ [[ -z '' ]]
2832024-01-11T00:52:25.434Z+ zpool create -f oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2842024-01-11T00:52:25.435Z+ success 'ZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
2852024-01-11T00:52:25.435Z+ set +x
2862024-01-11T00:52:25.435ZZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists
2872024-01-11T00:52:25.435Z+ for ZPOOL in "${ZPOOLS[@]}"
2882024-01-11T00:52:25.435Z+ echo 'Zpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
2892024-01-11T00:52:25.435ZZpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]
2902024-01-11T00:52:25.435Z+ VDEV_PATH=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
2912024-01-11T00:52:25.435Z+ [[ -f /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
2922024-01-11T00:52:25.435Z+ dd if=/dev/zero of=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
2932024-01-11T00:52:25.435Z0+0 records in
2942024-01-11T00:52:25.436Z0+0 records out
2952024-01-11T00:52:25.436Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
2962024-01-11T00:52:25.436Z+ success 'ZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
2972024-01-11T00:52:25.436Z+ set +x
2982024-01-11T00:52:25.436ZZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
2992024-01-11T00:52:25.436Z++ zpool list -o name
3002024-01-11T00:52:25.436Z++ grep oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
3012024-01-11T00:52:25.436Z+ [[ -z '' ]]
3022024-01-11T00:52:25.436Z+ zpool create -f oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3032024-01-11T00:52:25.436Z+ success 'ZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3042024-01-11T00:52:25.436Z+ set +x
3052024-01-11T00:52:25.436ZZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3062024-01-11T00:52:25.436Z+ for ZPOOL in "${ZPOOLS[@]}"
3072024-01-11T00:52:25.436Z+ echo 'Zpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3082024-01-11T00:52:25.436ZZpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]
3092024-01-11T00:52:25.436Z+ VDEV_PATH=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3102024-01-11T00:52:25.437Z+ [[ -f /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3112024-01-11T00:52:25.437Z+ dd if=/dev/zero of=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3122024-01-11T00:52:25.437Z0+0 records in
3132024-01-11T00:52:25.437Z0+0 records out
3142024-01-11T00:52:25.437Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3152024-01-11T00:52:25.437Z+ success 'ZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3162024-01-11T00:52:25.437Z+ set +x
3172024-01-11T00:52:25.437ZZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3182024-01-11T00:52:25.437Z++ zpool list -o name
3192024-01-11T00:52:25.437Z++ grep oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
3202024-01-11T00:52:25.437Z+ [[ -z '' ]]
3212024-01-11T00:52:25.437Z+ zpool create -f oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3222024-01-11T00:52:25.437Z+ success 'ZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3232024-01-11T00:52:25.438Z+ set +x
3242024-01-11T00:52:25.438ZZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3252024-01-11T00:52:25.438ZZpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]
3262024-01-11T00:52:25.438Z+ for ZPOOL in "${ZPOOLS[@]}"
3272024-01-11T00:52:25.438Z+ echo 'Zpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3282024-01-11T00:52:25.438Z+ VDEV_PATH=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3292024-01-11T00:52:25.438Z+ [[ -f /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3302024-01-11T00:52:25.438Z+ dd if=/dev/zero of=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3312024-01-11T00:52:25.438Z0+0 records in
3322024-01-11T00:52:25.439Z0+0 records out
3332024-01-11T00:52:25.439Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3342024-01-11T00:52:25.439Z+ success 'ZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3352024-01-11T00:52:25.439Z+ set +x
3362024-01-11T00:52:25.439ZZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3372024-01-11T00:52:25.439Z++ zpool list -o name
3382024-01-11T00:52:25.439Z++ grep oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03
3392024-01-11T00:52:25.439Z+ [[ -z '' ]]
3402024-01-11T00:52:25.439Z+ zpool create -f oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3412024-01-11T00:52:25.439Z+ success 'ZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3422024-01-11T00:52:25.440Z+ set +x
3432024-01-11T00:52:25.440ZZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3442024-01-11T00:52:25.440Z+ for ZPOOL in "${ZPOOLS[@]}"
3452024-01-11T00:52:25.441ZZpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]
3462024-01-11T00:52:25.441Z+ echo 'Zpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3472024-01-11T00:52:25.441Z+ VDEV_PATH=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3482024-01-11T00:52:25.441Z+ [[ -f /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3492024-01-11T00:52:25.441Z+ dd if=/dev/zero of=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3502024-01-11T00:52:25.441Z0+0 records in
3512024-01-11T00:52:25.441Z0+0 records out
3522024-01-11T00:52:25.441Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3532024-01-11T00:52:25.441Z+ success 'ZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3542024-01-11T00:52:25.441Z+ set +x
3552024-01-11T00:52:25.441ZZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3562024-01-11T00:52:25.441Z++ zpool list -o name
3572024-01-11T00:52:25.442Z++ grep oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03
3582024-01-11T00:52:25.442Z+ [[ -z '' ]]
3592024-01-11T00:52:25.442Z+ zpool create -f oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3602024-01-11T00:52:25.442Z+ success 'ZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3612024-01-11T00:52:25.442Z+ set +x
3622024-01-11T00:52:25.442ZZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3632024-01-11T00:52:25.443Z+ for ZPOOL in "${ZPOOLS[@]}"
3642024-01-11T00:52:25.443Z+ echo 'Zpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]'
3652024-01-11T00:52:25.443ZZpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]
3662024-01-11T00:52:25.443Z+ VDEV_PATH=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3672024-01-11T00:52:25.443Z+ [[ -f /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev ]]
3682024-01-11T00:52:25.443Z+ dd if=/dev/zero of=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev bs=1 count=0 seek=20G
3692024-01-11T00:52:25.443Z0+0 records in
3702024-01-11T00:52:25.443Z0+0 records out
3712024-01-11T00:52:25.443Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3722024-01-11T00:52:25.443Z+ success 'ZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists'
3732024-01-11T00:52:25.443Z+ set +x
3742024-01-11T00:52:25.491ZZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists
3752024-01-11T00:52:25.491Z++ zpool list -o name
3762024-01-11T00:52:25.491Z++ grep oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930
3772024-01-11T00:52:25.491Z+ [[ -z '' ]]
3782024-01-11T00:52:25.491Z+ zpool create -f oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3792024-01-11T00:52:25.492Z+ success 'ZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists'
3802024-01-11T00:52:25.492Z+ set +x
3812024-01-11T00:52:25.492ZZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists
3822024-01-11T00:52:25.493Z+ for ZPOOL in "${ZPOOLS[@]}"
3832024-01-11T00:52:25.493Z+ echo 'Zpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]'
3842024-01-11T00:52:25.493ZZpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]
3852024-01-11T00:52:25.493Z+ VDEV_PATH=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
3862024-01-11T00:52:25.493Z+ [[ -f /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev ]]
3872024-01-11T00:52:25.493Z+ dd if=/dev/zero of=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev bs=1 count=0 seek=20G
3882024-01-11T00:52:25.493Z0+0 records in
3892024-01-11T00:52:25.493Z0+0 records out
3902024-01-11T00:52:25.493Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3912024-01-11T00:52:25.494Z+ success 'ZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists'
3922024-01-11T00:52:25.494Z+ set +x
3932024-01-11T00:52:25.494ZZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists
3942024-01-11T00:52:25.494Z++ zpool list -o name
3952024-01-11T00:52:25.494Z++ grep oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60
3962024-01-11T00:52:25.494Z+ [[ -z '' ]]
3972024-01-11T00:52:25.494Z+ zpool create -f oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
3982024-01-11T00:52:25.494Z+ success 'ZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists'
3992024-01-11T00:52:25.494Z+ set +x
4002024-01-11T00:52:25.494ZZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists
4012024-01-11T00:52:25.494ZZpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]
4022024-01-11T00:52:25.494Z+ for ZPOOL in "${ZPOOLS[@]}"
4032024-01-11T00:52:25.494Z+ echo 'Zpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]'
4042024-01-11T00:52:25.495Z+ VDEV_PATH=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4052024-01-11T00:52:25.495Z+ [[ -f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev ]]
4062024-01-11T00:52:25.496Z+ dd if=/dev/zero of=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev bs=1 count=0 seek=20G
4072024-01-11T00:52:25.496Z0+0 records in
4082024-01-11T00:52:25.496Z0+0 records out
4092024-01-11T00:52:25.496Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4102024-01-11T00:52:25.496Z+ success 'ZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists'
4112024-01-11T00:52:25.496Z+ set +x
4122024-01-11T00:52:25.496ZZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists
4132024-01-11T00:52:25.496Z++ zpool list -o name
4142024-01-11T00:52:25.496Z++ grep oxp_31bd71cd-4736-4a12-a387-9b74b050396f
4152024-01-11T00:52:25.496Z+ [[ -z '' ]]
4162024-01-11T00:52:25.496Z+ zpool create -f oxp_31bd71cd-4736-4a12-a387-9b74b050396f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4172024-01-11T00:52:25.496Z+ success 'ZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists'
4182024-01-11T00:52:25.497Z+ set +x
4192024-01-11T00:52:25.497ZZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists
4202024-01-11T00:52:25.497Z+ for ZPOOL in "${ZPOOLS[@]}"
4212024-01-11T00:52:25.497Z+ echo 'Zpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]'
4222024-01-11T00:52:25.497ZZpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]
4232024-01-11T00:52:25.497Z+ VDEV_PATH=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4242024-01-11T00:52:25.497Z+ [[ -f /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev ]]
4252024-01-11T00:52:25.497Z+ dd if=/dev/zero of=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev bs=1 count=0 seek=20G
4262024-01-11T00:52:25.497Z0+0 records in
4272024-01-11T00:52:25.497Z0+0 records out
4282024-01-11T00:52:25.497Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4292024-01-11T00:52:25.497Z+ success 'ZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists'
4302024-01-11T00:52:25.497Z+ set +x
4312024-01-11T00:52:25.498ZZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists
4322024-01-11T00:52:25.498Z++ zpool list -o name
4332024-01-11T00:52:25.498Z++ grep oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7
4342024-01-11T00:52:25.498Z+ [[ -z '' ]]
4352024-01-11T00:52:25.498Z+ zpool create -f oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4362024-01-11T00:52:25.498Z+ success 'ZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists'
4372024-01-11T00:52:25.498Z+ set +x
4382024-01-11T00:52:25.499ZZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists
4392024-01-11T00:52:25.499Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
4402024-01-11T00:52:25.499Z+ readarray -t ZPOOLS
4412024-01-11T00:52:25.499Z++ grep '"oxi_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
4422024-01-11T00:52:25.499Z++ sed 's/[ ",]//g'
4432024-01-11T00:52:25.499Z+ for ZPOOL in "${ZPOOLS[@]}"
4442024-01-11T00:52:25.499Z+ echo 'Zpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4452024-01-11T00:52:25.499ZZpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]
4462024-01-11T00:52:25.499Z+ VDEV_PATH=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4472024-01-11T00:52:25.499Z+ [[ -f /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4482024-01-11T00:52:25.500Z+ dd if=/dev/zero of=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4492024-01-11T00:52:25.500Z0+0 records in
4502024-01-11T00:52:25.500Z0+0 records out
4512024-01-11T00:52:25.500Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4522024-01-11T00:52:25.500Z+ success 'ZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4532024-01-11T00:52:25.500Z+ set +x
4542024-01-11T00:52:25.500ZZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4552024-01-11T00:52:25.500Z++ zpool list -o name
4562024-01-11T00:52:25.500Z++ grep oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b
4572024-01-11T00:52:25.500Z+ [[ -z '' ]]
4582024-01-11T00:52:25.500Z+ zpool create -f oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4592024-01-11T00:52:25.533Z+ success 'ZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4602024-01-11T00:52:25.533Z+ set +x
4612024-01-11T00:52:25.533ZZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4622024-01-11T00:52:25.533Z+ for ZPOOL in "${ZPOOLS[@]}"
4632024-01-11T00:52:25.534Z+ echo 'Zpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4642024-01-11T00:52:25.534ZZpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]
4652024-01-11T00:52:25.534Z+ VDEV_PATH=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4662024-01-11T00:52:25.534Z+ [[ -f /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4672024-01-11T00:52:25.534Z+ dd if=/dev/zero of=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4682024-01-11T00:52:25.534Z0+0 records in
4692024-01-11T00:52:25.534Z0+0 records out
4702024-01-11T00:52:25.534Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4712024-01-11T00:52:25.534Z+ success 'ZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4722024-01-11T00:52:25.534Z+ set +x
4732024-01-11T00:52:25.534ZZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4742024-01-11T00:52:25.534Z++ zpool list -o name
4752024-01-11T00:52:25.534Z++ grep oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b
4762024-01-11T00:52:25.535Z+ [[ -z '' ]]
4772024-01-11T00:52:25.535Z+ zpool create -f oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4782024-01-11T00:52:25.564Z+ success 'ZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4792024-01-11T00:52:25.564Z+ set +x
4802024-01-11T00:52:25.564ZZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4812024-01-11T00:52:25.564Z+ [[ zone == \z\o\n\e ]]
4822024-01-11T00:52:25.564Z+ ensure_simulated_links igb0
4832024-01-11T00:52:25.565Z+ local PHYSICAL_LINK=igb0
4842024-01-11T00:52:25.565Z+ INDICES=("0" "1")
4852024-01-11T00:52:25.565Z+ for I in "${INDICES[@]}"
4862024-01-11T00:52:25.565Z++ get_simnet_name_if_exists net0
4872024-01-11T00:52:25.565Z++ dladm show-simnet -p -o LINK net0
4882024-01-11T00:52:25.565Z++ echo ''
4892024-01-11T00:52:25.565Z+ [[ -z '' ]]
4902024-01-11T00:52:25.565Z+ dladm create-simnet -t net0
4912024-01-11T00:52:25.565Z+ dladm create-simnet -t sc0_0
4922024-01-11T00:52:25.565Z+ dladm modify-simnet -t -p net0 sc0_0
4932024-01-11T00:52:25.565Z+ dladm set-linkprop -p mtu=9000 sc0_0
4942024-01-11T00:52:25.565Z+ success 'Simnet net0/sc0_0 exists'
4952024-01-11T00:52:25.566Z+ set +x
4962024-01-11T00:52:25.566ZSimnet net0/sc0_0 exists
4972024-01-11T00:52:25.566Z+ for I in "${INDICES[@]}"
4982024-01-11T00:52:25.566Z++ get_simnet_name_if_exists net1
4992024-01-11T00:52:25.566Z++ dladm show-simnet -p -o LINK net1
5002024-01-11T00:52:25.566Z++ echo ''
5012024-01-11T00:52:25.591Z+ [[ -z '' ]]
5022024-01-11T00:52:25.591Z+ dladm create-simnet -t net1
5032024-01-11T00:52:25.591Z+ dladm create-simnet -t sc1_0
5042024-01-11T00:52:25.591Z+ dladm modify-simnet -t -p net1 sc1_0
5052024-01-11T00:52:25.592Z+ dladm set-linkprop -p mtu=9000 sc1_0
5062024-01-11T00:52:25.592Z+ success 'Simnet net1/sc1_0 exists'
5072024-01-11T00:52:25.592Z+ set +x
5082024-01-11T00:52:25.592ZSimnet net1/sc1_0 exists
5092024-01-11T00:52:25.593Z++ get_vnic_name_if_exists sc0_1
5102024-01-11T00:52:25.593Z++ dladm show-vnic -p -o LINK sc0_1
5112024-01-11T00:52:25.593Z++ echo ''
5122024-01-11T00:52:25.593Z+ [[ -z '' ]]
5132024-01-11T00:52:25.593Z+ dladm create-vnic -t sc0_1 -l igb0 -m a8:e1:de:01:70:1d
5142024-01-11T00:52:25.593Z+ [[ -v PROMISC_FILT_OFF ]]
5152024-01-11T00:52:25.593Z+ success 'Vnic sc0_1 exists'
5162024-01-11T00:52:25.593Z+ set +x
5172024-01-11T00:52:25.593ZVnic sc0_1 exists
5182024-01-11T00:52:25.593Z+ warn_if_no_proxy_arp
5192024-01-11T00:52:25.593Z+ [[ -v PXA_START ]]
5202024-01-11T00:52:25.593Z+ [[ -v PXA_END ]]
5212024-01-11T00:52:25.593Z+ ensure_softnpu_zone
5222024-01-11T00:52:25.618Z+ zoneadm list
5232024-01-11T00:52:25.618Z+ grep -q sidecar_softnpu
5242024-01-11T00:52:25.618Z+ [[ -f out/npuzone/npuzone ]]
5252024-01-11T00:52:25.618Z+ out/npuzone/npuzone create sidecar --omicron-zone --ports sc0_0,tfportrear0_0 --ports sc0_1,tfportqsfp0_0 --sidecar-lite-commit e3ea4b495ba0a71801ded0776ae4bbd31df57e26 --softnpu-commit dbab082dfa89da5db5ca2325c257089d2f130092
5262024-01-11T00:52:25.618Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/softnpu/image/dbab082dfa89da5db5ca2325c257089d2f130092/softnpu.sha256.txt
5272024-01-11T00:52:31.419Zdownloading softnpu ...done
5282024-01-11T00:52:31.444Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/libsidecar_lite.so.sha256.txt
5292024-01-11T00:52:37.558Zdownloading asic_program.so ...done
5302024-01-11T00:52:37.584Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/scadm.sha256.txt
5312024-01-11T00:52:44.386Zdownloading scadm ...done
5322024-01-11T00:52:44.411Zcreating port tfportrear0_0 for sc0_0
5332024-01-11T00:52:44.412Zcreating port tfportqsfp0_0 for sc0_1
5342024-01-11T00:52:44.412Zsoftnpu zone setup
5352024-01-11T00:52:44.412Zconfigure zone
5362024-01-11T00:52:44.412Zinstall zone
5372024-01-11T00:52:45.474Zboot zone
5382024-01-11T00:52:48.581Z[sidecar_softnpu] svcs milestone/network
5392024-01-11T00:52:48.608ZSTATE STIME FMRI
5402024-01-11T00:52:48.608Zonline 0:52:46 svc:/milestone/network:default
5412024-01-11T00:52:48.608Z[sidecar_softnpu] /softnpu/npu start
5422024-01-11T00:52:48.635Z
5432024-01-11T00:52:48.661Z+ /opt/oxide/work/tools/scrimlet/softnpu-init.sh
5442024-01-11T00:52:48.662Z+ GATEWAY_IP=10.151.1.100
5452024-01-11T00:52:48.662Z+ echo 'Using 10.151.1.100 as gateway ip'
5462024-01-11T00:52:48.662ZUsing 10.151.1.100 as gateway ip
5472024-01-11T00:52:48.662Z+ [[ ! -v GATEWAY_MAC ]]
5482024-01-11T00:52:48.662Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
5492024-01-11T00:52:48.662Z++ grep -c -E '^([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}$' /dev/fd/63
5502024-01-11T00:52:48.663Z+++ echo 18:c0:4d:0d:9f:b2
5512024-01-11T00:52:48.663Z+ COUNT=1
5522024-01-11T00:52:48.663Z+ [[ 1 -ne 1 ]]
5532024-01-11T00:52:48.663Z+ echo 'Using 18:c0:4d:0d:9f:b2 as gateway mac'
5542024-01-11T00:52:48.663ZUsing 18:c0:4d:0d:9f:b2 as gateway mac
5552024-01-11T00:52:48.663Z+ z_scadm add-arp-entry 10.151.1.100 18:c0:4d:0d:9f:b2
5562024-01-11T00:52:48.663Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-arp-entry 10.151.1.100 18:c0:4d:0d:9f:b2
5572024-01-11T00:52:48.688Z+ PXA_MAC=a8:e1:de:01:70:1d
5582024-01-11T00:52:48.689Z+ [[ -v PXA_START ]]
5592024-01-11T00:52:48.689Z+ z_scadm add-proxy-arp 10.151.1.164 10.151.1.227 a8:e1:de:01:70:1d
5602024-01-11T00:52:48.689Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-proxy-arp 10.151.1.164 10.151.1.227 a8:e1:de:01:70:1d
5612024-01-11T00:52:48.714Z+ z_scadm dump-state
5622024-01-11T00:52:48.714Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
5632024-01-11T00:52:48.740Zlocal v6:
5642024-01-11T00:52:48.740Zlocal v4:
5652024-01-11T00:52:48.740Zrouter v6:
5662024-01-11T00:52:48.740Zrouter v4_idx:
5672024-01-11T00:52:48.740Zrouter v4_routes:
5682024-01-11T00:52:48.740Zresolver v4:
5692024-01-11T00:52:48.740Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
5702024-01-11T00:52:48.740Zresolver v6:
5712024-01-11T00:52:48.741Znat v4:
5722024-01-11T00:52:48.741Znat_v6:
5732024-01-11T00:52:48.742Zport_mac:
5742024-01-11T00:52:48.742Zproxy arp:
5752024-01-11T00:52:48.742Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
5762024-01-11T00:52:48.742Z+ success 'softnpu zone exists'
5772024-01-11T00:52:48.742Z+ set +x
5782024-01-11T00:52:48.742Zsoftnpu zone exists
5792024-01-11T00:52:48.742Z
5802024-01-11T00:52:48.742Zreal 23.846237620
5812024-01-11T00:52:48.742Zuser 0.681526137
5822024-01-11T00:52:48.742Zsys 1.613507839
5832024-01-11T00:52:48.742Ztrap 0.000071437
5842024-01-11T00:52:48.742Ztflt 0.000115075
5852024-01-11T00:52:48.742Zdflt 0.000936808
5862024-01-11T00:52:48.742Zkflt 0.000000000
5872024-01-11T00:52:48.742Zlock 18:23.644738945
5882024-01-11T00:52:48.742Zslp 1:11.268351319
5892024-01-11T00:52:48.742Zlat 0.068094317
5902024-01-11T00:52:48.742Zstop 0.346209724
5912024-01-11T00:52:48.742Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml
5922024-01-11T00:52:48.769Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
5932024-01-11T00:52:48.800Z+ SILO_NAME=recovery
5942024-01-11T00:52:48.800Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
5952024-01-11T00:52:48.800Z+ EXTERNAL_DNS_DOMAIN=oxide.test
5962024-01-11T00:52:48.800Z+ sed -i~ '
5972024-01-11T00:52:48.800Z /^external_dns_ips/c\
5982024-01-11T00:52:48.800Zexternal_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
5992024-01-11T00:52:48.801Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
6002024-01-11T00:52:48.801Z /^first/c\
6012024-01-11T00:52:48.801Zfirst = "10.151.1.164"
6022024-01-11T00:52:48.801Z /^last/c\
6032024-01-11T00:52:48.801Zlast = "10.151.1.173"
6042024-01-11T00:52:48.801Z }
6052024-01-11T00:52:48.801Z /^\[rack_network_config/,/^$/ {
6062024-01-11T00:52:48.801Z /^infra_ip_first/c\
6072024-01-11T00:52:48.801Zinfra_ip_first = "10.151.1.174"
6082024-01-11T00:52:48.801Z /^infra_ip_last/c\
6092024-01-11T00:52:48.801Zinfra_ip_last = "10.151.1.174"
6102024-01-11T00:52:48.801Z }
6112024-01-11T00:52:48.802Z /^\[\[rack_network_config.ports/,/^$/ {
6122024-01-11T00:52:48.802Z /^routes/c\
6132024-01-11T00:52:48.802Zroutes = \[{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}\]
6142024-01-11T00:52:48.802Z /^addresses/c\
6152024-01-11T00:52:48.802Zaddresses = \["10.151.1.174/32"\]
6162024-01-11T00:52:48.802Z }
6172024-01-11T00:52:48.802Z' pkg/config-rss.toml
6182024-01-11T00:52:48.802Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
6192024-01-11T00:52:48.802Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
6202024-01-11T00:52:48.802Z+++ pkg/config-rss.toml Thu Jan 11 00:52:48 2024
6212024-01-11T00:52:48.802Z@@ -41,7 +41,7 @@
6222024-01-11T00:52:48.802Z # the DNS domain delegated to the rack by the customer. Each of these addresses
6232024-01-11T00:52:48.802Z # must be contained in one of the "internal services" IP Pool ranges listed
6242024-01-11T00:52:48.802Z # below.
6252024-01-11T00:52:48.803Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
6262024-01-11T00:52:48.803Z+external_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
6272024-01-11T00:52:48.803Z
6282024-01-11T00:52:48.803Z # Initial TLS certificates for the external API
6292024-01-11T00:52:48.803Z #
6302024-01-11T00:52:48.803Z@@ -77,8 +77,8 @@
6312024-01-11T00:52:48.803Z #
6322024-01-11T00:52:48.803Z # For more on this and what to put here, see docs/how-to-run.adoc.
6332024-01-11T00:52:48.803Z [[internal_services_ip_pool_ranges]]
6342024-01-11T00:52:48.803Z-first = "192.168.1.20"
6352024-01-11T00:52:48.803Z-last = "192.168.1.29"
6362024-01-11T00:52:48.803Z+first = "10.151.1.164"
6372024-01-11T00:52:48.804Z+last = "10.151.1.173"
6382024-01-11T00:52:48.804Z
6392024-01-11T00:52:48.804Z # TODO - this configuration is subject to change going forward. Ultimately these
6402024-01-11T00:52:48.804Z # parameters should be provided to the control plane via wicket, but we need to
6412024-01-11T00:52:48.804Z@@ -92,8 +92,8 @@
6422024-01-11T00:52:48.804Z # A range of IP addresses used by Boundary Services on the external network. In
6432024-01-11T00:52:48.804Z # a real system, these would be addresses of the uplink ports on the Sidecar.
6442024-01-11T00:52:48.804Z # With softnpu, only one address is used.
6452024-01-11T00:52:48.804Z-infra_ip_first = "192.168.1.30"
6462024-01-11T00:52:48.805Z-infra_ip_last = "192.168.1.30"
6472024-01-11T00:52:48.805Z+infra_ip_first = "10.151.1.174"
6482024-01-11T00:52:48.805Z+infra_ip_last = "10.151.1.174"
6492024-01-11T00:52:48.805Z
6502024-01-11T00:52:48.805Z # Configurations for BGP routers to run on the scrimlets.
6512024-01-11T00:52:48.806Z bgp = []
6522024-01-11T00:52:48.806Z@@ -101,9 +101,9 @@
6532024-01-11T00:52:48.806Z # You can configure multiple uplinks by repeating the following stanza
6542024-01-11T00:52:48.806Z [[rack_network_config.ports]]
6552024-01-11T00:52:48.806Z # Routes associated with this port.
6562024-01-11T00:52:48.806Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
6572024-01-11T00:52:48.806Z+routes = [{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}]
6582024-01-11T00:52:48.807Z # Addresses associated with this port.
6592024-01-11T00:52:48.807Z-addresses = ["192.168.1.30/32"]
6602024-01-11T00:52:48.807Z+addresses = ["10.151.1.174/32"]
6612024-01-11T00:52:48.807Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
6622024-01-11T00:52:48.808Z port = "qsfp0"
6632024-01-11T00:52:48.808Z # The speed of this port.
6642024-01-11T00:52:48.808Z+ true
6652024-01-11T00:52:48.808Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml
6662024-01-11T00:52:48.808Za pkg/config-rss.toml 7K
6672024-01-11T00:52:48.809Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~
6682024-01-11T00:52:48.809Z+ TLS_NAME=recovery.sys.oxide.test
6692024-01-11T00:52:48.809Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
6702024-01-11T00:52:49.181Z.....+..........+.....+...............+............+.......+...+..+...+.......+......+.........+...........+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+...+............+.+.....+...+.........+.......+...+...............+......+..+.......+..+...+.......+......+..+......+.........................+..+...............+...+.+.....+.........+...................+..+......+....+........................+......+.....+..........+...+...+..+......+.+......+.....+...+.......+.....+.........+...+.............+.....+.........+...+.+.....+.......+..+.+......+...............+.....+....+...........+....+.........+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6712024-01-11T00:52:50.605Z...+...+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+.+.........+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+......+...+......+...+...+..+............+.........+.+...+........+....+.......................+.+..+.......+............+...+...+..+...............+.......+......+.....+....+...+...+........+......+.........+..........+...........+...+.+.....+.........+......................+.....+....+.....+............+...+.+...+...+...+.........+..+....................................+..................+..........+...........+.........+....+..+....+...........+..........+...+.....+.+..+...............+...+....+..+..........+.....+.+..+.......+...+.....+................+.....+................+.....+.............+.....+.............+...+......+.......................+..................+...+.....................+..........+...+............+.....................+...+..+.......+..+......+...+....+...+...............+..+............+......+....+...+......+..+....+........+.+..............+............+.+.....+......................+..+.+...+..+....+..............+....+......+..+.........+.+..+.............+..+......................+...+..+.+............+......+.........+.....+...+....+........+...+...+..........+...............+......+.....+..........+.....+.+............+.....+.......+........+.+......+.....................+..+.+......+.....+..........+......+..+.......+.......................+.......+.........+......+...+......+..+..........+..+.+.....+.........+......+.+......+..............+....+.........+.....+...+....+...+..+....+.........+...+........+.......+................................+.......+......+........+......+...+...+......................+........+..........+.............................+......+...+......+.+...+............+..+...............+...+.......+.....+............+...............+.........+.......+..............+.........+.........+...+.........+..........+........+..............................+...+.......+.....+.....................+......+..........+.........+.....+...................+..+.+..............................+..+......+......+....+..+.........+.+.........+...+........+...+.......+.....+.........+..........+...........+....+...+.....+............+.+...............+..+..................+..........+..+.......+...............+...........+...+.+...+..+.+..+...+.......+...+.....+...................+..............+....+......+.....+....+...+..............+....+..+...+.........+..........+..+...............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6722024-01-11T00:52:50.632Z-----
6732024-01-11T00:52:50.632Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6742024-01-11T00:52:50.659Za pkg/initial-tls-cert.pem 2K
6752024-01-11T00:52:50.659Za pkg/initial-tls-key.pem 4K
6762024-01-11T00:52:50.659Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6772024-01-11T00:52:50.660Z+ rmdir pkg
6782024-01-11T00:52:50.660Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
6792024-01-11T00:52:50.660Z+ pfexec mkdir -p /usr/oxide
6802024-01-11T00:52:50.660Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0
6812024-01-11T00:52:51.441Z+ pfexec chmod +x /usr/oxide/oxide
6822024-01-11T00:52:51.466Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
6832024-01-11T00:53:09.737Z+ OMICRON_NO_UNINSTALL=1
6842024-01-11T00:53:09.737Z+ ptime -m pfexec ./target/release/omicron-package -t test install
6852024-01-11T00:53:09.763ZJan 11 00:53:09.204 DEBG target[test]: Target({"image": "standard", "machine": "non-gimlet", "rack-topology": "single-sled", "switch": "softnpu"})
6862024-01-11T00:53:09.763ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/clickhouse_keeper.tar.gz, src: out/clickhouse_keeper.tar.gz
6872024-01-11T00:53:09.763ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/clickhouse.tar.gz, src: out/clickhouse.tar.gz
6882024-01-11T00:53:09.763ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/cockroachdb.tar.gz, src: out/cockroachdb.tar.gz
6892024-01-11T00:53:09.763ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/crucible_pantry.tar.gz, src: out/crucible-pantry.tar.gz
6902024-01-11T00:53:09.763ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/sled-agent.tar, src: out/omicron-sled-agent.tar
6912024-01-11T00:53:09.763ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/nexus.tar.gz, src: out/omicron-nexus.tar.gz
6922024-01-11T00:53:09.763ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/external_dns.tar.gz, src: out/external-dns.tar.gz
6932024-01-11T00:53:09.763ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/oximeter.tar.gz, src: out/oximeter-collector.tar.gz
6942024-01-11T00:53:09.763ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/overlay.tar.gz, src: out/overlay.tar.gz
6952024-01-11T00:53:09.764ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/internal_dns.tar.gz, src: out/internal-dns.tar.gz
6962024-01-11T00:53:09.764ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/crucible.tar.gz, src: out/crucible.tar.gz
6972024-01-11T00:53:09.764ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/propolis-server.tar.gz, src: out/propolis-server.tar.gz
6982024-01-11T00:53:09.764ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/ntp.tar.gz, src: out/ntp.tar.gz
6992024-01-11T00:53:09.764ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/mg-ddm.tar, src: out/mg-ddm-gz.tar
7002024-01-11T00:53:09.764ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/switch.tar.gz, src: out/switch-softnpu.tar.gz
7012024-01-11T00:53:09.764ZJan 11 00:53:09.206 INFO Installing service, dst: /opt/oxide/mgs.tar.gz, src: out/omicron-gateway-softnpu.tar.gz
7022024-01-11T00:53:10.003ZJan 11 00:53:09.455 INFO Unpacking service tarball, service_path: /opt/oxide/mg-ddm, tar_path: /opt/oxide/mg-ddm.tar
7032024-01-11T00:53:10.037ZJan 11 00:53:09.490 INFO Unpacking service tarball, service_path: /opt/oxide/sled-agent, tar_path: /opt/oxide/sled-agent.tar
7042024-01-11T00:53:10.097ZJan 11 00:53:09.549 INFO Installing bootstrap service from /opt/oxide/sled-agent/pkg/manifest.xml
7052024-01-11T00:53:10.258Z
7062024-01-11T00:53:10.283Zreal 0.484464098
7072024-01-11T00:53:10.283Zuser 0.099549482
7082024-01-11T00:53:10.283Zsys 2.102109081
7092024-01-11T00:53:10.283Ztrap 0.000019210
7102024-01-11T00:53:10.284Ztflt 0.000614523
7112024-01-11T00:53:10.284Zdflt 0.000705951
7122024-01-11T00:53:10.284Zkflt 0.000000000
7132024-01-11T00:53:10.284Zlock 43.360983283
7142024-01-11T00:53:10.284Zslp 0.695055223
7152024-01-11T00:53:10.284Zlat 0.003234201
7162024-01-11T00:53:10.284Zstop 0.186974038
7172024-01-11T00:53:10.285Z+ retry=0
7182024-01-11T00:53:10.285Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7192024-01-11T00:53:10.285Z+ [[ 0 -gt 30 ]]
7202024-01-11T00:53:10.285Z+ sleep 1
7212024-01-11T00:53:11.249Z+ retry=1
7222024-01-11T00:53:11.249Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7232024-01-11T00:53:11.290Z+ [[ 1 -gt 30 ]]
7242024-01-11T00:53:11.290Z+ sleep 1
7252024-01-11T00:53:12.258Z+ retry=2
7262024-01-11T00:53:12.258Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7272024-01-11T00:53:12.286Z+ [[ 2 -gt 30 ]]
7282024-01-11T00:53:12.286Z+ sleep 1
7292024-01-11T00:53:13.273Z+ retry=3
7302024-01-11T00:53:13.274Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7312024-01-11T00:53:13.305Z+ [[ 3 -gt 30 ]]
7322024-01-11T00:53:13.305Z+ sleep 1
7332024-01-11T00:53:14.290Z+ retry=4
7342024-01-11T00:53:14.290Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7352024-01-11T00:53:39.461Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
7362024-01-11T00:53:39.487Zlocal v6:
7372024-01-11T00:53:39.488Zfe80::c00a:14ff:fe01:d11a
7382024-01-11T00:53:39.488Zfd00:99::1
7392024-01-11T00:53:39.488Zlocal v4:
7402024-01-11T00:53:39.488Z10.151.1.174
7412024-01-11T00:53:39.488Zrouter v6:
7422024-01-11T00:53:39.488Zfd00:1122:3344:3::/64 -> fe80::f893:c0ff:fedf:895b (1)
7432024-01-11T00:53:39.488Zfd00:1122:3344:101::/64 -> fe80::f893:c0ff:fedf:895b (1)
7442024-01-11T00:53:39.488Zfd00:1122:3344:1::/64 -> fe80::f893:c0ff:fedf:895b (1)
7452024-01-11T00:53:39.488Zfd00:1122:3344:2::/64 -> fe80::f893:c0ff:fedf:895b (1)
7462024-01-11T00:53:39.488Zfdb0:18c0:4d0d:9fb2::/64 -> fe80::f893:c0ff:fedf:895b (1)
7472024-01-11T00:53:39.488Zrouter v4_idx:
7482024-01-11T00:53:39.488Z0.0.0.0/0 -> 0
7492024-01-11T00:53:39.488Zrouter v4_routes:
7502024-01-11T00:53:39.488Z0 -> 10.151.1.100 (2)
7512024-01-11T00:53:39.488Zresolver v4:
7522024-01-11T00:53:39.488Z10.151.1.174 -> 8a:20:c5:bb:3a:77
7532024-01-11T00:53:39.488Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
7542024-01-11T00:53:39.488Zresolver v6:
7552024-01-11T00:53:39.513Zfe80::f893:c0ff:fedf:895b -> fa:93:c0:df:89:5b
7562024-01-11T00:53:39.514Zfe80::c00a:14ff:fe01:d11a -> c2:0a:14:01:d1:1a
7572024-01-11T00:53:39.514Znat v4:
7582024-01-11T00:53:39.514Z10.151.1.169 0/16383 -> fd00:1122:3344:101::1 100/a8:40:25:ff:ca:46
7592024-01-11T00:53:39.514Znat_v6:
7602024-01-11T00:53:39.514Zport_mac:
7612024-01-11T00:53:39.514Z1: a8:40:25:71:68:a9
7622024-01-11T00:53:39.514Z1000: a8:40:25:71:68:a8
7632024-01-11T00:53:39.514Zproxy arp:
7642024-01-11T00:53:39.514Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
7652024-01-11T00:53:39.514Z+ retry=0
7662024-01-11T00:53:39.514Z+++ zoneadm list -n
7672024-01-11T00:53:39.514Z+++ grep oxz_ntp
7682024-01-11T00:53:39.514Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
7692024-01-11T00:53:39.539Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7702024-01-11T00:53:39.539Z++ true
7712024-01-11T00:53:39.539Z+ [[ '' != online ]]
7722024-01-11T00:53:39.539Z+ [[ 0 -gt 60 ]]
7732024-01-11T00:53:39.539Z+ sleep 1
7742024-01-11T00:53:40.522Z+ retry=1
7752024-01-11T00:53:40.547Z+++ zoneadm list -n
7762024-01-11T00:53:40.547Z+++ grep oxz_ntp
7772024-01-11T00:53:40.547Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
7782024-01-11T00:53:40.572Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7792024-01-11T00:53:40.572Z++ true
7802024-01-11T00:53:40.572Z+ [[ '' != online ]]
7812024-01-11T00:53:40.572Z+ [[ 1 -gt 60 ]]
7822024-01-11T00:53:40.572Z+ sleep 1
7832024-01-11T00:53:41.552Z+ retry=2
7842024-01-11T00:53:41.580Z+++ zoneadm list -n
7852024-01-11T00:53:41.580Z+++ grep oxz_ntp
7862024-01-11T00:53:41.580Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
7872024-01-11T00:53:41.580Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7882024-01-11T00:53:41.606Z++ true
7892024-01-11T00:53:41.606Z+ [[ '' != online ]]
7902024-01-11T00:53:41.606Z+ [[ 2 -gt 60 ]]
7912024-01-11T00:53:41.606Z+ sleep 1
7922024-01-11T00:53:42.581Z+ retry=3
7932024-01-11T00:53:42.606Z+++ zoneadm list -n
7942024-01-11T00:53:42.607Z+++ grep oxz_ntp
7952024-01-11T00:53:42.607Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
7962024-01-11T00:53:42.607Z+ [[ disabled != online ]]
7972024-01-11T00:53:42.607Z+ [[ 3 -gt 60 ]]
7982024-01-11T00:53:42.607Z+ sleep 1
7992024-01-11T00:53:43.606Z+ retry=4
8002024-01-11T00:53:43.632Z+++ zoneadm list -n
8012024-01-11T00:53:43.632Z+++ grep oxz_ntp
8022024-01-11T00:53:43.633Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
8032024-01-11T00:53:43.633Z+ [[ disabled != online ]]
8042024-01-11T00:53:43.633Z+ [[ 4 -gt 60 ]]
8052024-01-11T00:53:43.633Z+ sleep 1
8062024-01-11T00:53:44.626Z+ retry=5
8072024-01-11T00:53:44.651Z+++ zoneadm list -n
8082024-01-11T00:53:44.651Z+++ grep oxz_ntp
8092024-01-11T00:53:44.651Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
8102024-01-11T00:53:44.651Z+ [[ disabled != online ]]
8112024-01-11T00:53:44.652Z+ [[ 5 -gt 60 ]]
8122024-01-11T00:53:44.652Z+ sleep 1
8132024-01-11T00:53:45.655Z+ retry=6
8142024-01-11T00:53:45.679Z+++ zoneadm list -n
8152024-01-11T00:53:45.679Z+++ grep oxz_ntp
8162024-01-11T00:53:45.679Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
8172024-01-11T00:53:45.680Z+ [[ disabled != online ]]
8182024-01-11T00:53:45.680Z+ [[ 6 -gt 60 ]]
8192024-01-11T00:53:45.680Z+ sleep 1
8202024-01-11T00:53:46.674Z+ retry=7
8212024-01-11T00:53:46.698Z+++ zoneadm list -n
8222024-01-11T00:53:46.699Z+++ grep oxz_ntp
8232024-01-11T00:53:46.699Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
8242024-01-11T00:53:46.699Z+ [[ disabled != online ]]
8252024-01-11T00:53:46.699Z+ [[ 7 -gt 60 ]]
8262024-01-11T00:53:46.699Z+ sleep 1
8272024-01-11T00:53:47.695Z+ retry=8
8282024-01-11T00:53:47.719Z+++ zoneadm list -n
8292024-01-11T00:53:47.720Z+++ grep oxz_ntp
8302024-01-11T00:53:47.720Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
8312024-01-11T00:53:47.720Z+ [[ disabled != online ]]
8322024-01-11T00:53:47.720Z+ [[ 8 -gt 60 ]]
8332024-01-11T00:53:47.720Z+ sleep 1
8342024-01-11T00:53:48.717Z+ retry=9
8352024-01-11T00:53:48.742Z+++ zoneadm list -n
8362024-01-11T00:53:48.742Z+++ grep oxz_ntp
8372024-01-11T00:53:48.742Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
8382024-01-11T00:53:48.742Z+ [[ disabled != online ]]
8392024-01-11T00:53:48.743Z+ [[ 9 -gt 60 ]]
8402024-01-11T00:53:48.743Z+ sleep 1
8412024-01-11T00:53:49.747Z+ retry=10
8422024-01-11T00:53:50.185Z+++ zoneadm list -n
8432024-01-11T00:53:50.186Z+++ grep oxz_ntp
8442024-01-11T00:53:50.186Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
8452024-01-11T00:53:50.186Z+ [[ disabled != online ]]
8462024-01-11T00:53:50.186Z+ [[ 10 -gt 60 ]]
8472024-01-11T00:53:50.186Z+ sleep 1
8482024-01-11T00:53:50.763Z+ retry=11
8492024-01-11T00:53:50.793Z+++ zoneadm list -n
8502024-01-11T00:53:50.793Z+++ grep oxz_ntp
8512024-01-11T00:53:50.793Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
8522024-01-11T00:53:50.794Z+ [[ disabled != online ]]
8532024-01-11T00:53:50.794Z+ [[ 11 -gt 60 ]]
8542024-01-11T00:53:50.794Z+ sleep 1
 
8552024-01-11T00:53:50.906Zbuskin console login: Jan 11 00:53:49 buskin ip: ip: joining multicasts failed (4) on vopte0 - will use link layer broadcasts for multicast
 
8562024-01-11T00:53:51.785Z+ retry=12
8572024-01-11T00:53:51.812Z+++ zoneadm list -n
8582024-01-11T00:53:51.814Z+++ grep oxz_ntp
8592024-01-11T00:53:51.814Z++ pfexec svcs -z oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363 -Hostate oxide/ntp
8602024-01-11T00:53:51.814Z+ [[ online != online ]]
8612024-01-11T00:53:51.814Z+ echo 'Waited for chrony: 12s'
8622024-01-11T00:53:51.815Z+ export RUST_BACKTRACE=1
8632024-01-11T00:53:51.815ZWaited for chrony: 12s
8642024-01-11T00:53:51.815Z+ RUST_BACKTRACE=1
8652024-01-11T00:53:51.815Z+ export E2E_TLS_CERT IPPOOL_START IPPOOL_END
8662024-01-11T00:53:51.815Z++ ./tests/bootstrap
8672024-01-11T00:53:51.843ZUsing external DNS server at 10.151.1.164:53
8682024-01-11T00:53:51.843Z2024-01-11 00:53:51.287379934 UTC: attempting to log into API
8692024-01-11T00:54:06.873Z2024-01-11 00:54:06.851943450 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8702024-01-11T00:54:07.873Z2024-01-11 00:54:07.855664548 UTC: attempting to log into API
8712024-01-11T00:54:22.914Z2024-01-11 00:54:22.895450942 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8722024-01-11T00:54:23.913Z2024-01-11 00:54:23.896789664 UTC: attempting to log into API
8732024-01-11T00:54:38.947Z2024-01-11 00:54:38.936120939 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8742024-01-11T00:54:39.949Z2024-01-11 00:54:39.938308629 UTC: attempting to log into API
8752024-01-11T00:54:55.000Z2024-01-11 00:54:54.981051374 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8762024-01-11T00:54:55.993Z2024-01-11 00:54:55.982101929 UTC: attempting to log into API
8772024-01-11T00:55:11.032Z2024-01-11 00:55:11.020272104 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8782024-01-11T00:55:12.033Z2024-01-11 00:55:12.020827239 UTC: attempting to log into API
 
8792024-01-11T00:55:19.732ZJan 11 00:55:19 buskin ip: ip: joining multicasts failed (4) on vopte1 - will use link layer broadcasts for multicast
8802024-01-11T00:55:26.917ZJan 11 00:55:25 buskin ip: ip: joining multicasts failed (4) on vopte2 - will use link layer broadcasts for multicast
 
8812024-01-11T00:55:27.112Z2024-01-11 00:55:27.058023723 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8822024-01-11T00:55:28.107Z2024-01-11 00:55:28.059822054 UTC: attempting to log into API
8832024-01-11T00:55:46.545Z2024-01-11 00:55:43.092055698 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8842024-01-11T00:55:46.576Z2024-01-11 00:55:44.093075022 UTC: attempting to log into API
 
8852024-01-11T00:55:46.773ZJan 11 00:55:33 buskin ip: ip: joining multicasts failed (4) on vopte3 - will use link layer broadcasts for multicast
8862024-01-11T00:55:46.817ZJan 11 00:55:39 buskin ip: ip: joining multicasts failed (4) on vopte4 - will use link layer broadcasts for multicast
8872024-01-11T00:55:46.875ZJan 11 00:55:46 buskin ip: ip: joining multicasts failed (4) on vopte5 - will use link layer broadcasts for multicast
 
8882024-01-11T00:55:59.137Z2024-01-11 00:55:59.124928464 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8892024-01-11T00:56:00.142Z2024-01-11 00:56:00.125938998 UTC: attempting to log into API
8902024-01-11T00:56:01.108Z2024-01-11 00:56:00.946246877 UTC: login succeeded
8912024-01-11T00:56:01.386Zwaiting for nexus to come up...
8922024-01-11T00:56:01.386Zcreating IP pool... 10.151.1.175 - 10.151.1.227
8932024-01-11T00:56:01.620Zsetting up quotas...
8942024-01-11T00:56:01.620Zensuring datasets are ready...
8952024-01-11T00:56:03.350Zlet's roll.
8962024-01-11T00:56:03.498Z+ eval 'OXIDE_HOST=https://recovery.sys.oxide.test
8972024-01-11T00:56:03.498ZOXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
8982024-01-11T00:56:03.498ZOXIDE_TOKEN=oxide-token-77e2e59fa52f27300b351c02ab5dd57924ded953'
8992024-01-11T00:56:03.498Z++ OXIDE_HOST=https://recovery.sys.oxide.test
9002024-01-11T00:56:03.499Z++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
9012024-01-11T00:56:03.499Z++ OXIDE_TOKEN=oxide-token-77e2e59fa52f27300b351c02ab5dd57924ded953
9022024-01-11T00:56:03.499Z+ export OXIDE_HOST OXIDE_TOKEN
9032024-01-11T00:56:03.629Z+ retry=0
9042024-01-11T00:56:03.629Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9052024-01-11T00:56:03.629Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9062024-01-11T00:56:03.629Z+ [[ 0 -gt 60 ]]
9072024-01-11T00:56:03.629Z+ sleep 1
9082024-01-11T00:56:04.382Z+ retry=1
9092024-01-11T00:56:04.383Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9102024-01-11T00:56:04.466Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9112024-01-11T00:56:04.467Z+ [[ 1 -gt 60 ]]
9122024-01-11T00:56:04.467Z+ sleep 1
9132024-01-11T00:56:21.358Z+ retry=2
9142024-01-11T00:56:21.359Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9152024-01-11T00:56:21.548Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9162024-01-11T00:56:21.548Z+ [[ 2 -gt 60 ]]
9172024-01-11T00:56:21.548Z+ sleep 1
9182024-01-11T00:56:21.652Z+ retry=3
9192024-01-11T00:56:21.653Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9202024-01-11T00:56:21.659Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9212024-01-11T00:56:21.659Z+ [[ 3 -gt 60 ]]
9222024-01-11T00:56:21.659Z+ sleep 1
9232024-01-11T00:56:21.659Z+ retry=4
9242024-01-11T00:56:21.659Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9252024-01-11T00:56:21.659Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9262024-01-11T00:56:21.659Z+ [[ 4 -gt 60 ]]
9272024-01-11T00:56:21.659Z+ sleep 1
9282024-01-11T00:56:21.660Z+ retry=5
9292024-01-11T00:56:21.660Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9302024-01-11T00:56:21.660Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9312024-01-11T00:56:21.660Z+ [[ 5 -gt 60 ]]
9322024-01-11T00:56:21.660Z+ sleep 1
9332024-01-11T00:56:21.660Z+ retry=6
9342024-01-11T00:56:21.660Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9352024-01-11T00:56:21.660Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9362024-01-11T00:56:21.660Z+ [[ 6 -gt 60 ]]
9372024-01-11T00:56:21.660Z+ sleep 1
9382024-01-11T00:56:21.660Z+ retry=7
9392024-01-11T00:56:21.660Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9402024-01-11T00:56:21.661Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9412024-01-11T00:56:21.661Z+ [[ 7 -gt 60 ]]
9422024-01-11T00:56:21.661Z+ sleep 1
9432024-01-11T00:56:21.663Z+ retry=8
9442024-01-11T00:56:21.663Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9452024-01-11T00:56:21.663Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9462024-01-11T00:56:21.663Z+ [[ 8 -gt 60 ]]
9472024-01-11T00:56:21.663Z+ sleep 1
9482024-01-11T00:56:21.663Z+ retry=9
9492024-01-11T00:56:21.663Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9502024-01-11T00:56:21.663Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9512024-01-11T00:56:21.663Z+ [[ 9 -gt 60 ]]
9522024-01-11T00:56:21.663Z+ sleep 1
9532024-01-11T00:56:21.664Z+ retry=10
9542024-01-11T00:56:21.664Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9552024-01-11T00:56:21.664Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9562024-01-11T00:56:21.664Z+ [[ 10 -gt 60 ]]
9572024-01-11T00:56:21.664Z+ sleep 1
9582024-01-11T00:56:21.664Z+ retry=11
9592024-01-11T00:56:21.665Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9602024-01-11T00:56:21.700Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9612024-01-11T00:56:21.700Z+ [[ 11 -gt 60 ]]
9622024-01-11T00:56:21.701Z+ sleep 1
9632024-01-11T00:56:21.701Z+ retry=12
9642024-01-11T00:56:21.701Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9652024-01-11T00:56:21.701Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9662024-01-11T00:56:21.701Z+ [[ 12 -gt 60 ]]
9672024-01-11T00:56:21.701Z+ sleep 1
9682024-01-11T00:56:21.701Z+ retry=13
9692024-01-11T00:56:21.702Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9702024-01-11T00:56:21.702Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9712024-01-11T00:56:21.702Z+ [[ 13 -gt 60 ]]
9722024-01-11T00:56:21.702Z+ sleep 1
9732024-01-11T00:56:21.702Z+ retry=14
9742024-01-11T00:56:21.702Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9752024-01-11T00:56:21.703Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9762024-01-11T00:56:21.703Z+ [[ 14 -gt 60 ]]
9772024-01-11T00:56:21.703Z+ sleep 1
9782024-01-11T00:56:21.703Z+ retry=15
9792024-01-11T00:56:21.703Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9802024-01-11T00:56:21.703Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9812024-01-11T00:56:21.703Z+ [[ 15 -gt 60 ]]
9822024-01-11T00:56:21.703Z+ sleep 1
9832024-01-11T00:56:21.704Z+ retry=16
9842024-01-11T00:56:21.704Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9852024-01-11T00:56:21.704Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9862024-01-11T00:56:21.704Z+ [[ 16 -gt 60 ]]
9872024-01-11T00:56:21.704Z+ sleep 1
9882024-01-11T00:56:21.704Z+ retry=17
9892024-01-11T00:56:21.704Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9902024-01-11T00:56:21.704Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9912024-01-11T00:56:21.704Z+ [[ 17 -gt 60 ]]
9922024-01-11T00:56:21.704Z+ sleep 1
9932024-01-11T00:56:21.704Z+ retry=18
9942024-01-11T00:56:21.705Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9952024-01-11T00:56:21.705Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
9962024-01-11T00:56:21.705Z+ [[ 18 -gt 60 ]]
9972024-01-11T00:56:21.705Z+ sleep 1
9982024-01-11T00:56:22.667Z+ retry=19
9992024-01-11T00:56:22.667Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10002024-01-11T00:56:22.710Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10012024-01-11T00:56:22.711Z+ [[ 19 -gt 60 ]]
10022024-01-11T00:56:22.711Z+ sleep 1
10032024-01-11T00:56:23.686Z+ retry=20
10042024-01-11T00:56:23.687Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10052024-01-11T00:56:23.734Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10062024-01-11T00:56:23.734Z+ [[ 20 -gt 60 ]]
10072024-01-11T00:56:23.734Z+ sleep 1
10082024-01-11T00:56:24.701Z+ retry=21
10092024-01-11T00:56:24.701Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10102024-01-11T00:56:24.728Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10112024-01-11T00:56:24.729Z+ [[ 21 -gt 60 ]]
10122024-01-11T00:56:24.729Z+ sleep 1
10132024-01-11T00:56:25.715Z+ retry=22
10142024-01-11T00:56:25.715Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10152024-01-11T00:56:25.740Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10162024-01-11T00:56:25.741Z+ [[ 22 -gt 60 ]]
10172024-01-11T00:56:25.741Z+ sleep 1
10182024-01-11T00:56:26.732Z+ retry=23
10192024-01-11T00:56:26.733Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10202024-01-11T00:56:26.761Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10212024-01-11T00:56:26.761Z+ [[ 23 -gt 60 ]]
10222024-01-11T00:56:26.761Z+ sleep 1
10232024-01-11T00:56:27.747Z+ retry=24
10242024-01-11T00:56:27.747Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10252024-01-11T00:56:27.771Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10262024-01-11T00:56:27.772Z+ [[ 24 -gt 60 ]]
10272024-01-11T00:56:27.772Z+ sleep 1
10282024-01-11T00:56:28.768Z+ retry=25
10292024-01-11T00:56:28.768Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10302024-01-11T00:56:28.803Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10312024-01-11T00:56:28.803Z+ [[ 25 -gt 60 ]]
10322024-01-11T00:56:28.803Z+ sleep 1
10332024-01-11T00:56:29.781Z+ retry=26
10342024-01-11T00:56:29.781Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10352024-01-11T00:56:29.808Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10362024-01-11T00:56:29.808Z+ [[ 26 -gt 60 ]]
10372024-01-11T00:56:29.808Z+ sleep 1
10382024-01-11T00:56:30.797Z+ retry=27
10392024-01-11T00:56:30.797Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10402024-01-11T00:56:30.828Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10412024-01-11T00:56:30.828Z+ [[ 27 -gt 60 ]]
10422024-01-11T00:56:30.829Z+ sleep 1
10432024-01-11T00:56:31.812Z+ retry=28
10442024-01-11T00:56:31.813Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10452024-01-11T00:56:31.837Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10462024-01-11T00:56:31.838Z+ [[ 28 -gt 60 ]]
10472024-01-11T00:56:31.838Z+ sleep 1
10482024-01-11T00:56:32.835Z+ retry=29
10492024-01-11T00:56:32.835Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10502024-01-11T00:56:32.877Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10512024-01-11T00:56:32.877Z+ [[ 29 -gt 60 ]]
10522024-01-11T00:56:32.877Z+ sleep 1
10532024-01-11T00:56:33.872Z+ retry=30
10542024-01-11T00:56:33.872Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10552024-01-11T00:56:33.908Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10562024-01-11T00:56:33.909Z+ [[ 30 -gt 60 ]]
10572024-01-11T00:56:33.909Z+ sleep 1
10582024-01-11T00:56:34.884Z+ retry=31
10592024-01-11T00:56:34.884Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10602024-01-11T00:56:34.909Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10612024-01-11T00:56:34.909Z+ [[ 31 -gt 60 ]]
10622024-01-11T00:56:34.909Z+ sleep 1
10632024-01-11T00:56:35.899Z+ retry=32
10642024-01-11T00:56:35.900Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10652024-01-11T00:56:35.944Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10662024-01-11T00:56:35.945Z+ [[ 32 -gt 60 ]]
10672024-01-11T00:56:35.945Z+ sleep 1
10682024-01-11T00:56:36.891Z+ retry=33
10692024-01-11T00:56:36.891Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10702024-01-11T00:56:36.918Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10712024-01-11T00:56:36.918Z+ [[ 33 -gt 60 ]]
10722024-01-11T00:56:36.918Z+ sleep 1
10732024-01-11T00:56:37.905Z+ retry=34
10742024-01-11T00:56:37.906Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10752024-01-11T00:56:37.930Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10762024-01-11T00:56:37.930Z+ [[ 34 -gt 60 ]]
10772024-01-11T00:56:37.930Z+ sleep 1
10782024-01-11T00:56:38.922Z+ retry=35
10792024-01-11T00:56:38.923Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10802024-01-11T00:56:38.948Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10812024-01-11T00:56:38.948Z+ [[ 35 -gt 60 ]]
10822024-01-11T00:56:38.948Z+ sleep 1
10832024-01-11T00:56:39.937Z+ retry=36
10842024-01-11T00:56:39.937Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10852024-01-11T00:56:39.962Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10862024-01-11T00:56:39.962Z+ [[ 36 -gt 60 ]]
10872024-01-11T00:56:39.963Z+ sleep 1
10882024-01-11T00:56:40.953Z+ retry=37
10892024-01-11T00:56:40.954Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10902024-01-11T00:56:40.980Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10912024-01-11T00:56:40.984Z+ [[ 37 -gt 60 ]]
10922024-01-11T00:56:40.984Z+ sleep 1
10932024-01-11T00:56:41.968Z+ retry=38
10942024-01-11T00:56:41.968Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
10952024-01-11T00:56:41.993Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
10962024-01-11T00:56:41.993Z+ [[ 38 -gt 60 ]]
10972024-01-11T00:56:41.993Z+ sleep 1
10982024-01-11T00:56:42.984Z+ retry=39
10992024-01-11T00:56:42.985Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11002024-01-11T00:56:43.009Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11012024-01-11T00:56:43.009Z+ [[ 39 -gt 60 ]]
11022024-01-11T00:56:43.009Z+ sleep 1
11032024-01-11T00:56:44.002Z+ retry=40
11042024-01-11T00:56:44.002Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11052024-01-11T00:56:44.026Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11062024-01-11T00:56:44.027Z+ [[ 40 -gt 60 ]]
11072024-01-11T00:56:44.027Z+ sleep 1
11082024-01-11T00:56:45.017Z+ retry=41
11092024-01-11T00:56:45.017Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11102024-01-11T00:56:45.041Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11112024-01-11T00:56:45.042Z+ [[ 41 -gt 60 ]]
11122024-01-11T00:56:45.042Z+ sleep 1
11132024-01-11T00:56:46.033Z+ retry=42
11142024-01-11T00:56:46.033Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11152024-01-11T00:56:46.059Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11162024-01-11T00:56:46.060Z+ [[ 42 -gt 60 ]]
11172024-01-11T00:56:46.060Z+ sleep 1
11182024-01-11T00:56:47.050Z+ retry=43
11192024-01-11T00:56:47.050Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11202024-01-11T00:56:47.075Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
11212024-01-11T00:56:47.076Z+ [[ 43 -gt 60 ]]
11222024-01-11T00:56:47.076Z+ sleep 1
11232024-01-11T00:56:48.068Z+ retry=44
11242024-01-11T00:56:48.068Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
11252024-01-11T00:56:48.152Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images'
11262024-01-11T00:56:48.550Z{"status":"ok"}success
11272024-01-11T00:56:48.550ZProject {
11282024-01-11T00:56:48.550Z description: "some images",
11292024-01-11T00:56:48.550Z id: 71a61a2f-c63e-444b-9f48-907c05c10902,
11302024-01-11T00:56:48.550Z name: Name(
11312024-01-11T00:56:48.550Z "images",
11322024-01-11T00:56:48.550Z ),
11332024-01-11T00:56:48.550Z time_created: 2024-01-11T00:56:48.234171Z,
11342024-01-11T00:56:48.550Z time_modified: 2024-01-11T00:56:48.234171Z,
11352024-01-11T00:56:48.551Z}
11362024-01-11T00:56:48.575Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11
11372024-01-11T01:01:59.649Zdone!
11382024-01-11T01:01:59.678Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11
11392024-01-11T01:01:59.774Zsuccess
11402024-01-11T01:01:59.774ZImage {
11412024-01-11T01:01:59.775Z block_size: ByteCount(
11422024-01-11T01:01:59.813Z 512,
11432024-01-11T01:01:59.813Z ),
11442024-01-11T01:01:59.813Z description: "debian 11 original base image",
11452024-01-11T01:01:59.813Z digest: None,
11462024-01-11T01:01:59.813Z id: 7dd167ed-7d85-4af5-acd7-e51dbd4fc371,
11472024-01-11T01:01:59.813Z name: Name(
11482024-01-11T01:01:59.813Z "debian11",
11492024-01-11T01:01:59.813Z ),
11502024-01-11T01:01:59.813Z os: "debian",
11512024-01-11T01:01:59.813Z project_id: None,
11522024-01-11T01:01:59.813Z size: ByteCount(
11532024-01-11T01:01:59.813Z 2147483648,
11542024-01-11T01:01:59.814Z ),
11552024-01-11T01:01:59.814Z time_created: 2024-01-11T01:01:59.527875Z,
11562024-01-11T01:01:59.814Z time_modified: 2024-01-11T01:01:59.752303Z,
11572024-01-11T01:01:59.816Z url: None,
11582024-01-11T01:01:59.816Z version: "11",
11592024-01-11T01:01:59.816Z}
11602024-01-11T01:01:59.817Z+ rm ./tests/bootstrap
11612024-01-11T01:01:59.843Z+ for test_bin in tests/*
11622024-01-11T01:01:59.843Z+ ./tests/bootstrap-6bdde9b1284fa368
11632024-01-11T01:01:59.868Z
11642024-01-11T01:01:59.868Zrunning 0 tests
11652024-01-11T01:01:59.868Z
11662024-01-11T01:01:59.868Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
11672024-01-11T01:01:59.868Z
11682024-01-11T01:01:59.869Z+ for test_bin in tests/*
11692024-01-11T01:01:59.869Z+ ./tests/end_to_end_tests-826dc01617fce641
11702024-01-11T01:01:59.894Z
11712024-01-11T01:01:59.894Zrunning 2 tests
11722024-01-11T01:02:00.035Ztest no_spoof::no_spoof_here ... ok
11732024-01-11T01:02:32.458Ztest instance_launch::instance_launch ... ok
11742024-01-11T01:02:32.458Z
11752024-01-11T01:02:32.458Ztest result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 32.56s
11762024-01-11T01:02:32.458Z
11772024-01-11T01:02:32.484Z+ _exit_trap
11782024-01-11T01:02:32.484Z+ local status=0
11792024-01-11T01:02:32.484Z+ [[ 0 -eq 0 ]]
11802024-01-11T01:02:32.485Z+ exit 0
11812024-01-11T01:02:32.485Zprocess exited: duration 615625 ms, exit code 0
 
11822024-01-11T01:02:32.610Zfound 99 output files
11832024-01-11T01:02:32.610Zuploading: /var/svc/log/oxide-sled-agent:default.log (785389 bytes)
11842024-01-11T01:02:34.774Zuploaded: /var/svc/log/oxide-sled-agent:default.log
11852024-01-11T01:02:34.774Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_62c70745-9b1c-4f7a-9e16-ad803005659d/root/var/svc/log/oxide-crucible-agent:default.log (5930 bytes)
11862024-01-11T01:02:36.300Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_62c70745-9b1c-4f7a-9e16-ad803005659d/root/var/svc/log/oxide-crucible-agent:default.log
11872024-01-11T01:02:36.300Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_62c70745-9b1c-4f7a-9e16-ad803005659d/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
11882024-01-11T01:02:37.398Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_62c70745-9b1c-4f7a-9e16-ad803005659d/root/var/svc/log/oxide-crucible-agent:default.log.0
11892024-01-11T01:02:37.398Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_62c70745-9b1c-4f7a-9e16-ad803005659d/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11902024-01-11T01:02:38.473Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_62c70745-9b1c-4f7a-9e16-ad803005659d/root/var/svc/log/oxide-crucible-downstairs:default.log
11912024-01-11T01:02:38.499Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_62c70745-9b1c-4f7a-9e16-ad803005659d/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11922024-01-11T01:02:39.608Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_62c70745-9b1c-4f7a-9e16-ad803005659d/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11932024-01-11T01:02:39.608Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_62c70745-9b1c-4f7a-9e16-ad803005659d/root/var/svc/log/oxide-crucible-downstairs:downstairs-da423c14-6a0b-45b9-9eae-5b160196e8a1.log (2933 bytes)
11942024-01-11T01:02:40.705Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_62c70745-9b1c-4f7a-9e16-ad803005659d/root/var/svc/log/oxide-crucible-downstairs:downstairs-da423c14-6a0b-45b9-9eae-5b160196e8a1.log
11952024-01-11T01:02:40.750Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-agent:default.log (7757 bytes)
11962024-01-11T01:02:41.848Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-agent:default.log
11972024-01-11T01:02:41.848Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-agent:default.log.0 (19610 bytes)
11982024-01-11T01:02:42.989Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-agent:default.log.0
11992024-01-11T01:02:42.989Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
12002024-01-11T01:02:44.074Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:default.log
12012024-01-11T01:02:44.074Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
12022024-01-11T01:02:45.176Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:default.log.0
12032024-01-11T01:02:45.177Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:downstairs-1c41ad2e-2c34-45c8-8cfe-d19b7563abf7.log (0 bytes)
12042024-01-11T01:02:46.248Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:downstairs-1c41ad2e-2c34-45c8-8cfe-d19b7563abf7.log
12052024-01-11T01:02:46.248Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:downstairs-1c41ad2e-2c34-45c8-8cfe-d19b7563abf7.log.0 (3050 bytes)
12062024-01-11T01:02:47.347Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:downstairs-1c41ad2e-2c34-45c8-8cfe-d19b7563abf7.log.0
12072024-01-11T01:02:47.347Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:downstairs-e034286b-eb60-4729-86d0-736ddd646bcb.log (1092 bytes)
12082024-01-11T01:02:48.441Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:downstairs-e034286b-eb60-4729-86d0-736ddd646bcb.log
12092024-01-11T01:02:48.441Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:downstairs-e034286b-eb60-4729-86d0-736ddd646bcb.log.0 (4528 bytes)
12102024-01-11T01:02:49.546Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:downstairs-e034286b-eb60-4729-86d0-736ddd646bcb.log.0
12112024-01-11T01:02:49.546Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:snapshot-e034286b-eb60-4729-86d0-736ddd646bcb-3904e5c5-8a8d-4d9e-9f60-5b9f2fbf39cb.log (5158 bytes)
12122024-01-11T01:02:50.674Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b2fd8bb-b874-4526-9a11-42fcaa131d32/root/var/svc/log/oxide-crucible-downstairs:snapshot-e034286b-eb60-4729-86d0-736ddd646bcb-3904e5c5-8a8d-4d9e-9f60-5b9f2fbf39cb.log
12132024-01-11T01:02:50.674Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_e9bcc334-b9b2-4618-bf21-17df40372dd1/root/var/svc/log/oxide-crucible-agent:default.log (0 bytes)
12142024-01-11T01:02:51.749Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_e9bcc334-b9b2-4618-bf21-17df40372dd1/root/var/svc/log/oxide-crucible-agent:default.log
12152024-01-11T01:02:51.749Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_e9bcc334-b9b2-4618-bf21-17df40372dd1/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
12162024-01-11T01:02:52.858Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_e9bcc334-b9b2-4618-bf21-17df40372dd1/root/var/svc/log/oxide-crucible-agent:default.log.0
12172024-01-11T01:02:52.858Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_e9bcc334-b9b2-4618-bf21-17df40372dd1/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
12182024-01-11T01:02:53.937Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_e9bcc334-b9b2-4618-bf21-17df40372dd1/root/var/svc/log/oxide-crucible-downstairs:default.log
12192024-01-11T01:02:53.937Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_e9bcc334-b9b2-4618-bf21-17df40372dd1/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
12202024-01-11T01:02:55.109Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_e9bcc334-b9b2-4618-bf21-17df40372dd1/root/var/svc/log/oxide-crucible-downstairs:default.log.0
12212024-01-11T01:02:55.109Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_3ae41d66-cace-4ba2-ab89-97daf10b97b7/root/var/svc/log/oxide-clickhouse:default.log (278799 bytes)
12222024-01-11T01:02:55.690Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_3ae41d66-cace-4ba2-ab89-97daf10b97b7/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 278799 -> 322453
12232024-01-11T01:02:56.908Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_3ae41d66-cace-4ba2-ab89-97daf10b97b7/root/var/svc/log/oxide-clickhouse:default.log
12242024-01-11T01:02:56.908Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_3ae41d66-cace-4ba2-ab89-97daf10b97b7/root/var/svc/log/oxide-clickhouse:default.log.0 (567220 bytes)
12252024-01-11T01:02:58.723Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_3ae41d66-cace-4ba2-ab89-97daf10b97b7/root/var/svc/log/oxide-clickhouse:default.log.0
12262024-01-11T01:02:58.723Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_af78417a-a721-49ed-a763-214c44097285/root/var/svc/log/oxide-crucible-agent:default.log (10680 bytes)
12272024-01-11T01:02:59.859Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_af78417a-a721-49ed-a763-214c44097285/root/var/svc/log/oxide-crucible-agent:default.log
12282024-01-11T01:02:59.860Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_af78417a-a721-49ed-a763-214c44097285/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
12292024-01-11T01:03:01.107Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_af78417a-a721-49ed-a763-214c44097285/root/var/svc/log/oxide-crucible-agent:default.log.0
12302024-01-11T01:03:01.108Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_af78417a-a721-49ed-a763-214c44097285/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
12312024-01-11T01:03:02.315Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_af78417a-a721-49ed-a763-214c44097285/root/var/svc/log/oxide-crucible-downstairs:default.log
12322024-01-11T01:03:02.316Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_af78417a-a721-49ed-a763-214c44097285/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
12332024-01-11T01:03:03.680Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_af78417a-a721-49ed-a763-214c44097285/root/var/svc/log/oxide-crucible-downstairs:default.log.0
12342024-01-11T01:03:03.816Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_af78417a-a721-49ed-a763-214c44097285/root/var/svc/log/oxide-crucible-downstairs:downstairs-a48b3612-409b-4bba-aca7-4a43b3e4e246.log (5514 bytes)
12352024-01-11T01:03:04.909Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_af78417a-a721-49ed-a763-214c44097285/root/var/svc/log/oxide-crucible-downstairs:downstairs-a48b3612-409b-4bba-aca7-4a43b3e4e246.log
12362024-01-11T01:03:04.909Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_098215f9-08f2-4c7f-8daa-3fcf91e057aa/root/var/svc/log/oxide-crucible-pantry:default.log (426773 bytes)
12372024-01-11T01:03:05.195Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_098215f9-08f2-4c7f-8daa-3fcf91e057aa/root/var/svc/log/oxide-crucible-pantry:default.log" changed size mid upload: 426773 -> 442625
12382024-01-11T01:03:06.330Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_098215f9-08f2-4c7f-8daa-3fcf91e057aa/root/var/svc/log/oxide-crucible-pantry:default.log
12392024-01-11T01:03:06.330Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_098215f9-08f2-4c7f-8daa-3fcf91e057aa/root/var/svc/log/oxide-crucible-pantry:default.log.0 (795638 bytes)
12402024-01-11T01:03:08.347Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_098215f9-08f2-4c7f-8daa-3fcf91e057aa/root/var/svc/log/oxide-crucible-pantry:default.log.0
12412024-01-11T01:03:08.348Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_0f04d3e0-fb6f-4abe-af53-4879eb940ba9/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
12422024-01-11T01:03:09.489Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_0f04d3e0-fb6f-4abe-af53-4879eb940ba9/root/var/svc/log/oxide-cockroachdb:default.log
12432024-01-11T01:03:09.489Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_0f04d3e0-fb6f-4abe-af53-4879eb940ba9/root/var/svc/log/oxide-cockroachdb:default.log.0 (5461 bytes)
12442024-01-11T01:03:10.604Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_0f04d3e0-fb6f-4abe-af53-4879eb940ba9/root/var/svc/log/oxide-cockroachdb:default.log.0
12452024-01-11T01:03:10.604Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-agent:default.log (34335 bytes)
12462024-01-11T01:03:12.402Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-agent:default.log
12472024-01-11T01:03:12.402Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-agent:default.log.0 (9365 bytes)
12482024-01-11T01:03:13.710Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-agent:default.log.0
12492024-01-11T01:03:13.710Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
12502024-01-11T01:03:15.050Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:default.log
12512024-01-11T01:03:15.051Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
12522024-01-11T01:03:16.778Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:default.log.0
12532024-01-11T01:03:16.778Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:downstairs-ade455a2-80ad-4e67-a5cd-94e1a13604e5.log (1092 bytes)
12542024-01-11T01:03:18.364Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:downstairs-ade455a2-80ad-4e67-a5cd-94e1a13604e5.log
12552024-01-11T01:03:18.500Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:downstairs-ade455a2-80ad-4e67-a5cd-94e1a13604e5.log.0 (4528 bytes)
12562024-01-11T01:03:38.884Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:downstairs-ade455a2-80ad-4e67-a5cd-94e1a13604e5.log.0
12572024-01-11T01:03:38.885Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:downstairs-b799dd57-f706-4779-b0c9-256fbc6d6f08.log (2930 bytes)
12582024-01-11T01:03:40.072Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:downstairs-b799dd57-f706-4779-b0c9-256fbc6d6f08.log
12592024-01-11T01:03:40.072Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:downstairs-d161fa2e-d751-4c1c-9b9b-f07e812b1309.log (5518 bytes)
12602024-01-11T01:03:41.195Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:downstairs-d161fa2e-d751-4c1c-9b9b-f07e812b1309.log
12612024-01-11T01:03:41.195Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:snapshot-ade455a2-80ad-4e67-a5cd-94e1a13604e5-3904e5c5-8a8d-4d9e-9f60-5b9f2fbf39cb.log (5160 bytes)
12622024-01-11T01:03:42.285Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_30b3ed76-20e3-49c5-838c-fd67de7580c5/root/var/svc/log/oxide-crucible-downstairs:snapshot-ade455a2-80ad-4e67-a5cd-94e1a13604e5-3904e5c5-8a8d-4d9e-9f60-5b9f2fbf39cb.log
12632024-01-11T01:03:42.285Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_14ffded7-b4db-4aba-9862-d5fd8bea95bf/root/var/svc/log/oxide-internal_dns:default.log (8764 bytes)
12642024-01-11T01:03:42.404Zupload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_14ffded7-b4db-4aba-9862-d5fd8bea95bf/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 8764 -> 12520
12652024-01-11T01:03:43.481Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_14ffded7-b4db-4aba-9862-d5fd8bea95bf/root/var/svc/log/oxide-internal_dns:default.log
12662024-01-11T01:03:43.512Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_14ffded7-b4db-4aba-9862-d5fd8bea95bf/root/var/svc/log/oxide-internal_dns:default.log.0 (122405 bytes)
12672024-01-11T01:03:44.691Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_14ffded7-b4db-4aba-9862-d5fd8bea95bf/root/var/svc/log/oxide-internal_dns:default.log.0
12682024-01-11T01:03:44.692Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_9ea2101d-1ec2-46e1-96f7-9b596a8058fd/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
12692024-01-11T01:03:45.767Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_9ea2101d-1ec2-46e1-96f7-9b596a8058fd/root/var/svc/log/oxide-cockroachdb:default.log
12702024-01-11T01:03:45.767Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_9ea2101d-1ec2-46e1-96f7-9b596a8058fd/root/var/svc/log/oxide-cockroachdb:default.log.0 (5411 bytes)
12712024-01-11T01:03:46.862Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_9ea2101d-1ec2-46e1-96f7-9b596a8058fd/root/var/svc/log/oxide-cockroachdb:default.log.0
12722024-01-11T01:03:46.862Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_ea074c48-9863-4d28-bcf9-7b1725a29146/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
12732024-01-11T01:03:47.934Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_ea074c48-9863-4d28-bcf9-7b1725a29146/root/var/svc/log/oxide-cockroachdb:default.log
12742024-01-11T01:03:47.934Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_ea074c48-9863-4d28-bcf9-7b1725a29146/root/var/svc/log/oxide-cockroachdb:default.log.0 (5486 bytes)
12752024-01-11T01:03:49.054Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_ea074c48-9863-4d28-bcf9-7b1725a29146/root/var/svc/log/oxide-cockroachdb:default.log.0
12762024-01-11T01:03:49.054Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_67a08196-cb23-433a-b6f0-1f19ed39da24/root/var/svc/log/oxide-crucible-agent:default.log (0 bytes)
12772024-01-11T01:03:50.132Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_67a08196-cb23-433a-b6f0-1f19ed39da24/root/var/svc/log/oxide-crucible-agent:default.log
12782024-01-11T01:03:50.132Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_67a08196-cb23-433a-b6f0-1f19ed39da24/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
12792024-01-11T01:03:51.227Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_67a08196-cb23-433a-b6f0-1f19ed39da24/root/var/svc/log/oxide-crucible-agent:default.log.0
12802024-01-11T01:03:51.227Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_67a08196-cb23-433a-b6f0-1f19ed39da24/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
12812024-01-11T01:03:52.298Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_67a08196-cb23-433a-b6f0-1f19ed39da24/root/var/svc/log/oxide-crucible-downstairs:default.log
12822024-01-11T01:03:52.298Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_67a08196-cb23-433a-b6f0-1f19ed39da24/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
12832024-01-11T01:03:53.440Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_67a08196-cb23-433a-b6f0-1f19ed39da24/root/var/svc/log/oxide-crucible-downstairs:default.log.0
12842024-01-11T01:03:53.440Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_250f18c8-27e5-40dc-abe7-db9aec48e3bf/root/var/svc/log/oxide-nexus:default.log (323291 bytes)
12852024-01-11T01:03:53.670Zupload warning: file "/pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_250f18c8-27e5-40dc-abe7-db9aec48e3bf/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 323291 -> 400064
12862024-01-11T01:03:54.741Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_250f18c8-27e5-40dc-abe7-db9aec48e3bf/root/var/svc/log/oxide-nexus:default.log
12872024-01-11T01:03:54.742Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_250f18c8-27e5-40dc-abe7-db9aec48e3bf/root/var/svc/log/oxide-nexus:default.log.0 (2583577 bytes)
12882024-01-11T01:03:56.614Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_250f18c8-27e5-40dc-abe7-db9aec48e3bf/root/var/svc/log/oxide-nexus:default.log.0
12892024-01-11T01:03:56.614Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_85841234-5d63-46d3-b67d-91fff70cbcf4/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
12902024-01-11T01:03:57.713Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_85841234-5d63-46d3-b67d-91fff70cbcf4/root/var/svc/log/oxide-cockroachdb:default.log
12912024-01-11T01:03:57.713Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_85841234-5d63-46d3-b67d-91fff70cbcf4/root/var/svc/log/oxide-cockroachdb:default.log.0 (5486 bytes)
12922024-01-11T01:03:58.810Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_85841234-5d63-46d3-b67d-91fff70cbcf4/root/var/svc/log/oxide-cockroachdb:default.log.0
12932024-01-11T01:03:58.811Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_7801aea9-78b0-4fff-850a-881975927a1b/root/var/svc/log/oxide-crucible-agent:default.log (0 bytes)
12942024-01-11T01:03:59.882Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_7801aea9-78b0-4fff-850a-881975927a1b/root/var/svc/log/oxide-crucible-agent:default.log
12952024-01-11T01:03:59.882Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_7801aea9-78b0-4fff-850a-881975927a1b/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
12962024-01-11T01:04:00.976Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_7801aea9-78b0-4fff-850a-881975927a1b/root/var/svc/log/oxide-crucible-agent:default.log.0
12972024-01-11T01:04:00.976Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_7801aea9-78b0-4fff-850a-881975927a1b/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
12982024-01-11T01:04:02.468Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_7801aea9-78b0-4fff-850a-881975927a1b/root/var/svc/log/oxide-crucible-downstairs:default.log
12992024-01-11T01:04:02.468Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_7801aea9-78b0-4fff-850a-881975927a1b/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
13002024-01-11T01:04:03.564Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_7801aea9-78b0-4fff-850a-881975927a1b/root/var/svc/log/oxide-crucible-downstairs:default.log.0
13012024-01-11T01:04:03.565Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-agent:default.log (10246 bytes)
13022024-01-11T01:04:04.692Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-agent:default.log
13032024-01-11T01:04:04.692Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-agent:default.log.0 (13689 bytes)
13042024-01-11T01:04:05.818Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-agent:default.log.0
13052024-01-11T01:04:05.842Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
13062024-01-11T01:04:06.890Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-downstairs:default.log
13072024-01-11T01:04:06.891Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
13082024-01-11T01:04:07.986Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-downstairs:default.log.0
13092024-01-11T01:04:07.986Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-downstairs:downstairs-5b7679df-2322-40f7-8873-06afd4d6d0c4.log (0 bytes)
13102024-01-11T01:04:09.057Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-downstairs:downstairs-5b7679df-2322-40f7-8873-06afd4d6d0c4.log
13112024-01-11T01:04:09.057Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-downstairs:downstairs-5b7679df-2322-40f7-8873-06afd4d6d0c4.log.0 (3050 bytes)
13122024-01-11T01:04:10.162Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-downstairs:downstairs-5b7679df-2322-40f7-8873-06afd4d6d0c4.log.0
13132024-01-11T01:04:10.162Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-downstairs:downstairs-8e067953-95c7-4313-8c51-b23171d5cce3.log (5515 bytes)
13142024-01-11T01:04:11.265Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_f5f64775-c265-4b8b-aeaf-19fe96b1a1e0/root/var/svc/log/oxide-crucible-downstairs:downstairs-8e067953-95c7-4313-8c51-b23171d5cce3.log
13152024-01-11T01:04:11.265Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_83c2eb3c-0385-4b62-a74d-7a460df9f255/root/var/svc/log/oxide-crucible-pantry:default.log (0 bytes)
13162024-01-11T01:04:12.336Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_83c2eb3c-0385-4b62-a74d-7a460df9f255/root/var/svc/log/oxide-crucible-pantry:default.log
13172024-01-11T01:04:12.361Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_83c2eb3c-0385-4b62-a74d-7a460df9f255/root/var/svc/log/oxide-crucible-pantry:default.log.0 (2337 bytes)
13182024-01-11T01:04:13.454Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_83c2eb3c-0385-4b62-a74d-7a460df9f255/root/var/svc/log/oxide-crucible-pantry:default.log.0
13192024-01-11T01:04:13.454Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_nexus_674967f7-5706-4aa1-9447-54c2b322e390/root/var/svc/log/oxide-nexus:default.log (12759035 bytes)
13202024-01-11T01:04:21.568Zupload warning: file "/pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_nexus_674967f7-5706-4aa1-9447-54c2b322e390/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 12759035 -> 12902404
13212024-01-11T01:04:22.483Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_nexus_674967f7-5706-4aa1-9447-54c2b322e390/root/var/svc/log/oxide-nexus:default.log
13222024-01-11T01:04:22.483Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_nexus_674967f7-5706-4aa1-9447-54c2b322e390/root/var/svc/log/oxide-nexus:default.log.0 (21393279 bytes)
13232024-01-11T01:04:34.791Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_nexus_674967f7-5706-4aa1-9447-54c2b322e390/root/var/svc/log/oxide-nexus:default.log.0
13242024-01-11T01:04:34.791Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_external_dns_880fdd6d-7f4b-40db-870c-fa454ba5736f/root/var/svc/log/oxide-external_dns:default.log (8814 bytes)
13252024-01-11T01:04:35.059Zupload warning: file "/pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_external_dns_880fdd6d-7f4b-40db-870c-fa454ba5736f/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 8814 -> 16370
13262024-01-11T01:04:37.223Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_external_dns_880fdd6d-7f4b-40db-870c-fa454ba5736f/root/var/svc/log/oxide-external_dns:default.log
13272024-01-11T01:04:37.223Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_external_dns_880fdd6d-7f4b-40db-870c-fa454ba5736f/root/var/svc/log/oxide-external_dns:default.log.0 (19382 bytes)
13282024-01-11T01:04:37.809Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_external_dns_880fdd6d-7f4b-40db-870c-fa454ba5736f/root/var/svc/log/oxide-external_dns:default.log.0
13292024-01-11T01:04:37.809Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_oximeter_d73910c9-6d38-423e-81af-7c554381f9b2/root/var/svc/log/oxide-oximeter:default.log (94921 bytes)
13302024-01-11T01:04:57.407Zupload warning: file "/pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_oximeter_d73910c9-6d38-423e-81af-7c554381f9b2/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 94921 -> 181727
13312024-01-11T01:04:58.545Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_oximeter_d73910c9-6d38-423e-81af-7c554381f9b2/root/var/svc/log/oxide-oximeter:default.log
13322024-01-11T01:04:58.598Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_oximeter_d73910c9-6d38-423e-81af-7c554381f9b2/root/var/svc/log/oxide-oximeter:default.log.0 (161692 bytes)
13332024-01-11T01:04:59.832Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_oximeter_d73910c9-6d38-423e-81af-7c554381f9b2/root/var/svc/log/oxide-oximeter:default.log.0
13342024-01-11T01:04:59.832Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_cockroachdb_16c96cd2-7fb4-4292-aa08-48ddf97a6c6d/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
13352024-01-11T01:05:00.915Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_cockroachdb_16c96cd2-7fb4-4292-aa08-48ddf97a6c6d/root/var/svc/log/oxide-cockroachdb:default.log
13362024-01-11T01:05:00.915Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_cockroachdb_16c96cd2-7fb4-4292-aa08-48ddf97a6c6d/root/var/svc/log/oxide-cockroachdb:default.log.0 (5487 bytes)
13372024-01-11T01:05:02.025Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_cockroachdb_16c96cd2-7fb4-4292-aa08-48ddf97a6c6d/root/var/svc/log/oxide-cockroachdb:default.log.0
13382024-01-11T01:05:02.025Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_pantry_c07c89c4-aa2d-4855-b2d3-ef462abdb2f1/root/var/svc/log/oxide-crucible-pantry:default.log (0 bytes)
13392024-01-11T01:05:03.099Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_pantry_c07c89c4-aa2d-4855-b2d3-ef462abdb2f1/root/var/svc/log/oxide-crucible-pantry:default.log
13402024-01-11T01:05:03.099Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_pantry_c07c89c4-aa2d-4855-b2d3-ef462abdb2f1/root/var/svc/log/oxide-crucible-pantry:default.log.0 (2333 bytes)
13412024-01-11T01:05:04.603Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_pantry_c07c89c4-aa2d-4855-b2d3-ef462abdb2f1/root/var/svc/log/oxide-crucible-pantry:default.log.0
13422024-01-11T01:05:04.603Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_internal_dns_94bb5e85-8046-44ab-a65d-06255149cd9f/root/var/svc/log/oxide-internal_dns:default.log (8763 bytes)
13432024-01-11T01:05:04.695Zupload warning: file "/pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_internal_dns_94bb5e85-8046-44ab-a65d-06255149cd9f/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 8763 -> 18775
13442024-01-11T01:05:05.777Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_internal_dns_94bb5e85-8046-44ab-a65d-06255149cd9f/root/var/svc/log/oxide-internal_dns:default.log
13452024-01-11T01:05:05.777Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_internal_dns_94bb5e85-8046-44ab-a65d-06255149cd9f/root/var/svc/log/oxide-internal_dns:default.log.0 (117940 bytes)
13462024-01-11T01:05:07.023Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_internal_dns_94bb5e85-8046-44ab-a65d-06255149cd9f/root/var/svc/log/oxide-internal_dns:default.log.0
13472024-01-11T01:05:07.023Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_internal_dns_f812bc73-ca6d-40c9-bb4d-91a0cf0d87bc/root/var/svc/log/oxide-internal_dns:default.log (8761 bytes)
13482024-01-11T01:05:07.115Zupload warning: file "/pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_internal_dns_f812bc73-ca6d-40c9-bb4d-91a0cf0d87bc/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 8761 -> 18774
13492024-01-11T01:05:08.188Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_internal_dns_f812bc73-ca6d-40c9-bb4d-91a0cf0d87bc/root/var/svc/log/oxide-internal_dns:default.log
13502024-01-11T01:05:08.216Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_internal_dns_f812bc73-ca6d-40c9-bb4d-91a0cf0d87bc/root/var/svc/log/oxide-internal_dns:default.log.0 (119161 bytes)
13512024-01-11T01:05:09.424Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_internal_dns_f812bc73-ca6d-40c9-bb4d-91a0cf0d87bc/root/var/svc/log/oxide-internal_dns:default.log.0
13522024-01-11T01:05:09.425Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_7bd43ebd-0ac9-48eb-8c7d-5e4b0f4bf67b/root/var/svc/log/oxide-external_dns:default.log (8814 bytes)
13532024-01-11T01:05:09.515Zupload warning: file "/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_7bd43ebd-0ac9-48eb-8c7d-5e4b0f4bf67b/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 8814 -> 18892
13542024-01-11T01:05:10.586Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_7bd43ebd-0ac9-48eb-8c7d-5e4b0f4bf67b/root/var/svc/log/oxide-external_dns:default.log
13552024-01-11T01:05:10.586Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_7bd43ebd-0ac9-48eb-8c7d-5e4b0f4bf67b/root/var/svc/log/oxide-external_dns:default.log.0 (19382 bytes)
13562024-01-11T01:05:11.761Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_7bd43ebd-0ac9-48eb-8c7d-5e4b0f4bf67b/root/var/svc/log/oxide-external_dns:default.log.0
13572024-01-11T01:05:11.761Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_4ba26a20-90b7-4ecb-a2e2-2e8942866d69/root/var/svc/log/oxide-nexus:default.log (150789 bytes)
13582024-01-11T01:05:12.532Zupload warning: file "/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_4ba26a20-90b7-4ecb-a2e2-2e8942866d69/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 150789 -> 360438
13592024-01-11T01:05:13.176Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_4ba26a20-90b7-4ecb-a2e2-2e8942866d69/root/var/svc/log/oxide-nexus:default.log
13602024-01-11T01:05:13.202Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_4ba26a20-90b7-4ecb-a2e2-2e8942866d69/root/var/svc/log/oxide-nexus:default.log.0 (768583 bytes)
13612024-01-11T01:05:15.266Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_4ba26a20-90b7-4ecb-a2e2-2e8942866d69/root/var/svc/log/oxide-nexus:default.log.0
13622024-01-11T01:05:15.266Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/svc/log/oxide-ntp:default.log (0 bytes)
13632024-01-11T01:05:16.340Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/svc/log/oxide-ntp:default.log
13642024-01-11T01:05:16.340Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/svc/log/oxide-ntp:default.log.0 (983 bytes)
13652024-01-11T01:05:17.434Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/svc/log/oxide-ntp:default.log.0
13662024-01-11T01:05:17.434Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-agent:default.log (13682 bytes)
13672024-01-11T01:05:18.581Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-agent:default.log
13682024-01-11T01:05:18.581Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-agent:default.log.0 (20042 bytes)
13692024-01-11T01:05:19.745Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-agent:default.log.0
13702024-01-11T01:05:19.746Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
13712024-01-11T01:05:20.821Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:default.log
13722024-01-11T01:05:20.821Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
13732024-01-11T01:05:21.929Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:default.log.0
13742024-01-11T01:05:21.929Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:downstairs-24684a61-b76f-4f0e-9cab-7d7f7bb6c027.log (2932 bytes)
13752024-01-11T01:05:23.032Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:downstairs-24684a61-b76f-4f0e-9cab-7d7f7bb6c027.log
13762024-01-11T01:05:23.032Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:downstairs-a232fb49-4aa0-455d-a395-050aed5bedf7.log (1092 bytes)
13772024-01-11T01:05:24.130Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:downstairs-a232fb49-4aa0-455d-a395-050aed5bedf7.log
13782024-01-11T01:05:24.130Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:downstairs-a232fb49-4aa0-455d-a395-050aed5bedf7.log.0 (4530 bytes)
13792024-01-11T01:05:25.232Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:downstairs-a232fb49-4aa0-455d-a395-050aed5bedf7.log.0
13802024-01-11T01:05:25.238Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:downstairs-ba2e3452-0747-4b13-9037-16317d0b768c.log (0 bytes)
13812024-01-11T01:05:26.303Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:downstairs-ba2e3452-0747-4b13-9037-16317d0b768c.log
13822024-01-11T01:05:26.303Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:downstairs-ba2e3452-0747-4b13-9037-16317d0b768c.log.0 (3051 bytes)
13832024-01-11T01:05:27.400Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:downstairs-ba2e3452-0747-4b13-9037-16317d0b768c.log.0
13842024-01-11T01:05:27.400Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:snapshot-a232fb49-4aa0-455d-a395-050aed5bedf7-3904e5c5-8a8d-4d9e-9f60-5b9f2fbf39cb.log (5158 bytes)
13852024-01-11T01:05:28.501Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_93bc0d9f-6f5e-466e-8b22-5edd7ebce230/root/var/svc/log/oxide-crucible-downstairs:snapshot-a232fb49-4aa0-455d-a395-050aed5bedf7-3904e5c5-8a8d-4d9e-9f60-5b9f2fbf39cb.log
13862024-01-11T01:05:28.502Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/log/chrony/measurements.log (20276 bytes)
13872024-01-11T01:05:28.637Zupload warning: file "/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/log/chrony/measurements.log" changed size mid upload: 20276 -> 26578
13882024-01-11T01:05:29.710Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/log/chrony/measurements.log
13892024-01-11T01:05:29.711Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/log/chrony/statistics.log (11583 bytes)
13902024-01-11T01:05:29.803Zupload warning: file "/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/log/chrony/statistics.log" changed size mid upload: 11583 -> 16848
13912024-01-11T01:05:30.876Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/log/chrony/statistics.log
13922024-01-11T01:05:30.876Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/log/chrony/tracking.log (1984 bytes)
13932024-01-11T01:05:30.925Zupload warning: file "/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/log/chrony/tracking.log" changed size mid upload: 1984 -> 2248
13942024-01-11T01:05:32.000Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_d10dcfb2-ec4c-480c-8716-54f8af888363/root/var/log/chrony/tracking.log