01HWBB0PBAX453NJHCVF772CVR: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log*",
        "%/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.28",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01HWBB2D6EHN3MDANH43XB1V92

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12024-04-25T19:58:14.698Zjob dependencies complete; ready to run (waiting for 40 m 31 s)
22024-04-26T00:05:49.752Zjob assigned to worker 01HWBVEDFV4ECSZM6B130KDVM2 (queued for 4 h 7 m 35 s)
32024-04-26T00:05:49.847Zhttp://172.20.0.5:9969/ipxe/sock... ok
42024-04-26T00:05:49.894Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/kernel/amd64/unix... ok
52024-04-26T00:05:49.932Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/amd64/boot_archive... ok
62024-04-26T00:05:49.965Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/amd64/boot_archive.hash... ok
72024-04-26T00:05:50.018Zhttp://172.20.0.5:9969/postboot/sock... ok
82024-04-26T00:05:50.061ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
92024-04-26T00:05:50.099Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d1
102024-04-26T00:05:50.133ZOxide Helios Version helios-2.0.22550 64-bit
112024-04-26T00:05:50.167ZConfiguring devices.
122024-04-26T00:05:50.200Z##################################################
132024-04-26T00:05:50.234Z#### ############# ##
142024-04-26T00:05:50.273Z### ### ############ ##
152024-04-26T00:05:50.306Z## ### # ## ### ## Oxide Computer Company ##
162024-04-26T00:05:50.339Z## ## ## ### # ### ##
172024-04-26T00:05:50.373Z## # ### #### #### This Station Under ##
182024-04-26T00:05:50.406Z### ### #### # ### Computer Control ##
192024-04-26T00:05:50.440Z#### #### ### ## ##
202024-04-26T00:05:50.476Z##################################################
212024-04-26T00:05:50.513ZHostname: sock
222024-04-26T00:05:50.546ZLoading smf(7) service descriptions:
232024-04-26T00:05:51.133Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap-72919e54dd0a1e8f.gz
242024-04-26T00:05:52.984Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap-72919e54dd0a1e8f.gz
252024-04-26T00:05:52.984Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
262024-04-26T00:06:17.007Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
272024-04-26T00:06:17.007Zdownloading input: /input/ci-tools/work/end-to-end-tests/commtest-07d1782656389f6e.gz
282024-04-26T00:06:19.392Zdownloaded input: /input/ci-tools/work/end-to-end-tests/commtest-07d1782656389f6e.gz
292024-04-26T00:06:19.392Zdownloading input: /input/ci-tools/work/end-to-end-tests/dhcp_server-9e0ac39a731573d2.gz
302024-04-26T00:06:22.078Zdownloaded input: /input/ci-tools/work/end-to-end-tests/dhcp_server-9e0ac39a731573d2.gz
312024-04-26T00:06:22.103Zdownloading input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-1a59c060357c1b59.gz
322024-04-26T00:06:49.834Zdownloaded input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-1a59c060357c1b59.gz
332024-04-26T00:06:49.834Zdownloading input: /input/ci-tools/work/caboose-util.gz
342024-04-26T00:06:50.992Zdownloaded input: /input/ci-tools/work/caboose-util.gz
352024-04-26T00:06:51.054Zdownloading input: /input/ci-tools/work/tufaceous.gz
362024-04-26T00:06:55.479Zdownloaded input: /input/ci-tools/work/tufaceous.gz
372024-04-26T00:06:55.479Zdownloading input: /input/ci-tools/work/commtest
382024-04-26T00:09:22.632Zdownloaded input: /input/ci-tools/work/commtest
392024-04-26T00:09:22.657Zdownloading input: /input/ci-tools/work/permslip.gz
402024-04-26T00:09:25.397Zdownloaded input: /input/ci-tools/work/permslip.gz
412024-04-26T00:09:25.422Zdownloading input: /input/package/work/version.txt
422024-04-26T00:09:25.458Zdownloaded input: /input/package/work/version.txt
432024-04-26T00:09:25.458Zdownloading input: /input/package/work/package.tar.gz
442024-04-26T00:09:44.603Zdownloaded input: /input/package/work/package.tar.gz
452024-04-26T00:09:44.604Zdownloading input: /input/package/work/global-zone-packages.tar.gz
462024-04-26T00:10:13.757Zdownloaded input: /input/package/work/global-zone-packages.tar.gz
472024-04-26T00:10:13.757Zdownloading input: /input/package/work/trampoline-global-zone-packages.tar.gz
482024-04-26T00:10:22.393Zdownloaded input: /input/package/work/trampoline-global-zone-packages.tar.gz
492024-04-26T00:10:22.419Zdownloading input: /input/package/work/zones/clickhouse.tar.gz
502024-04-26T00:11:30.551Zdownloaded input: /input/package/work/zones/clickhouse.tar.gz
512024-04-26T00:11:30.551Zdownloading input: /input/package/work/zones/clickhouse_keeper.tar.gz
522024-04-26T00:12:17.281Zdownloaded input: /input/package/work/zones/clickhouse_keeper.tar.gz
532024-04-26T00:12:17.281Zdownloading input: /input/package/work/zones/cockroachdb.tar.gz
542024-04-26T00:12:45.187Zdownloaded input: /input/package/work/zones/cockroachdb.tar.gz
552024-04-26T00:12:45.187Zdownloading input: /input/package/work/zones/crucible-pantry-zone.tar.gz
562024-04-26T00:12:51.146Zdownloaded input: /input/package/work/zones/crucible-pantry-zone.tar.gz
572024-04-26T00:12:51.171Zdownloading input: /input/package/work/zones/crucible-zone.tar.gz
582024-04-26T00:12:59.195Zdownloaded input: /input/package/work/zones/crucible-zone.tar.gz
592024-04-26T00:12:59.195Zdownloading input: /input/package/work/zones/external-dns.tar.gz
602024-04-26T00:13:05.047Zdownloaded input: /input/package/work/zones/external-dns.tar.gz
612024-04-26T00:13:05.072Zdownloading input: /input/package/work/zones/internal-dns.tar.gz
622024-04-26T00:13:09.955Zdownloaded input: /input/package/work/zones/internal-dns.tar.gz
632024-04-26T00:13:09.955Zdownloading input: /input/package/work/zones/nexus-single-sled.tar.gz
642024-04-26T00:13:25.399Zdownloaded input: /input/package/work/zones/nexus-single-sled.tar.gz
652024-04-26T00:13:25.399Zdownloading input: /input/package/work/zones/nexus.tar.gz
662024-04-26T00:13:41.420Zdownloaded input: /input/package/work/zones/nexus.tar.gz
672024-04-26T00:13:41.420Zdownloading input: /input/package/work/zones/ntp.tar.gz
682024-04-26T00:13:43.582Zdownloaded input: /input/package/work/zones/ntp.tar.gz
692024-04-26T00:13:43.582Zdownloading input: /input/package/work/zones/omicron-gateway-asic.tar.gz
702024-04-26T00:13:47.312Zdownloaded input: /input/package/work/zones/omicron-gateway-asic.tar.gz
712024-04-26T00:13:47.337Zdownloading input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
722024-04-26T00:13:50.426Zdownloaded input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
732024-04-26T00:13:50.426Zdownloading input: /input/package/work/zones/overlay.tar.gz
742024-04-26T00:13:50.508Zdownloaded input: /input/package/work/zones/overlay.tar.gz
752024-04-26T00:13:50.508Zdownloading input: /input/package/work/zones/oximeter.tar.gz
762024-04-26T00:13:59.390Zdownloaded input: /input/package/work/zones/oximeter.tar.gz
772024-04-26T00:13:59.390Zdownloading input: /input/package/work/zones/probe.tar.gz
782024-04-26T00:14:00.031Zdownloaded input: /input/package/work/zones/probe.tar.gz
792024-04-26T00:14:00.031Zdownloading input: /input/package/work/zones/propolis-server.tar.gz
802024-04-26T00:14:13.650Zdownloaded input: /input/package/work/zones/propolis-server.tar.gz
812024-04-26T00:14:13.650Zdownloading input: /input/package/work/zones/switch-asic.tar.gz
822024-04-26T00:15:20.859Zdownloaded input: /input/package/work/zones/switch-asic.tar.gz
832024-04-26T00:15:20.859Zdownloading input: /input/package/work/zones/switch-softnpu.tar.gz
842024-04-26T00:16:01.732Zdownloaded input: /input/package/work/zones/switch-softnpu.tar.gz
 
852024-04-26T00:16:01.757Zstarting task 0: "setup"
862024-04-26T00:16:01.757Z++ uname -s
872024-04-26T00:16:01.757Z+ kern=SunOS
882024-04-26T00:16:01.757Z+ case "$kern" in
892024-04-26T00:16:01.757Z+ groupadd -g 12345 build
902024-04-26T00:16:01.757Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
912024-04-26T00:16:03.751Z+ zfs create -o mountpoint=/work rpool/work
922024-04-26T00:16:03.787Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
932024-04-26T00:16:03.813Z+ home_fs=zfs
942024-04-26T00:16:03.813Z+ [[ zfs == autofs ]]
952024-04-26T00:16:03.813Z+ mkdir -p /home/build
962024-04-26T00:16:03.813Z+ chown build:build /home/build /work
972024-04-26T00:16:05.795Z+ chmod 0700 /home/build /work
982024-04-26T00:16:05.819Zprocess exited: duration 4066 ms, exit code 0
 
992024-04-26T00:16:05.879Zstarting task 1: "authentication"
1002024-04-26T00:16:05.924Zprocess exited: duration 45 ms, exit code 0
 
1012024-04-26T00:16:05.974Zstarting task 2: "build"
1022024-04-26T00:16:05.999Z+ trap _exit_trap EXIT
1032024-04-26T00:16:05.999Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
1042024-04-26T00:16:06.114Zsecpolicy_fs_linkdir: 0x55 = 0x55
1052024-04-26T00:16:06.114Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
1062024-04-26T00:16:06.114Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
1072024-04-26T00:16:06.114Zsecpolicy_fs_linkdir+3: 0 = 0xe5
1082024-04-26T00:16:06.114Zsecpolicy_fs_linkdir+4: 0 = 0xb8
1092024-04-26T00:16:06.114Zsecpolicy_fs_linkdir+5: 0 = 0x1
1102024-04-26T00:16:06.114Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
1112024-04-26T00:16:06.115Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
1122024-04-26T00:16:06.115Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
1132024-04-26T00:16:06.115Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
1142024-04-26T00:16:06.115Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
1152024-04-26T00:16:06.115Z+ [[ -d /opt/oxide ]]
1162024-04-26T00:16:06.115Z+ pfexec mv /opt/oxide /opt/oxide-underneath
1172024-04-26T00:16:06.115Z+ pfexec mkdir /opt/oxide
1182024-04-26T00:16:06.115Z+ pfexec mount -F tmpfs -O swap /opt/oxide
1192024-04-26T00:16:06.115Z+ [[ -d /opt/oxide-underneath ]]
1202024-04-26T00:16:06.115Z+ cd /opt/oxide-underneath
1212024-04-26T00:16:06.115Z+ pfexec tar ceEp@/f - .
1222024-04-26T00:16:06.115Z+ cd /opt/oxide
1232024-04-26T00:16:06.115Z+ pfexec tar xveEp@/f -
1242024-04-26T00:16:06.203Zx ., 0 bytes, 0 tape blocks
1252024-04-26T00:16:06.203Zx ./opte, 0 bytes, 0 tape blocks
1262024-04-26T00:16:06.203Zx ./opte/bin, 0 bytes, 0 tape blocks
1272024-04-26T00:16:06.203Zx ./opte/bin/opteadm, 43221608 bytes, 84418 tape blocks
1282024-04-26T00:16:06.228Z+ pfexec rm -rf /opt/oxide-underneath
1292024-04-26T00:16:06.228Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
1302024-04-26T00:16:06.252Z+ pfexec mkdir /opt/oxide/work
1312024-04-26T00:16:06.252Z+ pfexec chown build:build /opt/oxide/work
1322024-04-26T00:16:06.252Z+ cd /opt/oxide/work
1332024-04-26T00:16:06.252Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
1342024-04-26T00:16:07.979ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
1352024-04-26T00:16:07.979Zx out/mg-ddm-gz.tar, 75678720 bytes, 147810 tape blocks
1362024-04-26T00:16:07.979Zx out/omicron-sled-agent.tar, 97925632 bytes, 191261 tape blocks
1372024-04-26T00:16:07.979Zx out/oxlog.tar, 2982912 bytes, 5826 tape blocks
1382024-04-26T00:16:07.979Zx out/pumpkind-gz.tar, 27181056 bytes, 53088 tape blocks
1392024-04-26T00:16:07.979Zx out/target/test, 75 bytes, 1 tape blocks
1402024-04-26T00:16:07.979Zx out/npuzone/npuzone, 37184104 bytes, 72626 tape blocks
1412024-04-26T00:16:07.979Zx package-manifest.toml, 27341 bytes, 54 tape blocks
1422024-04-26T00:16:07.980Zx smf/sled-agent/non-gimlet/config.toml, 3154 bytes, 7 tape blocks
1432024-04-26T00:16:07.980Zx target/release/omicron-package, 31127544 bytes, 60796 tape blocks
1442024-04-26T00:16:07.980Zx target/release/xtask, 6821896 bytes, 13325 tape blocks
1452024-04-26T00:16:08.011Z
1462024-04-26T00:16:08.011Zreal 1.729652379
1472024-04-26T00:16:08.011Zuser 1.555127788
1482024-04-26T00:16:08.011Zsys 0.304612626
1492024-04-26T00:16:08.011Ztrap 0.000079870
1502024-04-26T00:16:08.012Ztflt 0.000000000
1512024-04-26T00:16:08.012Zdflt 0.000000000
1522024-04-26T00:16:08.012Zkflt 0.000000000
1532024-04-26T00:16:08.012Zlock 0.000000000
1542024-04-26T00:16:08.012Zslp 1.556251582
1552024-04-26T00:16:08.012Zlat 0.040994092
1562024-04-26T00:16:08.012Zstop 0.000126010
1572024-04-26T00:16:08.012Z+ cp /input/package/work/zones/clickhouse_keeper.tar.gz /input/package/work/zones/clickhouse.tar.gz /input/package/work/zones/cockroachdb.tar.gz /input/package/work/zones/crucible-pantry-zone.tar.gz /input/package/work/zones/crucible-zone.tar.gz /input/package/work/zones/external-dns.tar.gz /input/package/work/zones/internal-dns.tar.gz /input/package/work/zones/nexus-single-sled.tar.gz /input/package/work/zones/nexus.tar.gz /input/package/work/zones/ntp.tar.gz /input/package/work/zones/omicron-gateway-asic.tar.gz /input/package/work/zones/omicron-gateway-softnpu.tar.gz /input/package/work/zones/overlay.tar.gz /input/package/work/zones/oximeter.tar.gz /input/package/work/zones/probe.tar.gz /input/package/work/zones/propolis-server.tar.gz /input/package/work/zones/switch-asic.tar.gz /input/package/work/zones/switch-softnpu.tar.gz out/
1582024-04-26T00:16:09.203Z+ mv out/nexus-single-sled.tar.gz out/nexus.tar.gz
1592024-04-26T00:16:09.229Z+ mkdir tests
1602024-04-26T00:16:09.229Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1612024-04-26T00:16:09.229Z+ ptime -m gunzip
1622024-04-26T00:16:09.229Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-72919e54dd0a1e8f
1632024-04-26T00:16:09.429Z
1642024-04-26T00:16:09.456Zreal 0.204796010
1652024-04-26T00:16:09.456Zuser 0.189573125
1662024-04-26T00:16:09.456Zsys 0.014964650
1672024-04-26T00:16:09.456Ztrap 0.000018310
1682024-04-26T00:16:09.456Ztflt 0.000000000
1692024-04-26T00:16:09.456Zdflt 0.000000000
1702024-04-26T00:16:09.456Zkflt 0.000000000
1712024-04-26T00:16:09.456Zlock 0.000000000
1722024-04-26T00:16:09.456Zslp 0.000018660
1732024-04-26T00:16:09.457Zlat 0.000030840
1742024-04-26T00:16:09.457Zstop 0.000127252
1752024-04-26T00:16:09.457Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-72919e54dd0a1e8f
1762024-04-26T00:16:09.457Z+ chmod a+x tests/bootstrap-72919e54dd0a1e8f
1772024-04-26T00:16:09.457Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1782024-04-26T00:16:09.457Z+ ptime -m gunzip
1792024-04-26T00:16:09.457Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1802024-04-26T00:16:11.741Z
1812024-04-26T00:16:11.765Zreal 2.303780462
1822024-04-26T00:16:11.765Zuser 2.168024519
1832024-04-26T00:16:11.766Zsys 0.135346890
1842024-04-26T00:16:11.766Ztrap 0.000173230
1852024-04-26T00:16:11.766Ztflt 0.000000000
1862024-04-26T00:16:11.766Zdflt 0.000000000
1872024-04-26T00:16:11.766Zkflt 0.000000000
1882024-04-26T00:16:11.766Zlock 0.000000000
1892024-04-26T00:16:11.766Zslp 0.000025450
1902024-04-26T00:16:11.766Zlat 0.000023200
1912024-04-26T00:16:11.766Zstop 0.000126321
1922024-04-26T00:16:11.766Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1932024-04-26T00:16:11.766Z+ chmod a+x tests/bootstrap
1942024-04-26T00:16:11.766Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1952024-04-26T00:16:11.766Z+ ptime -m gunzip
1962024-04-26T00:16:11.766Z++ basename /input/ci-tools/work/end-to-end-tests/commtest-07d1782656389f6e
1972024-04-26T00:16:11.955Z
1982024-04-26T00:16:11.980Zreal 0.203508230
1992024-04-26T00:16:11.980Zuser 0.189655187
2002024-04-26T00:16:11.981Zsys 0.013592348
2012024-04-26T00:16:11.981Ztrap 0.000022800
2022024-04-26T00:16:11.981Ztflt 0.000000000
2032024-04-26T00:16:11.981Zdflt 0.000000000
2042024-04-26T00:16:11.981Zkflt 0.000000000
2052024-04-26T00:16:11.981Zlock 0.000000000
2062024-04-26T00:16:11.981Zslp 0.000022310
2072024-04-26T00:16:11.981Zlat 0.000029500
2082024-04-26T00:16:11.981Zstop 0.000129151
2092024-04-26T00:16:11.981Z++ basename /input/ci-tools/work/end-to-end-tests/commtest-07d1782656389f6e
2102024-04-26T00:16:11.981Z+ chmod a+x tests/commtest-07d1782656389f6e
2112024-04-26T00:16:11.981Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
2122024-04-26T00:16:11.981Z+ ptime -m gunzip
2132024-04-26T00:16:11.981Z++ basename /input/ci-tools/work/end-to-end-tests/dhcp_server-9e0ac39a731573d2
2142024-04-26T00:16:12.169Z
2152024-04-26T00:16:12.194Zreal 0.202858169
2162024-04-26T00:16:12.194Zuser 0.189311312
2172024-04-26T00:16:12.194Zsys 0.013304622
2182024-04-26T00:16:12.194Ztrap 0.000024740
2192024-04-26T00:16:12.194Ztflt 0.000000000
2202024-04-26T00:16:12.194Zdflt 0.000000000
2212024-04-26T00:16:12.194Zkflt 0.000000000
2222024-04-26T00:16:12.194Zlock 0.000000000
2232024-04-26T00:16:12.194Zslp 0.000013910
2242024-04-26T00:16:12.194Zlat 0.000025580
2252024-04-26T00:16:12.194Zstop 0.000113611
2262024-04-26T00:16:12.194Z++ basename /input/ci-tools/work/end-to-end-tests/dhcp_server-9e0ac39a731573d2
2272024-04-26T00:16:12.194Z+ chmod a+x tests/dhcp_server-9e0ac39a731573d2
2282024-04-26T00:16:12.194Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
2292024-04-26T00:16:12.194Z+ ptime -m gunzip
2302024-04-26T00:16:12.194Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-1a59c060357c1b59
2312024-04-26T00:16:14.946Z
2322024-04-26T00:16:15.005Zreal 2.327867629
2332024-04-26T00:16:15.005Zuser 2.195668239
2342024-04-26T00:16:15.005Zsys 0.131786637
2352024-04-26T00:16:15.005Ztrap 0.000178640
2362024-04-26T00:16:15.005Ztflt 0.000000000
2372024-04-26T00:16:15.005Zdflt 0.000000000
2382024-04-26T00:16:15.005Zkflt 0.000000000
2392024-04-26T00:16:15.006Zlock 0.000000000
2402024-04-26T00:16:15.006Zslp 0.000023220
2412024-04-26T00:16:15.006Zlat 0.000023610
2422024-04-26T00:16:15.006Zstop 0.000125671
2432024-04-26T00:16:15.006Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-1a59c060357c1b59
2442024-04-26T00:16:15.006Z+ chmod a+x tests/end_to_end_tests-1a59c060357c1b59
2452024-04-26T00:16:15.006Z+ bmat address ls
2462024-04-26T00:16:15.006ZNAME CIDR FIRST LAST COUNT
2472024-04-26T00:16:15.006Zextra 10.151.2.0/24 10.151.2.164 10.151.2.227 64
2482024-04-26T00:16:15.006Z++ bmat address ls -f extra -Ho first,last,count
2492024-04-26T00:16:15.007Z+ set -- 10.151.2.164 10.151.2.227 64
2502024-04-26T00:16:15.007Z+ EXTRA_IP_START=10.151.2.164
2512024-04-26T00:16:15.007Z+ EXTRA_IP_END=10.151.2.227
2522024-04-26T00:16:15.007Z+ EXTRA_IP_COUNT=64
2532024-04-26T00:16:15.007Z+ (( EXTRA_IP_COUNT >= 32 ))
2542024-04-26T00:16:15.007Z+ EXTRA_IP_BASE=10.151.2
2552024-04-26T00:16:15.007Z+ EXTRA_IP_FOCTET=164
2562024-04-26T00:16:15.007Z+ EXTRA_IP_LOCTET=227
2572024-04-26T00:16:15.007Z+ SERVICE_IP_POOL_START=10.151.2.164
2582024-04-26T00:16:15.007Z+ SERVICE_IP_POOL_END=10.151.2.173
2592024-04-26T00:16:15.007Z+ DNS_IP1=10.151.2.164
2602024-04-26T00:16:15.008Z+ DNS_IP2=10.151.2.165
2612024-04-26T00:16:15.008Z+ UPLINK_IP=10.151.2.174
2622024-04-26T00:16:15.008Z+ IPPOOL_START=10.151.2.175
2632024-04-26T00:16:15.008Z+ IPPOOL_END=10.151.2.227
2642024-04-26T00:16:15.008Z++ ipadm show-addr -po type,addr
2652024-04-26T00:16:15.008Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
2662024-04-26T00:16:15.008Z+ GATEWAY_IP=10.151.2.100
2672024-04-26T00:16:15.008Z+ [[ -n 10.151.2.100 ]]
2682024-04-26T00:16:15.008Z+ ping -s 10.151.2.100 56 1
2692024-04-26T00:16:15.008ZPING 10.151.2.100: 56 data bytes
2702024-04-26T00:16:15.009Z64 bytes from 10.151.2.100: icmp_seq=0. time=0.011 ms
2712024-04-26T00:16:15.009Z
2722024-04-26T00:16:15.009Z----10.151.2.100 PING Statistics----
2732024-04-26T00:16:15.009Z1 packets transmitted, 1 packets received, 0% packet loss
2742024-04-26T00:16:15.009Zround-trip (ms) min/avg/max/stddev = 0.011/0.011/0.011/-nan
2752024-04-26T00:16:15.009Z++ arp -an
2762024-04-26T00:16:15.009Z++ awk -vgw=10.151.2.100 '$2 == gw {print $NF}'
2772024-04-26T00:16:15.009Z+ GATEWAY_MAC=18:c0:4d:0d:a0:2a
2782024-04-26T00:16:15.010Z+ routeadm -e ipv4-forwarding -u
2792024-04-26T00:16:15.010Z+ PXA_START=10.151.2.164
2802024-04-26T00:16:15.010Z+ PXA_END=10.151.2.227
2812024-04-26T00:16:15.010Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
2822024-04-26T00:16:15.010Z+ ptime -m pfexec ./target/release/xtask virtual-hardware --vdev-dir /scratch create --gateway-ip 10.151.2.100 --gateway-mac 18:c0:4d:0d:a0:2a --pxa-start 10.151.2.164 --pxa-end 10.151.2.227
2832024-04-26T00:16:15.010ZCouldn't find Cargo.toml, using /opt/oxide/work as workspace root
2842024-04-26T00:16:15.011Zcreating virtual hardware
2852024-04-26T00:16:15.011Zcreating /scratch/m2_0.vdev
2862024-04-26T00:16:15.011Zcreating /scratch/m2_1.vdev
2872024-04-26T00:16:15.011Zcreating /scratch/u2_0.vdev
2882024-04-26T00:16:15.011Zcreating /scratch/u2_1.vdev
2892024-04-26T00:16:15.011Zcreating /scratch/u2_2.vdev
2902024-04-26T00:16:15.011Zcreating /scratch/u2_3.vdev
2912024-04-26T00:16:15.011Zcreating /scratch/u2_4.vdev
2922024-04-26T00:16:15.012Zcreating /scratch/u2_5.vdev
2932024-04-26T00:16:15.012Zcreating /scratch/u2_6.vdev
2942024-04-26T00:16:15.012Zcreating /scratch/u2_7.vdev
2952024-04-26T00:16:15.012Zcreating /scratch/u2_8.vdev
2962024-04-26T00:16:15.012ZSimnet net0/sc0_0 exists
2972024-04-26T00:16:15.012ZSimnet net1/sc1_0 exists
2982024-04-26T00:16:15.013ZVnic sc0_1 exists
2992024-04-26T00:16:49.443ZUsing 10.151.2.100 as gateway ip
3002024-04-26T00:16:49.444Zusing 18:C0:4D:0D:A0:2A as gateway mac
3012024-04-26T00:16:49.444Zconfiguring SoftNPU ARP entry
3022024-04-26T00:16:49.471Zconfiguring SoftNPU proxy ARP
3032024-04-26T00:16:49.528ZSoftNPU state:
3042024-04-26T00:16:49.528Z local v6:
3052024-04-26T00:16:49.528Z local v4:
3062024-04-26T00:16:49.528Z router v6:
3072024-04-26T00:16:49.528Z router v4_idx:
3082024-04-26T00:16:49.528Z router v4_routes:
3092024-04-26T00:16:49.528Z resolver v4:
3102024-04-26T00:16:49.528Z 10.151.2.100 -> 18:c0:4d:0d:a0:2a
3112024-04-26T00:16:49.528Z resolver v6:
3122024-04-26T00:16:49.528Z nat v4:
3132024-04-26T00:16:49.528Z nat_v6:
3142024-04-26T00:16:49.528Z port_mac:
3152024-04-26T00:16:49.528Z proxy arp:
3162024-04-26T00:16:49.528Z 10.151.2.164/10.151.2.227: a8:e1:de:01:70:1d
3172024-04-26T00:16:49.529Zcreated virtual hardware
3182024-04-26T00:16:49.559Z
3192024-04-26T00:16:49.559Zreal 34.715437948
3202024-04-26T00:16:49.559Zuser 0.594206120
3212024-04-26T00:16:49.559Zsys 1.350827386
3222024-04-26T00:16:49.559Ztrap 0.000092598
3232024-04-26T00:16:49.559Ztflt 0.000118203
3242024-04-26T00:16:49.559Zdflt 0.001418857
3252024-04-26T00:16:49.559Zkflt 0.000001620
3262024-04-26T00:16:49.559Zlock 27:32.764100400
3272024-04-26T00:16:49.559Zslp 1:44.538877220
3282024-04-26T00:16:49.559Zlat 0.047990591
3292024-04-26T00:16:49.559Zstop 0.345356642
3302024-04-26T00:16:49.559Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml pkg/config.toml
3312024-04-26T00:16:49.559Z+ sed -E -i~ 's/(m2|u2)(.*\.vdev)/\/scratch\/\1\2/g' pkg/config.toml
3322024-04-26T00:16:49.607Z+ diff -u pkg/config.toml~ pkg/config.toml
3332024-04-26T00:16:49.607Z--- pkg/config.toml~ Mon Jul 24 01:21:28 2006
3342024-04-26T00:16:49.607Z+++ pkg/config.toml Fri Apr 26 00:16:49 2024
3352024-04-26T00:16:49.607Z@@ -29,18 +29,18 @@
3362024-04-26T00:16:49.608Z # These paths have the prefix of either "u2" or "m2", followed by an underscore,
3372024-04-26T00:16:49.608Z # followed by a string that is embedded into their fake serial values.
3382024-04-26T00:16:49.608Z vdevs = [
3392024-04-26T00:16:49.608Z- "m2_0.vdev",
3402024-04-26T00:16:49.608Z- "m2_1.vdev",
3412024-04-26T00:16:49.608Z+ "/scratch/m2_0.vdev",
3422024-04-26T00:16:49.608Z+ "/scratch/m2_1.vdev",
3432024-04-26T00:16:49.608Z
3442024-04-26T00:16:49.608Z- "u2_0.vdev",
3452024-04-26T00:16:49.608Z- "u2_1.vdev",
3462024-04-26T00:16:49.608Z- "u2_2.vdev",
3472024-04-26T00:16:49.608Z- "u2_3.vdev",
3482024-04-26T00:16:49.608Z- "u2_4.vdev",
3492024-04-26T00:16:49.608Z- "u2_5.vdev",
3502024-04-26T00:16:49.609Z- "u2_6.vdev",
3512024-04-26T00:16:49.609Z- "u2_7.vdev",
3522024-04-26T00:16:49.609Z- "u2_8.vdev",
3532024-04-26T00:16:49.609Z+ "/scratch/u2_0.vdev",
3542024-04-26T00:16:49.609Z+ "/scratch/u2_1.vdev",
3552024-04-26T00:16:49.609Z+ "/scratch/u2_2.vdev",
3562024-04-26T00:16:49.609Z+ "/scratch/u2_3.vdev",
3572024-04-26T00:16:49.609Z+ "/scratch/u2_4.vdev",
3582024-04-26T00:16:49.609Z+ "/scratch/u2_5.vdev",
3592024-04-26T00:16:49.609Z+ "/scratch/u2_6.vdev",
3602024-04-26T00:16:49.609Z+ "/scratch/u2_7.vdev",
3612024-04-26T00:16:49.609Z+ "/scratch/u2_8.vdev",
3622024-04-26T00:16:49.609Z ]
3632024-04-26T00:16:49.609Z
3642024-04-26T00:16:49.610Z # Percentage of usable physical DRAM to use for the VMM reservoir, which
3652024-04-26T00:16:49.610Z+ true
3662024-04-26T00:16:49.610Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
3672024-04-26T00:16:49.610Z+ SILO_NAME=recovery
3682024-04-26T00:16:49.610Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
3692024-04-26T00:16:49.610Z+ EXTERNAL_DNS_DOMAIN=oxide.test
3702024-04-26T00:16:49.610Z+ sed -i~ '
3712024-04-26T00:16:49.610Z /^external_dns_ips/c\
3722024-04-26T00:16:49.610Zexternal_dns_ips = [ "10.151.2.164", "10.151.2.165" ]
3732024-04-26T00:16:49.610Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
3742024-04-26T00:16:49.610Z /^first/c\
3752024-04-26T00:16:49.610Zfirst = "10.151.2.164"
3762024-04-26T00:16:49.610Z /^last/c\
3772024-04-26T00:16:49.611Zlast = "10.151.2.173"
3782024-04-26T00:16:49.611Z }
3792024-04-26T00:16:49.611Z /^infra_ip_first/c\
3802024-04-26T00:16:49.611Zinfra_ip_first = "10.151.2.174"
3812024-04-26T00:16:49.611Z /^infra_ip_last/c\
3822024-04-26T00:16:49.611Zinfra_ip_last = "10.151.2.174"
3832024-04-26T00:16:49.611Z /^\[\[rack_network_config.ports/,/^$/ {
3842024-04-26T00:16:49.611Z /^routes/c\
3852024-04-26T00:16:49.611Zroutes = \[{nexthop = "10.151.2.100", destination = "0.0.0.0/0"}\]
3862024-04-26T00:16:49.611Z /^addresses/c\
3872024-04-26T00:16:49.611Zaddresses = \["10.151.2.174/24"\]
3882024-04-26T00:16:49.611Z }
3892024-04-26T00:16:49.611Z' pkg/config-rss.toml
3902024-04-26T00:16:49.611Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
3912024-04-26T00:16:49.611Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
3922024-04-26T00:16:49.612Z+++ pkg/config-rss.toml Fri Apr 26 00:16:49 2024
3932024-04-26T00:16:49.612Z@@ -33,7 +33,7 @@
3942024-04-26T00:16:49.612Z # the DNS domain delegated to the rack by the customer. Each of these addresses
3952024-04-26T00:16:49.612Z # must be contained in one of the "internal services" IP Pool ranges listed
3962024-04-26T00:16:49.612Z # below.
3972024-04-26T00:16:49.612Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
3982024-04-26T00:16:49.612Z+external_dns_ips = [ "10.151.2.164", "10.151.2.165" ]
3992024-04-26T00:16:49.612Z
4002024-04-26T00:16:49.612Z # Initial TLS certificates for the external API
4012024-04-26T00:16:49.612Z #
4022024-04-26T00:16:49.612Z@@ -69,8 +69,8 @@
4032024-04-26T00:16:49.613Z #
4042024-04-26T00:16:49.613Z # For more on this and what to put here, see docs/how-to-run.adoc.
4052024-04-26T00:16:49.613Z [[internal_services_ip_pool_ranges]]
4062024-04-26T00:16:49.613Z-first = "192.168.1.20"
4072024-04-26T00:16:49.613Z-last = "192.168.1.29"
4082024-04-26T00:16:49.613Z+first = "10.151.2.164"
4092024-04-26T00:16:49.613Z+last = "10.151.2.173"
4102024-04-26T00:16:49.613Z
4112024-04-26T00:16:49.613Z # TODO - this configuration is subject to change going forward. Ultimately these
4122024-04-26T00:16:49.613Z # parameters should be provided to the control plane via wicket, but we need to
4132024-04-26T00:16:49.613Z@@ -91,8 +91,8 @@
4142024-04-26T00:16:49.614Z # A range of IP addresses used by Boundary Services on the external network. In
4152024-04-26T00:16:49.614Z # a real system, these would be addresses of the uplink ports on the Sidecar.
4162024-04-26T00:16:49.614Z # With softnpu, only one address is used.
4172024-04-26T00:16:49.614Z-infra_ip_first = "192.168.1.30"
4182024-04-26T00:16:49.614Z-infra_ip_last = "192.168.1.30"
4192024-04-26T00:16:49.614Z+infra_ip_first = "10.151.2.174"
4202024-04-26T00:16:49.614Z+infra_ip_last = "10.151.2.174"
4212024-04-26T00:16:49.614Z
4222024-04-26T00:16:49.614Z # Configurations for BGP routers to run on the scrimlets.
4232024-04-26T00:16:49.614Z bgp = []
4242024-04-26T00:16:49.614Z@@ -100,9 +100,9 @@
4252024-04-26T00:16:49.614Z # You can configure multiple uplinks by repeating the following stanza
4262024-04-26T00:16:49.615Z [[rack_network_config.ports]]
4272024-04-26T00:16:49.615Z # Routes associated with this port.
4282024-04-26T00:16:49.615Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
4292024-04-26T00:16:49.615Z+routes = [{nexthop = "10.151.2.100", destination = "0.0.0.0/0"}]
4302024-04-26T00:16:49.615Z # Addresses associated with this port.
4312024-04-26T00:16:49.615Z-addresses = ["192.168.1.30/24"]
4322024-04-26T00:16:49.641Z+addresses = ["10.151.2.174/24"]
4332024-04-26T00:16:49.641Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
4342024-04-26T00:16:49.641Z port = "qsfp0"
4352024-04-26T00:16:49.641Z # The speed of this port.
4362024-04-26T00:16:49.641Z+ true
4372024-04-26T00:16:49.641Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml pkg/config.toml
4382024-04-26T00:16:49.641Za pkg/config-rss.toml 7K
4392024-04-26T00:16:49.641Za pkg/config.toml 4K
4402024-04-26T00:16:49.642Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~ pkg/config.toml pkg/config.toml~
4412024-04-26T00:16:49.642Z+ TLS_NAME=recovery.sys.oxide.test
4422024-04-26T00:16:49.642Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
4432024-04-26T00:16:50.474Z.+............+..+.+..+...+....+...+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+....+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+.....+....+......+........+.+...........+..............................+...+..........+........+....+...+......+......+.....+....+.................+.......+...+.....+.......+...+........+....+..+..........+.....................+..+.+.....+.+..............+.+..................+............+..........................+.......+..+..........+...+...........+......+....+...........+......+...+....+..+.+............+..+.........+............+............+.......+......+...+..+...+.........+.+.....+.......+.....+.......+.....+...+.......+...+........+.......+..............+.....................+...+.........+...............+.+.....+......+.+..+.............+.....+......................+...........+.......+........+....+...+...+..........................+.............+......+........................+.....+.+..+....+...........+.........+..........+........+............+...+......+.+...+........+.......+........+....+........+.+..+.......+......+......+.....+...+................+.....+.+...+............+...........+........................+...+.+...............+..+....+..............+.......+..............+...+......+.+......+...............+............+...............+...+...+.....................+.....+......+...+....+..+...+......+......+..................................+......+...+...+........+................+.....+...+.+.........+...........+......+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
4442024-04-26T00:16:51.015Z............+..+.+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+...+.+......+..............+.+......+...............+..+......+.........+.+.....+.+......+..+...+...+...................+...+..+.......+...+..+...+.+..+..........+...+.........+......+.....+......+...+.+.....+.+...+..+...+......+......+....+..+.......+..+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+.+...+.....+............+.+.....+...+.+...+...+..+..........+.....................+..+.........+...+.+..+............+.+............+................................+......+....+......+............+...+...........+...+..........+............+...+..+...+......+.+.....+......+...............+..........+...+.........+...+...+..+............+...+.......+...+.........+...............+.....+...+.+..+...+............+...+....+...........+....+...+..+...+....+...............+...+........+.............+............+........+.+.....+...+.......+.....+......+.......+..+...+...+....+........+.+.....+.+..............+.+..............+.......+.....+....+....................................+............+..+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
4452024-04-26T00:16:51.044Z-----
4462024-04-26T00:16:51.044Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
4472024-04-26T00:16:51.044Za pkg/initial-tls-cert.pem 2K
4482024-04-26T00:16:51.044Za pkg/initial-tls-key.pem 4K
4492024-04-26T00:16:51.044Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
4502024-04-26T00:16:51.070Z+ rmdir pkg
4512024-04-26T00:16:51.070Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
4522024-04-26T00:16:51.070Z+ pfexec mkdir -p /usr/oxide
4532024-04-26T00:16:51.070Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0
4542024-04-26T00:16:51.737Z+ pfexec chmod +x /usr/oxide/oxide
4552024-04-26T00:16:51.764Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
4562024-04-26T00:17:10.304Z+ OMICRON_NO_UNINSTALL=1
4572024-04-26T00:17:10.305Z+ ptime -m pfexec ./target/release/omicron-package -t test install
4582024-04-26T00:17:10.330ZLogging to: /opt/oxide/work/out/LOG
4592024-04-26T00:17:10.840Z
4602024-04-26T00:17:10.841Zreal 0.531069542
4612024-04-26T00:17:10.866Zuser 0.111988232
4622024-04-26T00:17:10.866Zsys 2.230446094
4632024-04-26T00:17:10.866Ztrap 0.000008289
4642024-04-26T00:17:10.866Ztflt 0.000260841
4652024-04-26T00:17:10.867Zdflt 0.001005677
4662024-04-26T00:17:10.867Zkflt 0.000000000
4672024-04-26T00:17:10.867Zlock 48.200571690
4682024-04-26T00:17:10.867Zslp 0.745579332
4692024-04-26T00:17:10.867Zlat 0.003190136
4702024-04-26T00:17:10.867Zstop 0.204642460
4712024-04-26T00:17:10.867Z+ retry=0
4722024-04-26T00:17:10.867Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
4732024-04-26T00:17:10.868Z+ [[ 0 -gt 30 ]]
4742024-04-26T00:17:10.868Z+ sleep 1
4752024-04-26T00:17:11.862Z+ retry=1
4762024-04-26T00:17:11.863Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
4772024-04-26T00:17:11.888Z+ [[ 1 -gt 30 ]]
4782024-04-26T00:17:11.888Z+ sleep 1
4792024-04-26T00:17:12.871Z+ retry=2
4802024-04-26T00:17:12.871Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
4812024-04-26T00:17:12.895Z+ [[ 2 -gt 30 ]]
4822024-04-26T00:17:12.896Z+ sleep 1
4832024-04-26T00:17:13.886Z+ retry=3
4842024-04-26T00:17:13.886Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
4852024-04-26T00:17:39.058Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
4862024-04-26T00:17:39.086Zlocal v6:
4872024-04-26T00:17:39.087Zfd0a:9ce6:e92c:992f::1
4882024-04-26T00:17:39.087Zfe80::940d:73ff:fedb:7e56
4892024-04-26T00:17:39.087Zlocal v4:
4902024-04-26T00:17:39.087Z10.151.2.174
4912024-04-26T00:17:39.087Zrouter v6:
4922024-04-26T00:17:39.087Zfd00:1122:3344:2::/64 -> fe80::ec40:56ff:fe4d:7235 (1)
4932024-04-26T00:17:39.087Zfd00:1122:3344:101::/64 -> fe80::ec40:56ff:fe4d:7235 (1)
4942024-04-26T00:17:39.088Zfd00:1122:3344:1::/64 -> fe80::ec40:56ff:fe4d:7235 (1)
4952024-04-26T00:17:39.088Zfdb0:18c0:4d0d:a02a::/64 -> fe80::ec40:56ff:fe4d:7235 (1)
4962024-04-26T00:17:39.088Zfd00:1122:3344:3::/64 -> fe80::ec40:56ff:fe4d:7235 (1)
4972024-04-26T00:17:39.088Zrouter v4_idx:
4982024-04-26T00:17:39.088Z0.0.0.0/0 -> 0
4992024-04-26T00:17:39.088Zrouter v4_routes:
5002024-04-26T00:17:39.088Z0 -> 10.151.2.100 (2)
5012024-04-26T00:17:39.088Zresolver v4:
5022024-04-26T00:17:39.088Z10.151.2.174 -> 76:9e:c1:dd:de:7e
5032024-04-26T00:17:39.088Z10.151.2.100 -> 18:c0:4d:0d:a0:2a
5042024-04-26T00:17:39.089Zresolver v6:
5052024-04-26T00:17:39.089Zfe80::940d:73ff:fedb:7e56 -> 96:0d:73:db:7e:56
5062024-04-26T00:17:39.089Zfe80::ec40:56ff:fe4d:7235 -> ee:40:56:4d:72:35
5072024-04-26T00:17:39.089Znat v4:
5082024-04-26T00:17:39.526Z10.151.2.169 0/16383 -> fd00:1122:3344:101::1 100/a8:40:25:ff:c8:dc
5092024-04-26T00:17:39.526Znat_v6:
5102024-04-26T00:17:39.526Zport_mac:
5112024-04-26T00:17:39.526Z1000: a8:40:25:03:52:38
5122024-04-26T00:17:39.527Z1: a8:40:25:03:52:39
5132024-04-26T00:17:39.527Z2: a8:40:25:03:52:59
5142024-04-26T00:17:39.527Zproxy arp:
5152024-04-26T00:17:39.527Z10.151.2.164/10.151.2.227: a8:e1:de:01:70:1d
5162024-04-26T00:17:39.527Z+ retry=0
5172024-04-26T00:17:39.528Z+++ zoneadm list -n
5182024-04-26T00:17:39.528Z+++ grep oxz_ntp
5192024-04-26T00:17:39.528Z++ pfexec svcs -z oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0 -Hostate oxide/ntp
5202024-04-26T00:17:39.553Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
5212024-04-26T00:17:39.554Z++ true
5222024-04-26T00:17:39.554Z+ [[ '' != online ]]
5232024-04-26T00:17:39.554Z+ [[ 0 -gt 60 ]]
5242024-04-26T00:17:39.554Z+ sleep 1
5252024-04-26T00:17:40.125Z+ retry=1
5262024-04-26T00:17:40.157Z+++ zoneadm list -n
5272024-04-26T00:17:40.157Z+++ grep oxz_ntp
5282024-04-26T00:17:40.157Z++ pfexec svcs -z oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0 -Hostate oxide/ntp
5292024-04-26T00:17:40.158Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
5302024-04-26T00:17:40.158Z++ true
5312024-04-26T00:17:40.158Z+ [[ '' != online ]]
5322024-04-26T00:17:40.158Z+ [[ 1 -gt 60 ]]
5332024-04-26T00:17:40.158Z+ sleep 1
5342024-04-26T00:17:41.154Z+ retry=2
5352024-04-26T00:17:41.180Z+++ zoneadm list -n
5362024-04-26T00:17:41.180Z+++ grep oxz_ntp
5372024-04-26T00:17:41.180Z++ pfexec svcs -z oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0 -Hostate oxide/ntp
5382024-04-26T00:17:41.180Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
5392024-04-26T00:17:41.180Z++ true
5402024-04-26T00:17:41.204Z+ [[ '' != online ]]
5412024-04-26T00:17:41.204Z+ [[ 2 -gt 60 ]]
5422024-04-26T00:17:41.204Z+ sleep 1
5432024-04-26T00:17:42.184Z+ retry=3
5442024-04-26T00:17:42.209Z+++ zoneadm list -n
5452024-04-26T00:17:42.209Z+++ grep oxz_ntp
5462024-04-26T00:17:42.209Z++ pfexec svcs -z oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0 -Hostate oxide/ntp
5472024-04-26T00:17:42.209Z+ [[ offline != online ]]
5482024-04-26T00:17:42.209Z+ [[ 3 -gt 60 ]]
5492024-04-26T00:17:42.209Z+ sleep 1
5502024-04-26T00:17:43.205Z+ retry=4
5512024-04-26T00:17:43.229Z+++ zoneadm list -n
5522024-04-26T00:17:43.229Z+++ grep oxz_ntp
5532024-04-26T00:17:43.229Z++ pfexec svcs -z oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0 -Hostate oxide/ntp
5542024-04-26T00:17:43.229Z+ [[ offline != online ]]
5552024-04-26T00:17:43.229Z+ [[ 4 -gt 60 ]]
5562024-04-26T00:17:43.229Z+ sleep 1
5572024-04-26T00:17:44.229Z+ retry=5
5582024-04-26T00:17:44.263Z+++ zoneadm list -n
5592024-04-26T00:17:44.263Z+++ grep oxz_ntp
5602024-04-26T00:17:44.263Z++ pfexec svcs -z oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0 -Hostate oxide/ntp
5612024-04-26T00:17:44.263Z+ [[ offline != online ]]
5622024-04-26T00:17:44.263Z+ [[ 5 -gt 60 ]]
5632024-04-26T00:17:44.263Z+ sleep 1
5642024-04-26T00:17:45.249Z+ retry=6
5652024-04-26T00:17:45.275Z+++ zoneadm list -n
5662024-04-26T00:17:45.275Z+++ grep oxz_ntp
5672024-04-26T00:17:45.275Z++ pfexec svcs -z oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0 -Hostate oxide/ntp
5682024-04-26T00:17:45.276Z+ [[ offline != online ]]
5692024-04-26T00:17:45.276Z+ [[ 6 -gt 60 ]]
5702024-04-26T00:17:45.276Z+ sleep 1
5712024-04-26T00:17:46.272Z+ retry=7
5722024-04-26T00:17:46.297Z+++ zoneadm list -n
5732024-04-26T00:17:46.297Z+++ grep oxz_ntp
5742024-04-26T00:17:46.297Z++ pfexec svcs -z oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0 -Hostate oxide/ntp
5752024-04-26T00:17:46.297Z+ [[ offline != online ]]
5762024-04-26T00:17:46.297Z+ [[ 7 -gt 60 ]]
5772024-04-26T00:17:46.297Z+ sleep 1
5782024-04-26T00:17:47.294Z+ retry=8
5792024-04-26T00:17:47.318Z+++ zoneadm list -n
5802024-04-26T00:17:47.318Z+++ grep oxz_ntp
5812024-04-26T00:17:47.318Z++ pfexec svcs -z oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0 -Hostate oxide/ntp
5822024-04-26T00:17:47.318Z+ [[ offline != online ]]
5832024-04-26T00:17:47.318Z+ [[ 8 -gt 60 ]]
5842024-04-26T00:17:47.318Z+ sleep 1
 
5852024-04-26T00:17:47.341Zsock console login: Apr 26 00:17:47 sock ip: ip: joining multicasts failed (4) on vopte0 - will use link layer broadcasts for multicast
 
5862024-04-26T00:17:48.315Z+ retry=9
5872024-04-26T00:17:48.340Z+++ zoneadm list -n
5882024-04-26T00:17:48.340Z+++ grep oxz_ntp
5892024-04-26T00:17:48.340Z++ pfexec svcs -z oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0 -Hostate oxide/ntp
5902024-04-26T00:17:48.340Z+ [[ online != online ]]
5912024-04-26T00:17:48.340Z+ echo 'Waited for chrony: 9s'
5922024-04-26T00:17:48.340ZWaited for chrony: 9s
5932024-04-26T00:17:48.340Z+ retry=0
5942024-04-26T00:17:48.340Z+ zoneadm list
5952024-04-26T00:17:48.340Z+ grep nexus
5962024-04-26T00:17:48.364Z+ [[ 0 -gt 300 ]]
5972024-04-26T00:17:48.364Z+ sleep 1
5982024-04-26T00:17:49.343Z+ retry=1
5992024-04-26T00:17:49.367Z+ zoneadm list
6002024-04-26T00:17:49.367Z+ grep nexus
6012024-04-26T00:17:49.367Z+ [[ 1 -gt 300 ]]
6022024-04-26T00:17:49.367Z+ sleep 1
6032024-04-26T00:17:50.364Z+ retry=2
6042024-04-26T00:17:50.391Z+ zoneadm list
6052024-04-26T00:17:50.391Z+ grep nexus
6062024-04-26T00:17:50.391Z+ [[ 2 -gt 300 ]]
6072024-04-26T00:17:50.391Z+ sleep 1
6082024-04-26T00:17:51.365Z+ retry=3
6092024-04-26T00:17:51.390Z+ zoneadm list
6102024-04-26T00:17:51.390Z+ grep nexus
6112024-04-26T00:17:51.390Z+ [[ 3 -gt 300 ]]
6122024-04-26T00:17:51.390Z+ sleep 1
6132024-04-26T00:17:52.376Z+ retry=4
6142024-04-26T00:17:52.403Z+ zoneadm list
6152024-04-26T00:17:52.403Z+ grep nexus
6162024-04-26T00:17:52.403Z+ [[ 4 -gt 300 ]]
6172024-04-26T00:17:52.403Z+ sleep 1
6182024-04-26T00:17:53.385Z+ retry=5
6192024-04-26T00:17:53.410Z+ zoneadm list
6202024-04-26T00:17:53.410Z+ grep nexus
6212024-04-26T00:17:53.410Z+ [[ 5 -gt 300 ]]
6222024-04-26T00:17:53.410Z+ sleep 1
6232024-04-26T00:17:54.396Z+ retry=6
6242024-04-26T00:17:54.420Z+ zoneadm list
6252024-04-26T00:17:54.420Z+ grep nexus
6262024-04-26T00:17:54.420Z+ [[ 6 -gt 300 ]]
6272024-04-26T00:17:54.420Z+ sleep 1
6282024-04-26T00:17:55.407Z+ retry=7
6292024-04-26T00:17:55.431Z+ zoneadm list
6302024-04-26T00:17:55.431Z+ grep nexus
6312024-04-26T00:17:55.431Z+ [[ 7 -gt 300 ]]
6322024-04-26T00:17:55.431Z+ sleep 1
6332024-04-26T00:17:56.419Z+ retry=8
6342024-04-26T00:17:56.443Z+ zoneadm list
6352024-04-26T00:17:56.443Z+ grep nexus
6362024-04-26T00:17:56.443Z+ [[ 8 -gt 300 ]]
6372024-04-26T00:17:56.443Z+ sleep 1
6382024-04-26T00:17:57.430Z+ retry=9
6392024-04-26T00:17:57.455Z+ zoneadm list
6402024-04-26T00:17:57.455Z+ grep nexus
6412024-04-26T00:17:57.455Z+ [[ 9 -gt 300 ]]
6422024-04-26T00:17:57.455Z+ sleep 1
6432024-04-26T00:17:58.443Z+ retry=10
6442024-04-26T00:17:58.467Z+ zoneadm list
6452024-04-26T00:17:58.467Z+ grep nexus
6462024-04-26T00:17:58.492Z+ [[ 10 -gt 300 ]]
6472024-04-26T00:17:58.492Z+ sleep 1
6482024-04-26T00:17:59.491Z+ retry=11
6492024-04-26T00:17:59.524Z+ zoneadm list
6502024-04-26T00:17:59.550Z+ grep nexus
6512024-04-26T00:17:59.576Z+ [[ 11 -gt 300 ]]
6522024-04-26T00:17:59.576Z+ sleep 1
6532024-04-26T00:18:00.579Z+ retry=12
6542024-04-26T00:18:00.603Z+ zoneadm list
6552024-04-26T00:18:00.603Z+ grep nexus
6562024-04-26T00:18:00.603Z+ [[ 12 -gt 300 ]]
6572024-04-26T00:18:00.603Z+ sleep 1
6582024-04-26T00:18:01.593Z+ retry=13
6592024-04-26T00:18:01.618Z+ zoneadm list
6602024-04-26T00:18:01.618Z+ grep nexus
6612024-04-26T00:18:01.618Z+ [[ 13 -gt 300 ]]
6622024-04-26T00:18:01.618Z+ sleep 1
6632024-04-26T00:18:02.606Z+ retry=14
6642024-04-26T00:18:02.631Z+ zoneadm list
6652024-04-26T00:18:02.631Z+ grep nexus
6662024-04-26T00:18:02.631Z+ [[ 14 -gt 300 ]]
6672024-04-26T00:18:02.631Z+ sleep 1
6682024-04-26T00:18:03.623Z+ retry=15
6692024-04-26T00:18:03.648Z+ zoneadm list
6702024-04-26T00:18:03.648Z+ grep nexus
6712024-04-26T00:18:03.648Z+ [[ 15 -gt 300 ]]
6722024-04-26T00:18:03.648Z+ sleep 1
6732024-04-26T00:18:04.641Z+ retry=16
6742024-04-26T00:18:04.670Z+ zoneadm list
6752024-04-26T00:18:04.670Z+ grep nexus
6762024-04-26T00:18:04.670Z+ [[ 16 -gt 300 ]]
6772024-04-26T00:18:04.670Z+ sleep 1
6782024-04-26T00:18:05.645Z+ retry=17
6792024-04-26T00:18:05.670Z+ zoneadm list
6802024-04-26T00:18:05.670Z+ grep nexus
6812024-04-26T00:18:05.670Z+ [[ 17 -gt 300 ]]
6822024-04-26T00:18:05.670Z+ sleep 1
6832024-04-26T00:18:06.658Z+ retry=18
6842024-04-26T00:18:06.683Z+ zoneadm list
6852024-04-26T00:18:06.683Z+ grep nexus
6862024-04-26T00:18:06.683Z+ [[ 18 -gt 300 ]]
6872024-04-26T00:18:06.683Z+ sleep 1
6882024-04-26T00:18:07.670Z+ retry=19
6892024-04-26T00:18:07.694Z+ zoneadm list
6902024-04-26T00:18:07.694Z+ grep nexus
6912024-04-26T00:18:07.694Z+ [[ 19 -gt 300 ]]
6922024-04-26T00:18:07.694Z+ sleep 1
6932024-04-26T00:18:08.687Z+ retry=20
6942024-04-26T00:18:08.715Z+ zoneadm list
6952024-04-26T00:18:08.715Z+ grep nexus
6962024-04-26T00:18:08.715Z+ [[ 20 -gt 300 ]]
6972024-04-26T00:18:08.715Z+ sleep 1
6982024-04-26T00:18:09.704Z+ retry=21
6992024-04-26T00:18:09.728Z+ zoneadm list
7002024-04-26T00:18:09.728Z+ grep nexus
7012024-04-26T00:18:09.728Z+ [[ 21 -gt 300 ]]
7022024-04-26T00:18:09.728Z+ sleep 1
7032024-04-26T00:18:11.840Z+ retry=22
7042024-04-26T00:18:11.841Z+ zoneadm list
7052024-04-26T00:18:11.841Z+ grep nexus
7062024-04-26T00:18:11.841Z+ [[ 22 -gt 300 ]]
7072024-04-26T00:18:11.841Z+ sleep 1
7082024-04-26T00:18:12.830Z+ retry=23
7092024-04-26T00:18:12.854Z+ zoneadm list
7102024-04-26T00:18:12.854Z+ grep nexus
7112024-04-26T00:18:12.854Z+ [[ 23 -gt 300 ]]
7122024-04-26T00:18:12.855Z+ sleep 1
7132024-04-26T00:18:13.844Z+ retry=24
7142024-04-26T00:18:13.869Z+ zoneadm list
7152024-04-26T00:18:13.869Z+ grep nexus
7162024-04-26T00:18:13.869Z+ [[ 24 -gt 300 ]]
7172024-04-26T00:18:13.869Z+ sleep 1
7182024-04-26T00:18:14.862Z+ retry=25
7192024-04-26T00:18:14.887Z+ zoneadm list
7202024-04-26T00:18:14.887Z+ grep nexus
7212024-04-26T00:18:14.913Z+ [[ 25 -gt 300 ]]
7222024-04-26T00:18:14.913Z+ sleep 1
7232024-04-26T00:18:15.894Z+ retry=26
7242024-04-26T00:18:15.919Z+ zoneadm list
7252024-04-26T00:18:15.919Z+ grep nexus
7262024-04-26T00:18:15.919Z+ [[ 26 -gt 300 ]]
7272024-04-26T00:18:15.919Z+ sleep 1
7282024-04-26T00:18:16.906Z+ retry=27
7292024-04-26T00:18:16.930Z+ zoneadm list
7302024-04-26T00:18:16.930Z+ grep nexus
7312024-04-26T00:18:16.930Z+ [[ 27 -gt 300 ]]
7322024-04-26T00:18:16.931Z+ sleep 1
7332024-04-26T00:18:17.917Z+ retry=28
7342024-04-26T00:18:17.942Z+ zoneadm list
7352024-04-26T00:18:17.942Z+ grep nexus
7362024-04-26T00:18:17.942Z+ [[ 28 -gt 300 ]]
7372024-04-26T00:18:17.942Z+ sleep 1
7382024-04-26T00:18:18.929Z+ retry=29
7392024-04-26T00:18:18.953Z+ zoneadm list
7402024-04-26T00:18:18.953Z+ grep nexus
7412024-04-26T00:18:18.954Z+ [[ 29 -gt 300 ]]
7422024-04-26T00:18:18.954Z+ sleep 1
7432024-04-26T00:18:19.941Z+ retry=30
7442024-04-26T00:18:19.965Z+ zoneadm list
7452024-04-26T00:18:19.965Z+ grep nexus
7462024-04-26T00:18:19.966Z+ [[ 30 -gt 300 ]]
7472024-04-26T00:18:19.966Z+ sleep 1
7482024-04-26T00:18:20.953Z+ retry=31
7492024-04-26T00:18:20.978Z+ zoneadm list
7502024-04-26T00:18:20.978Z+ grep nexus
7512024-04-26T00:18:20.978Z+ [[ 31 -gt 300 ]]
7522024-04-26T00:18:20.978Z+ sleep 1
7532024-04-26T00:18:21.965Z+ retry=32
7542024-04-26T00:18:21.989Z+ zoneadm list
7552024-04-26T00:18:21.989Z+ grep nexus
7562024-04-26T00:18:21.989Z+ [[ 32 -gt 300 ]]
7572024-04-26T00:18:21.989Z+ sleep 1
7582024-04-26T00:18:22.978Z+ retry=33
7592024-04-26T00:18:23.002Z+ zoneadm list
7602024-04-26T00:18:23.002Z+ grep nexus
7612024-04-26T00:18:23.002Z+ [[ 33 -gt 300 ]]
7622024-04-26T00:18:23.002Z+ sleep 1
7632024-04-26T00:18:23.989Z+ retry=34
7642024-04-26T00:18:24.014Z+ zoneadm list
7652024-04-26T00:18:24.014Z+ grep nexus
7662024-04-26T00:18:24.014Z+ [[ 34 -gt 300 ]]
7672024-04-26T00:18:24.014Z+ sleep 1
7682024-04-26T00:18:25.001Z+ retry=35
7692024-04-26T00:18:25.025Z+ zoneadm list
7702024-04-26T00:18:25.026Z+ grep nexus
7712024-04-26T00:18:25.026Z+ [[ 35 -gt 300 ]]
7722024-04-26T00:18:25.026Z+ sleep 1
7732024-04-26T00:18:26.013Z+ retry=36
7742024-04-26T00:18:26.037Z+ zoneadm list
7752024-04-26T00:18:26.037Z+ grep nexus
7762024-04-26T00:18:26.037Z+ [[ 36 -gt 300 ]]
7772024-04-26T00:18:26.038Z+ sleep 1
7782024-04-26T00:18:27.026Z+ retry=37
7792024-04-26T00:18:27.051Z+ zoneadm list
7802024-04-26T00:18:27.051Z+ grep nexus
7812024-04-26T00:18:27.051Z+ [[ 37 -gt 300 ]]
7822024-04-26T00:18:27.051Z+ sleep 1
7832024-04-26T00:18:28.036Z+ retry=38
7842024-04-26T00:18:28.062Z+ zoneadm list
7852024-04-26T00:18:28.063Z+ grep nexus
7862024-04-26T00:18:28.063Z+ [[ 38 -gt 300 ]]
7872024-04-26T00:18:28.063Z+ sleep 1
7882024-04-26T00:18:29.049Z+ retry=39
7892024-04-26T00:18:29.074Z+ zoneadm list
7902024-04-26T00:18:29.074Z+ grep nexus
7912024-04-26T00:18:29.074Z+ [[ 39 -gt 300 ]]
7922024-04-26T00:18:29.074Z+ sleep 1
7932024-04-26T00:18:30.061Z+ retry=40
7942024-04-26T00:18:30.086Z+ zoneadm list
7952024-04-26T00:18:30.086Z+ grep nexus
7962024-04-26T00:18:30.086Z+ [[ 40 -gt 300 ]]
7972024-04-26T00:18:30.086Z+ sleep 1
7982024-04-26T00:18:31.079Z+ retry=41
7992024-04-26T00:18:31.129Z+ zoneadm list
8002024-04-26T00:18:31.129Z+ grep nexus
8012024-04-26T00:18:31.129Z+ [[ 41 -gt 300 ]]
8022024-04-26T00:18:31.129Z+ sleep 1
8032024-04-26T00:18:32.086Z+ retry=42
8042024-04-26T00:18:32.111Z+ zoneadm list
8052024-04-26T00:18:32.111Z+ grep nexus
8062024-04-26T00:18:32.111Z+ [[ 42 -gt 300 ]]
8072024-04-26T00:18:32.111Z+ sleep 1
8082024-04-26T00:18:33.098Z+ retry=43
8092024-04-26T00:18:33.123Z+ zoneadm list
8102024-04-26T00:18:33.123Z+ grep nexus
8112024-04-26T00:18:33.123Z+ [[ 43 -gt 300 ]]
8122024-04-26T00:18:33.123Z+ sleep 1
8132024-04-26T00:18:34.110Z+ retry=44
8142024-04-26T00:18:34.136Z+ zoneadm list
8152024-04-26T00:18:34.136Z+ grep nexus
8162024-04-26T00:18:34.136Z+ [[ 44 -gt 300 ]]
8172024-04-26T00:18:34.136Z+ sleep 1
8182024-04-26T00:18:35.123Z+ retry=45
8192024-04-26T00:18:35.148Z+ zoneadm list
8202024-04-26T00:18:35.149Z+ grep nexus
8212024-04-26T00:18:35.149Z+ [[ 45 -gt 300 ]]
8222024-04-26T00:18:35.149Z+ sleep 1
8232024-04-26T00:18:36.138Z+ retry=46
8242024-04-26T00:18:36.138Z+ zoneadm list
8252024-04-26T00:18:36.138Z+ grep nexus
8262024-04-26T00:18:36.163Z+ [[ 46 -gt 300 ]]
8272024-04-26T00:18:36.163Z+ sleep 1
8282024-04-26T00:18:37.148Z+ retry=47
8292024-04-26T00:18:37.175Z+ zoneadm list
8302024-04-26T00:18:37.175Z+ grep nexus
8312024-04-26T00:18:37.176Z+ [[ 47 -gt 300 ]]
8322024-04-26T00:18:37.176Z+ sleep 1
8332024-04-26T00:18:38.159Z+ retry=48
8342024-04-26T00:18:38.184Z+ zoneadm list
8352024-04-26T00:18:38.184Z+ grep nexus
8362024-04-26T00:18:38.184Z+ [[ 48 -gt 300 ]]
8372024-04-26T00:18:38.184Z+ sleep 1
8382024-04-26T00:18:39.171Z+ retry=49
8392024-04-26T00:18:39.204Z+ zoneadm list
8402024-04-26T00:18:39.204Z+ grep nexus
8412024-04-26T00:18:39.204Z+ [[ 49 -gt 300 ]]
8422024-04-26T00:18:39.204Z+ sleep 1
8432024-04-26T00:18:40.183Z+ retry=50
8442024-04-26T00:18:40.209Z+ zoneadm list
8452024-04-26T00:18:40.209Z+ grep nexus
8462024-04-26T00:18:40.210Z+ [[ 50 -gt 300 ]]
8472024-04-26T00:18:40.210Z+ sleep 1
8482024-04-26T00:18:41.195Z+ retry=51
8492024-04-26T00:18:41.220Z+ zoneadm list
8502024-04-26T00:18:41.220Z+ grep nexus
8512024-04-26T00:18:41.220Z+ [[ 51 -gt 300 ]]
8522024-04-26T00:18:41.220Z+ sleep 1
8532024-04-26T00:18:42.207Z+ retry=52
8542024-04-26T00:18:42.232Z+ zoneadm list
8552024-04-26T00:18:42.232Z+ grep nexus
8562024-04-26T00:18:42.232Z+ [[ 52 -gt 300 ]]
8572024-04-26T00:18:42.232Z+ sleep 1
8582024-04-26T00:18:43.219Z+ retry=53
8592024-04-26T00:18:43.244Z+ zoneadm list
8602024-04-26T00:18:43.244Z+ grep nexus
8612024-04-26T00:18:43.244Z+ [[ 53 -gt 300 ]]
8622024-04-26T00:18:43.244Z+ sleep 1
8632024-04-26T00:18:44.231Z+ retry=54
8642024-04-26T00:18:44.256Z+ zoneadm list
8652024-04-26T00:18:44.256Z+ grep nexus
8662024-04-26T00:18:44.256Z+ [[ 54 -gt 300 ]]
8672024-04-26T00:18:44.257Z+ sleep 1
8682024-04-26T00:18:45.244Z+ retry=55
8692024-04-26T00:18:45.269Z+ zoneadm list
8702024-04-26T00:18:45.269Z+ grep nexus
8712024-04-26T00:18:45.269Z+ [[ 55 -gt 300 ]]
8722024-04-26T00:18:45.269Z+ sleep 1
8732024-04-26T00:18:46.256Z+ retry=56
8742024-04-26T00:18:46.287Z+ zoneadm list
8752024-04-26T00:18:46.287Z+ grep nexus
8762024-04-26T00:18:46.287Z+ [[ 56 -gt 300 ]]
8772024-04-26T00:18:46.287Z+ sleep 1
8782024-04-26T00:18:47.268Z+ retry=57
8792024-04-26T00:18:47.293Z+ zoneadm list
8802024-04-26T00:18:47.293Z+ grep nexus
8812024-04-26T00:18:47.293Z+ [[ 57 -gt 300 ]]
8822024-04-26T00:18:47.293Z+ sleep 1
8832024-04-26T00:18:48.279Z+ retry=58
8842024-04-26T00:18:48.304Z+ zoneadm list
8852024-04-26T00:18:48.304Z+ grep nexus
8862024-04-26T00:18:48.304Z+ [[ 58 -gt 300 ]]
8872024-04-26T00:18:48.304Z+ sleep 1
8882024-04-26T00:18:49.291Z+ retry=59
8892024-04-26T00:18:49.316Z+ zoneadm list
8902024-04-26T00:18:49.316Z+ grep nexus
8912024-04-26T00:18:49.316Z+ [[ 59 -gt 300 ]]
8922024-04-26T00:18:49.316Z+ sleep 1
8932024-04-26T00:18:50.304Z+ retry=60
8942024-04-26T00:18:50.329Z+ zoneadm list
8952024-04-26T00:18:50.329Z+ grep nexus
8962024-04-26T00:18:50.330Z+ [[ 60 -gt 300 ]]
8972024-04-26T00:18:50.330Z+ sleep 1
8982024-04-26T00:18:51.316Z+ retry=61
8992024-04-26T00:18:51.342Z+ zoneadm list
9002024-04-26T00:18:51.342Z+ grep nexus
9012024-04-26T00:18:51.342Z+ [[ 61 -gt 300 ]]
9022024-04-26T00:18:51.342Z+ sleep 1
9032024-04-26T00:18:52.328Z+ retry=62
9042024-04-26T00:18:52.355Z+ zoneadm list
9052024-04-26T00:18:52.355Z+ grep nexus
9062024-04-26T00:18:52.355Z+ [[ 62 -gt 300 ]]
9072024-04-26T00:18:52.355Z+ sleep 1
9082024-04-26T00:18:53.340Z+ retry=63
9092024-04-26T00:18:53.365Z+ zoneadm list
9102024-04-26T00:18:53.365Z+ grep nexus
9112024-04-26T00:18:53.365Z+ [[ 63 -gt 300 ]]
9122024-04-26T00:18:53.365Z+ sleep 1
9132024-04-26T00:18:54.353Z+ retry=64
9142024-04-26T00:18:54.378Z+ zoneadm list
9152024-04-26T00:18:54.378Z+ grep nexus
9162024-04-26T00:18:54.378Z+ [[ 64 -gt 300 ]]
9172024-04-26T00:18:54.378Z+ sleep 1
9182024-04-26T00:18:55.365Z+ retry=65
9192024-04-26T00:18:55.392Z+ zoneadm list
9202024-04-26T00:18:55.392Z+ grep nexus
9212024-04-26T00:18:55.392Z+ [[ 65 -gt 300 ]]
9222024-04-26T00:18:55.392Z+ sleep 1
9232024-04-26T00:18:56.789Z+ retry=66
9242024-04-26T00:18:56.814Z+ zoneadm list
9252024-04-26T00:18:56.814Z+ grep nexus
9262024-04-26T00:18:56.814Z+ [[ 66 -gt 300 ]]
9272024-04-26T00:18:56.814Z+ sleep 1
9282024-04-26T00:18:57.388Z+ retry=67
9292024-04-26T00:18:57.413Z+ zoneadm list
9302024-04-26T00:18:57.413Z+ grep nexus
9312024-04-26T00:18:57.413Z+ [[ 67 -gt 300 ]]
9322024-04-26T00:18:57.413Z+ sleep 1
9332024-04-26T00:18:58.402Z+ retry=68
9342024-04-26T00:18:58.427Z+ zoneadm list
9352024-04-26T00:18:58.427Z+ grep nexus
9362024-04-26T00:18:58.427Z+ [[ 68 -gt 300 ]]
9372024-04-26T00:18:58.427Z+ sleep 1
9382024-04-26T00:18:59.411Z+ retry=69
9392024-04-26T00:18:59.437Z+ zoneadm list
9402024-04-26T00:18:59.437Z+ grep nexus
9412024-04-26T00:18:59.437Z+ [[ 69 -gt 300 ]]
9422024-04-26T00:18:59.437Z+ sleep 1
9432024-04-26T00:19:00.422Z+ retry=70
9442024-04-26T00:19:00.447Z+ zoneadm list
9452024-04-26T00:19:00.447Z+ grep nexus
9462024-04-26T00:19:00.447Z+ [[ 70 -gt 300 ]]
9472024-04-26T00:19:00.447Z+ sleep 1
9482024-04-26T00:19:01.434Z+ retry=71
9492024-04-26T00:19:01.462Z+ zoneadm list
9502024-04-26T00:19:01.462Z+ grep nexus
9512024-04-26T00:19:01.463Z+ [[ 71 -gt 300 ]]
9522024-04-26T00:19:01.463Z+ sleep 1
9532024-04-26T00:19:02.449Z+ retry=72
9542024-04-26T00:19:02.489Z+ zoneadm list
9552024-04-26T00:19:02.489Z+ grep nexus
9562024-04-26T00:19:02.489Z+ [[ 72 -gt 300 ]]
9572024-04-26T00:19:02.489Z+ sleep 1
9582024-04-26T00:19:03.455Z+ retry=73
9592024-04-26T00:19:03.481Z+ zoneadm list
9602024-04-26T00:19:03.482Z+ grep nexus
9612024-04-26T00:19:03.482Z+ [[ 73 -gt 300 ]]
9622024-04-26T00:19:03.482Z+ sleep 1
9632024-04-26T00:19:04.467Z+ retry=74
9642024-04-26T00:19:04.493Z+ zoneadm list
9652024-04-26T00:19:04.493Z+ grep nexus
9662024-04-26T00:19:04.493Z+ [[ 74 -gt 300 ]]
9672024-04-26T00:19:04.493Z+ sleep 1
9682024-04-26T00:19:05.479Z+ retry=75
9692024-04-26T00:19:05.505Z+ zoneadm list
9702024-04-26T00:19:05.505Z+ grep nexus
9712024-04-26T00:19:05.505Z+ [[ 75 -gt 300 ]]
9722024-04-26T00:19:05.505Z+ sleep 1
9732024-04-26T00:19:06.490Z+ retry=76
9742024-04-26T00:19:06.516Z+ zoneadm list
9752024-04-26T00:19:06.516Z+ grep nexus
9762024-04-26T00:19:06.516Z+ [[ 76 -gt 300 ]]
9772024-04-26T00:19:06.516Z+ sleep 1
9782024-04-26T00:19:07.502Z+ retry=77
9792024-04-26T00:19:07.528Z+ zoneadm list
9802024-04-26T00:19:07.528Z+ grep nexus
9812024-04-26T00:19:07.528Z+ [[ 77 -gt 300 ]]
9822024-04-26T00:19:07.529Z+ sleep 1
9832024-04-26T00:19:08.539Z+ retry=78
9842024-04-26T00:19:08.589Z+ zoneadm list
9852024-04-26T00:19:08.589Z+ grep nexus
9862024-04-26T00:19:08.589Z+ [[ 78 -gt 300 ]]
9872024-04-26T00:19:08.589Z+ sleep 1
9882024-04-26T00:19:15.043Z+ retry=79
9892024-04-26T00:19:15.077Z+ zoneadm list
9902024-04-26T00:19:15.078Z+ grep nexus
9912024-04-26T00:19:15.078Z+ [[ 79 -gt 300 ]]
9922024-04-26T00:19:15.078Z+ sleep 1
9932024-04-26T00:19:15.078Z+ retry=80
9942024-04-26T00:19:15.078Z+ zoneadm list
9952024-04-26T00:19:15.078Z+ grep nexus
9962024-04-26T00:19:15.078Z+ [[ 80 -gt 300 ]]
9972024-04-26T00:19:15.082Z+ sleep 1
9982024-04-26T00:19:15.156Z+ retry=81
9992024-04-26T00:19:15.156Z+ zoneadm list
10002024-04-26T00:19:15.156Z+ grep nexus
10012024-04-26T00:19:15.157Z+ [[ 81 -gt 300 ]]
10022024-04-26T00:19:15.157Z+ sleep 1
10032024-04-26T00:19:15.157Z+ retry=82
10042024-04-26T00:19:15.157Z+ zoneadm list
10052024-04-26T00:19:15.157Z+ grep nexus
10062024-04-26T00:19:15.157Z+ [[ 82 -gt 300 ]]
10072024-04-26T00:19:15.157Z+ sleep 1
10082024-04-26T00:19:15.157Z+ retry=83
10092024-04-26T00:19:15.157Z+ zoneadm list
10102024-04-26T00:19:15.157Z+ grep nexus
10112024-04-26T00:19:15.157Z+ [[ 83 -gt 300 ]]
10122024-04-26T00:19:15.158Z+ sleep 1
10132024-04-26T00:19:15.158Z+ retry=84
10142024-04-26T00:19:15.158Z+ zoneadm list
10152024-04-26T00:19:15.158Z+ grep nexus
10162024-04-26T00:19:15.158Z+ [[ 84 -gt 300 ]]
10172024-04-26T00:19:15.158Z+ sleep 1
10182024-04-26T00:19:15.595Z+ retry=85
10192024-04-26T00:19:15.621Z+ zoneadm list
10202024-04-26T00:19:15.621Z+ grep nexus
10212024-04-26T00:19:15.621Z+ [[ 85 -gt 300 ]]
10222024-04-26T00:19:15.621Z+ sleep 1
10232024-04-26T00:19:16.606Z+ retry=86
10242024-04-26T00:19:16.631Z+ zoneadm list
10252024-04-26T00:19:16.632Z+ grep nexus
10262024-04-26T00:19:16.632Z+ [[ 86 -gt 300 ]]
10272024-04-26T00:19:16.632Z+ sleep 1
10282024-04-26T00:19:17.617Z+ retry=87
10292024-04-26T00:19:17.642Z+ zoneadm list
10302024-04-26T00:19:17.643Z+ grep nexus
10312024-04-26T00:19:17.643Z+ [[ 87 -gt 300 ]]
10322024-04-26T00:19:17.643Z+ sleep 1
10332024-04-26T00:19:18.630Z+ retry=88
10342024-04-26T00:19:18.655Z+ zoneadm list
10352024-04-26T00:19:18.655Z+ grep nexus
10362024-04-26T00:19:18.655Z+ [[ 88 -gt 300 ]]
10372024-04-26T00:19:18.655Z+ sleep 1
10382024-04-26T00:19:19.643Z+ retry=89
10392024-04-26T00:19:19.668Z+ zoneadm list
10402024-04-26T00:19:19.668Z+ grep nexus
10412024-04-26T00:19:19.668Z+ [[ 89 -gt 300 ]]
10422024-04-26T00:19:19.668Z+ sleep 1
10432024-04-26T00:19:20.655Z+ retry=90
10442024-04-26T00:19:20.680Z+ zoneadm list
10452024-04-26T00:19:20.680Z+ grep nexus
10462024-04-26T00:19:20.681Z+ [[ 90 -gt 300 ]]
10472024-04-26T00:19:20.681Z+ sleep 1
10482024-04-26T00:19:21.668Z+ retry=91
10492024-04-26T00:19:21.693Z+ zoneadm list
10502024-04-26T00:19:21.693Z+ grep nexus
10512024-04-26T00:19:21.693Z+ [[ 91 -gt 300 ]]
10522024-04-26T00:19:21.693Z+ sleep 1
10532024-04-26T00:19:22.681Z+ retry=92
10542024-04-26T00:19:22.706Z+ zoneadm list
10552024-04-26T00:19:22.706Z+ grep nexus
10562024-04-26T00:19:22.706Z+ [[ 92 -gt 300 ]]
10572024-04-26T00:19:22.706Z+ sleep 1
10582024-04-26T00:19:23.693Z+ retry=93
10592024-04-26T00:19:23.718Z+ zoneadm list
10602024-04-26T00:19:23.718Z+ grep nexus
10612024-04-26T00:19:23.718Z+ [[ 93 -gt 300 ]]
10622024-04-26T00:19:23.718Z+ sleep 1
10632024-04-26T00:19:24.784Z+ retry=94
10642024-04-26T00:19:24.992Z+ zoneadm list
10652024-04-26T00:19:24.993Z+ grep nexus
10662024-04-26T00:19:24.993Z+ [[ 94 -gt 300 ]]
10672024-04-26T00:19:24.993Z+ sleep 1
10682024-04-26T00:19:25.740Z+ retry=95
10692024-04-26T00:19:25.943Z+ zoneadm list
10702024-04-26T00:19:25.944Z+ grep nexus
10712024-04-26T00:19:25.944Z+ [[ 95 -gt 300 ]]
10722024-04-26T00:19:25.944Z+ sleep 1
10732024-04-26T00:19:26.742Z+ retry=96
10742024-04-26T00:19:26.797Z+ zoneadm list
10752024-04-26T00:19:26.797Z+ grep nexus
10762024-04-26T00:19:26.797Z+ [[ 96 -gt 300 ]]
10772024-04-26T00:19:26.797Z+ sleep 1
10782024-04-26T00:19:27.740Z+ retry=97
10792024-04-26T00:19:27.766Z+ zoneadm list
10802024-04-26T00:19:27.766Z+ grep nexus
10812024-04-26T00:19:27.766Z+ [[ 97 -gt 300 ]]
10822024-04-26T00:19:27.766Z+ sleep 1
10832024-04-26T00:19:28.777Z+ retry=98
10842024-04-26T00:19:28.891Z+ zoneadm list
10852024-04-26T00:19:28.891Z+ grep nexus
10862024-04-26T00:19:28.891Z+ [[ 98 -gt 300 ]]
10872024-04-26T00:19:28.891Z+ sleep 1
10882024-04-26T00:19:29.868Z+ retry=99
10892024-04-26T00:19:29.960Z+ zoneadm list
10902024-04-26T00:19:29.960Z+ grep nexus
10912024-04-26T00:19:29.960Z+ [[ 99 -gt 300 ]]
10922024-04-26T00:19:29.960Z+ sleep 1
10932024-04-26T00:19:30.801Z+ retry=100
10942024-04-26T00:19:30.869Z+ zoneadm list
10952024-04-26T00:19:30.869Z+ grep nexus
10962024-04-26T00:19:30.869Z+ [[ 100 -gt 300 ]]
10972024-04-26T00:19:30.869Z+ sleep 1
10982024-04-26T00:19:31.833Z+ retry=101
10992024-04-26T00:19:31.884Z+ zoneadm list
11002024-04-26T00:19:31.884Z+ grep nexus
11012024-04-26T00:19:31.884Z+ [[ 101 -gt 300 ]]
11022024-04-26T00:19:31.884Z+ sleep 1
11032024-04-26T00:19:32.883Z+ retry=102
11042024-04-26T00:19:33.031Z+ zoneadm list
11052024-04-26T00:19:33.031Z+ grep nexus
11062024-04-26T00:19:33.031Z+ [[ 102 -gt 300 ]]
11072024-04-26T00:19:33.031Z+ sleep 1
11082024-04-26T00:19:33.835Z+ retry=103
11092024-04-26T00:19:33.952Z+ zoneadm list
11102024-04-26T00:19:33.953Z+ grep nexus
11112024-04-26T00:19:33.953Z+ [[ 103 -gt 300 ]]
11122024-04-26T00:19:33.953Z+ sleep 1
11132024-04-26T00:19:34.839Z+ retry=104
11142024-04-26T00:19:34.865Z+ zoneadm list
11152024-04-26T00:19:34.865Z+ grep nexus
11162024-04-26T00:19:34.865Z+ [[ 104 -gt 300 ]]
11172024-04-26T00:19:34.865Z+ sleep 1
11182024-04-26T00:19:35.940Z+ retry=105
11192024-04-26T00:19:36.079Z+ zoneadm list
11202024-04-26T00:19:36.080Z+ grep nexus
11212024-04-26T00:19:36.080Z+ [[ 105 -gt 300 ]]
11222024-04-26T00:19:36.080Z+ sleep 1
11232024-04-26T00:19:36.951Z+ retry=106
11242024-04-26T00:19:37.231Z+ zoneadm list
11252024-04-26T00:19:37.231Z+ grep nexus
11262024-04-26T00:19:37.231Z+ [[ 106 -gt 300 ]]
11272024-04-26T00:19:37.232Z+ sleep 1
11282024-04-26T00:19:37.883Z+ retry=107
11292024-04-26T00:19:37.914Z+ zoneadm list
11302024-04-26T00:19:37.914Z+ grep nexus
11312024-04-26T00:19:37.915Z+ [[ 107 -gt 300 ]]
11322024-04-26T00:19:37.915Z+ sleep 1
11332024-04-26T00:19:38.892Z+ retry=108
11342024-04-26T00:19:38.917Z+ zoneadm list
11352024-04-26T00:19:38.917Z+ grep nexus
11362024-04-26T00:19:38.917Z+ [[ 108 -gt 300 ]]
11372024-04-26T00:19:38.917Z+ sleep 1
11382024-04-26T00:19:39.912Z+ retry=109
11392024-04-26T00:19:39.937Z+ zoneadm list
11402024-04-26T00:19:39.937Z+ grep nexus
11412024-04-26T00:19:39.937Z+ [[ 109 -gt 300 ]]
11422024-04-26T00:19:39.937Z+ sleep 1
11432024-04-26T00:19:40.936Z+ retry=110
11442024-04-26T00:19:40.962Z+ zoneadm list
11452024-04-26T00:19:40.962Z+ grep nexus
11462024-04-26T00:19:40.962Z+ [[ 110 -gt 300 ]]
11472024-04-26T00:19:40.963Z+ sleep 1
11482024-04-26T00:19:43.935Z+ retry=111
11492024-04-26T00:19:44.248Z+ zoneadm list
11502024-04-26T00:19:44.401Z+ grep nexus
11512024-04-26T00:19:44.401Z+ [[ 111 -gt 300 ]]
11522024-04-26T00:19:44.401Z+ sleep 1
11532024-04-26T00:19:45.622Z+ retry=112
11542024-04-26T00:19:45.651Z+ zoneadm list
11552024-04-26T00:19:45.651Z+ grep nexus
11562024-04-26T00:19:45.693Z+ [[ 112 -gt 300 ]]
11572024-04-26T00:19:45.724Z+ sleep 1
11582024-04-26T00:19:47.985Z+ retry=113
11592024-04-26T00:19:47.985Z+ zoneadm list
11602024-04-26T00:19:48.011Z+ grep nexus
11612024-04-26T00:19:48.011Z+ [[ 113 -gt 300 ]]
11622024-04-26T00:19:48.012Z+ sleep 1
11632024-04-26T00:19:48.967Z+ retry=114
11642024-04-26T00:19:48.993Z+ zoneadm list
11652024-04-26T00:19:48.993Z+ grep nexus
11662024-04-26T00:19:49.019Z+ [[ 114 -gt 300 ]]
11672024-04-26T00:19:49.019Z+ sleep 1
11682024-04-26T00:19:50.032Z+ retry=115
11692024-04-26T00:19:50.057Z+ zoneadm list
11702024-04-26T00:19:50.057Z+ grep nexus
11712024-04-26T00:19:50.057Z+ [[ 115 -gt 300 ]]
11722024-04-26T00:19:50.058Z+ sleep 1
11732024-04-26T00:19:51.051Z+ retry=116
11742024-04-26T00:19:51.077Z+ zoneadm list
11752024-04-26T00:19:51.078Z+ grep nexus
11762024-04-26T00:19:51.078Z+ [[ 116 -gt 300 ]]
11772024-04-26T00:19:51.078Z+ sleep 1
11782024-04-26T00:19:52.172Z+ retry=117
11792024-04-26T00:19:52.286Z+ zoneadm list
11802024-04-26T00:19:52.287Z+ grep nexus
11812024-04-26T00:19:52.287Z+ [[ 117 -gt 300 ]]
11822024-04-26T00:19:52.287Z+ sleep 1
11832024-04-26T00:19:53.087Z+ retry=118
11842024-04-26T00:19:53.117Z+ zoneadm list
11852024-04-26T00:19:53.118Z+ grep nexus
11862024-04-26T00:19:53.118Z+ [[ 118 -gt 300 ]]
11872024-04-26T00:19:53.118Z+ sleep 1
11882024-04-26T00:19:54.102Z+ retry=119
11892024-04-26T00:19:54.127Z+ zoneadm list
11902024-04-26T00:19:54.127Z+ grep nexus
11912024-04-26T00:19:54.127Z+ [[ 119 -gt 300 ]]
11922024-04-26T00:19:54.127Z+ sleep 1
11932024-04-26T00:19:55.135Z+ retry=120
11942024-04-26T00:19:55.173Z+ zoneadm list
11952024-04-26T00:19:55.173Z+ grep nexus
11962024-04-26T00:19:55.173Z+ [[ 120 -gt 300 ]]
11972024-04-26T00:19:55.174Z+ sleep 1
11982024-04-26T00:19:56.148Z+ retry=121
11992024-04-26T00:19:56.173Z+ zoneadm list
12002024-04-26T00:19:56.173Z+ grep nexus
12012024-04-26T00:19:56.173Zoxz_nexus_2d696101-87c5-4c96-b623-2e415a336b1d
12022024-04-26T00:19:56.173Z+ echo 'Waited for nexus: 121s'
12032024-04-26T00:19:56.173ZWaited for nexus: 121s
12042024-04-26T00:19:56.174Z+ export RUST_BACKTRACE=1
12052024-04-26T00:19:56.174Z+ RUST_BACKTRACE=1
12062024-04-26T00:19:56.174Z+ export E2E_TLS_CERT IPPOOL_START IPPOOL_END
12072024-04-26T00:19:56.174Z++ ./tests/bootstrap
12082024-04-26T00:19:59.249ZUsing external DNS server at 10.151.2.164:53
12092024-04-26T00:19:59.307Z2024-04-26 00:19:59.284124650 UTC: attempting to log into API
12102024-04-26T00:20:14.367Z2024-04-26 00:20:14.344545198 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
12112024-04-26T00:20:15.370Z2024-04-26 00:20:15.346511438 UTC: attempting to log into API
 
12122024-04-26T00:20:19.445ZApr 26 00:20:18 sock ip: ip: joining multicasts failed (4) on vopte1 - will use link layer broadcasts for multicast
12132024-04-26T00:20:19.812ZApr 26 00:20:19 sock ip: ip: joining multicasts failed (4) on vopte4 - will use link layer broadcasts for multicast
12142024-04-26T00:20:20.500ZApr 26 00:20:20 sock ip: ip: joining multicasts failed (4) on vopte5 - will use link layer broadcasts for multicast
12152024-04-26T00:20:20.599ZApr 26 00:20:20 sock ip: ip: joining multicasts failed (4) on vopte2 - will use link layer broadcasts for multicast
12162024-04-26T00:20:20.912ZApr 26 00:20:20 sock ip: ip: joining multicasts failed (4) on vopte3 - will use link layer broadcasts for multicast
 
12172024-04-26T00:20:30.459Z2024-04-26 00:20:30.398473771 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
12182024-04-26T00:20:31.424Z2024-04-26 00:20:31.400097975 UTC: attempting to log into API
12192024-04-26T00:20:46.456Z2024-04-26 00:20:46.430255446 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
12202024-04-26T00:20:47.457Z2024-04-26 00:20:47.431276339 UTC: attempting to log into API
12212024-04-26T00:20:58.244Z2024-04-26 00:20:58.217675047 UTC: login succeeded
12222024-04-26T00:20:58.276Zwaiting for nexus to come up...
12232024-04-26T00:20:58.348Zcreating IP pool... 10.151.2.175 - 10.151.2.227
12242024-04-26T00:20:58.556Zsetting up quotas...
12252024-04-26T00:20:58.606Zensuring datasets are ready...
12262024-04-26T00:21:01.157Zlet's roll.
12272024-04-26T00:21:01.183Z+ eval 'OXIDE_HOST=https://recovery.sys.oxide.test
12282024-04-26T00:21:01.183ZOXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.2.166
12292024-04-26T00:21:01.183ZOXIDE_TOKEN=oxide-token-3dbaa58f3f1c757b34a45d01b502b18be5d04fc3'
12302024-04-26T00:21:01.183Z++ OXIDE_HOST=https://recovery.sys.oxide.test
12312024-04-26T00:21:01.183Z++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.2.166
12322024-04-26T00:21:01.183Z++ OXIDE_TOKEN=oxide-token-3dbaa58f3f1c757b34a45d01b502b18be5d04fc3
12332024-04-26T00:21:01.183Z+ export OXIDE_HOST OXIDE_TOKEN
12342024-04-26T00:21:01.183Z+ retry=0
12352024-04-26T00:21:01.183Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12362024-04-26T00:21:01.183Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12372024-04-26T00:21:01.183Z+ [[ 0 -gt 60 ]]
12382024-04-26T00:21:01.183Z+ sleep 1
12392024-04-26T00:21:02.184Z+ retry=1
12402024-04-26T00:21:02.185Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12412024-04-26T00:21:02.210Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12422024-04-26T00:21:02.210Z+ [[ 1 -gt 60 ]]
12432024-04-26T00:21:02.211Z+ sleep 1
12442024-04-26T00:21:03.204Z+ retry=2
12452024-04-26T00:21:03.205Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12462024-04-26T00:21:03.230Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12472024-04-26T00:21:03.230Z+ [[ 2 -gt 60 ]]
12482024-04-26T00:21:03.230Z+ sleep 1
12492024-04-26T00:21:04.225Z+ retry=3
12502024-04-26T00:21:04.225Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12512024-04-26T00:21:04.259Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12522024-04-26T00:21:04.259Z+ [[ 3 -gt 60 ]]
12532024-04-26T00:21:04.260Z+ sleep 1
12542024-04-26T00:21:05.242Z+ retry=4
12552024-04-26T00:21:05.242Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12562024-04-26T00:21:05.271Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12572024-04-26T00:21:05.271Z+ [[ 4 -gt 60 ]]
12582024-04-26T00:21:05.271Z+ sleep 1
12592024-04-26T00:21:06.279Z+ retry=5
12602024-04-26T00:21:06.279Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12612024-04-26T00:21:06.357Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12622024-04-26T00:21:06.357Z+ [[ 5 -gt 60 ]]
12632024-04-26T00:21:06.358Z+ sleep 1
12642024-04-26T00:21:07.269Z+ retry=6
12652024-04-26T00:21:07.269Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12662024-04-26T00:21:07.298Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12672024-04-26T00:21:07.298Z+ [[ 6 -gt 60 ]]
12682024-04-26T00:21:07.298Z+ sleep 1
12692024-04-26T00:21:08.286Z+ retry=7
12702024-04-26T00:21:08.286Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12712024-04-26T00:21:08.315Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12722024-04-26T00:21:08.316Z+ [[ 7 -gt 60 ]]
12732024-04-26T00:21:08.316Z+ sleep 1
12742024-04-26T00:21:09.300Z+ retry=8
12752024-04-26T00:21:09.300Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12762024-04-26T00:21:09.325Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12772024-04-26T00:21:09.325Z+ [[ 8 -gt 60 ]]
12782024-04-26T00:21:09.325Z+ sleep 1
12792024-04-26T00:21:10.326Z+ retry=9
12802024-04-26T00:21:10.326Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12812024-04-26T00:21:10.351Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12822024-04-26T00:21:10.352Z+ [[ 9 -gt 60 ]]
12832024-04-26T00:21:10.352Z+ sleep 1
12842024-04-26T00:21:11.540Z+ retry=10
12852024-04-26T00:21:11.540Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12862024-04-26T00:21:11.540Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12872024-04-26T00:21:11.540Z+ [[ 10 -gt 60 ]]
12882024-04-26T00:21:11.540Z+ sleep 1
12892024-04-26T00:21:12.347Z+ retry=11
12902024-04-26T00:21:12.347Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12912024-04-26T00:21:12.375Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12922024-04-26T00:21:12.376Z+ [[ 11 -gt 60 ]]
12932024-04-26T00:21:12.376Z+ sleep 1
12942024-04-26T00:21:13.416Z+ retry=12
12952024-04-26T00:21:13.416Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
12962024-04-26T00:21:13.460Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
12972024-04-26T00:21:13.461Z+ [[ 12 -gt 60 ]]
12982024-04-26T00:21:13.461Z+ sleep 1
12992024-04-26T00:21:14.383Z+ retry=13
13002024-04-26T00:21:14.383Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
13012024-04-26T00:21:14.421Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
13022024-04-26T00:21:14.421Z+ [[ 13 -gt 60 ]]
13032024-04-26T00:21:14.421Z+ sleep 1
13042024-04-26T00:21:15.398Z+ retry=14
13052024-04-26T00:21:15.399Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
13062024-04-26T00:21:15.428Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
13072024-04-26T00:21:15.428Z+ [[ 14 -gt 60 ]]
13082024-04-26T00:21:15.428Z+ sleep 1
13092024-04-26T00:21:16.415Z+ retry=15
13102024-04-26T00:21:16.415Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
13112024-04-26T00:21:16.441Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
13122024-04-26T00:21:16.441Z+ [[ 15 -gt 60 ]]
13132024-04-26T00:21:16.441Z+ sleep 1
13142024-04-26T00:21:17.441Z+ retry=16
13152024-04-26T00:21:17.441Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
13162024-04-26T00:21:17.476Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
13172024-04-26T00:21:17.476Z+ [[ 16 -gt 60 ]]
13182024-04-26T00:21:17.476Z+ sleep 1
13192024-04-26T00:21:18.449Z+ retry=17
13202024-04-26T00:21:18.449Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
13212024-04-26T00:21:18.480Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
13222024-04-26T00:21:18.480Z+ [[ 17 -gt 60 ]]
13232024-04-26T00:21:18.480Z+ sleep 1
13242024-04-26T00:21:19.462Z+ retry=18
13252024-04-26T00:21:19.463Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
13262024-04-26T00:21:19.487Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
13272024-04-26T00:21:19.488Z+ [[ 18 -gt 60 ]]
13282024-04-26T00:21:19.488Z+ sleep 1
13292024-04-26T00:21:20.479Z+ retry=19
13302024-04-26T00:21:20.479Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
13312024-04-26T00:21:20.505Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
13322024-04-26T00:21:20.506Z+ [[ 19 -gt 60 ]]
13332024-04-26T00:21:20.506Z+ sleep 1
13342024-04-26T00:21:21.500Z+ retry=20
13352024-04-26T00:21:21.501Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
13362024-04-26T00:21:21.531Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
13372024-04-26T00:21:21.531Z+ [[ 20 -gt 60 ]]
13382024-04-26T00:21:21.532Z+ sleep 1
13392024-04-26T00:21:22.535Z+ retry=21
13402024-04-26T00:21:22.535Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
13412024-04-26T00:21:22.587Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
13422024-04-26T00:21:22.587Z+ [[ 21 -gt 60 ]]
13432024-04-26T00:21:22.588Z+ sleep 1
13442024-04-26T00:21:23.530Z+ retry=22
13452024-04-26T00:21:23.530Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
13462024-04-26T00:21:23.574Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
13472024-04-26T00:21:23.574Z+ [[ 22 -gt 60 ]]
13482024-04-26T00:21:23.574Z+ sleep 1
13492024-04-26T00:21:24.547Z+ retry=23
13502024-04-26T00:21:24.547Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
13512024-04-26T00:21:24.572Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
13522024-04-26T00:21:24.573Z+ [[ 23 -gt 60 ]]
13532024-04-26T00:21:24.573Z+ sleep 1
13542024-04-26T00:21:25.564Z+ retry=24
13552024-04-26T00:21:25.564Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
13562024-04-26T00:21:25.649Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images'
13572024-04-26T00:21:26.185Z{"status":"ok"}success
13582024-04-26T00:21:26.185ZProject {
13592024-04-26T00:21:26.210Z description: "some images",
13602024-04-26T00:21:26.210Z id: 29530592-a6fe-4169-aa04-4d4fce1f784c,
13612024-04-26T00:21:26.210Z name: Name(
13622024-04-26T00:21:26.210Z "images",
13632024-04-26T00:21:26.210Z ),
13642024-04-26T00:21:26.210Z time_created: 2024-04-26T00:21:25.736910Z,
13652024-04-26T00:21:26.211Z time_modified: 2024-04-26T00:21:25.736910Z,
13662024-04-26T00:21:26.211Z}
13672024-04-26T00:21:26.211Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11
13682024-04-26T00:26:57.387Zdone!
13692024-04-26T00:26:57.490Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11
13702024-04-26T00:26:57.607Zsuccess
13712024-04-26T00:26:57.607ZImage {
13722024-04-26T00:26:57.632Z block_size: ByteCount(
13732024-04-26T00:26:57.632Z 512,
13742024-04-26T00:26:57.633Z ),
13752024-04-26T00:26:57.633Z description: "debian 11 original base image",
13762024-04-26T00:26:57.633Z digest: None,
13772024-04-26T00:26:57.633Z id: 0eb29643-22fd-4609-9614-3202bc451960,
13782024-04-26T00:26:57.633Z name: Name(
13792024-04-26T00:26:57.633Z "debian11",
13802024-04-26T00:26:57.633Z ),
13812024-04-26T00:26:57.633Z os: "debian",
13822024-04-26T00:26:57.633Z project_id: None,
13832024-04-26T00:26:57.633Z size: ByteCount(
13842024-04-26T00:26:57.633Z 2147483648,
13852024-04-26T00:26:57.633Z ),
13862024-04-26T00:26:57.633Z time_created: 2024-04-26T00:26:57.326743Z,
13872024-04-26T00:26:57.633Z time_modified: 2024-04-26T00:26:57.582582Z,
13882024-04-26T00:26:57.633Z url: None,
13892024-04-26T00:26:57.633Z version: "11",
13902024-04-26T00:26:57.633Z}
13912024-04-26T00:26:57.633Z+ rm ./tests/bootstrap
13922024-04-26T00:26:57.681Z+ for test_bin in tests/*
13932024-04-26T00:26:57.681Z+ ./tests/bootstrap-72919e54dd0a1e8f
13942024-04-26T00:26:57.743Z
13952024-04-26T00:26:57.743Zrunning 0 tests
13962024-04-26T00:26:57.768Z
13972024-04-26T00:26:57.768Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
13982024-04-26T00:26:57.768Z
13992024-04-26T00:26:57.768Z+ for test_bin in tests/*
14002024-04-26T00:26:57.768Z+ ./tests/commtest-07d1782656389f6e
14012024-04-26T00:26:57.793Z
14022024-04-26T00:26:57.793Zrunning 0 tests
14032024-04-26T00:26:57.793Z
14042024-04-26T00:26:57.793Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
14052024-04-26T00:26:57.793Z
14062024-04-26T00:26:57.793Z+ for test_bin in tests/*
14072024-04-26T00:26:57.793Z+ ./tests/dhcp_server-9e0ac39a731573d2
14082024-04-26T00:26:57.829Z
14092024-04-26T00:26:57.829Zrunning 0 tests
14102024-04-26T00:26:57.829Z
14112024-04-26T00:26:57.854Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
14122024-04-26T00:26:57.854Z
14132024-04-26T00:26:57.854Z+ for test_bin in tests/*
14142024-04-26T00:26:57.854Z+ ./tests/end_to_end_tests-1a59c060357c1b59
14152024-04-26T00:26:58.090Z
14162024-04-26T00:26:58.090Zrunning 2 tests
14172024-04-26T00:26:58.333Ztest no_spoof::no_spoof_here ... ok
14182024-04-26T00:27:30.293Ztest instance_launch::instance_launch ... ok
14192024-04-26T00:27:30.293Z
14202024-04-26T00:27:30.293Ztest result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 32.20s
14212024-04-26T00:27:30.294Z
14222024-04-26T00:27:30.319Z+ _exit_trap
14232024-04-26T00:27:30.319Z+ local status=0
14242024-04-26T00:27:30.319Z+ [[ 0 -eq 0 ]]
14252024-04-26T00:27:30.319Z+ exit 0
14262024-04-26T00:27:30.319Zprocess exited: duration 684327 ms, exit code 0
 
14272024-04-26T00:27:30.447Zfound 101 output files
14282024-04-26T00:27:30.447Zuploading: /var/svc/log/oxide-sled-agent:default.log (1759580 bytes)
14292024-04-26T00:27:33.022Zuploaded: /var/svc/log/oxide-sled-agent:default.log
14302024-04-26T00:27:33.022Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log (2705291 bytes)
14312024-04-26T00:27:33.839Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log" changed size mid upload: 2705291 -> 2720373
14322024-04-26T00:27:34.914Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log
14332024-04-26T00:27:34.914Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log (52279 bytes)
14342024-04-26T00:27:36.117Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log
14352024-04-26T00:27:36.117Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log (211029 bytes)
14362024-04-26T00:27:36.316Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log" changed size mid upload: 211029 -> 213424
14372024-04-26T00:27:37.389Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log
14382024-04-26T00:27:37.414Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log (262721 bytes)
14392024-04-26T00:27:37.624Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log" changed size mid upload: 262721 -> 265910
14402024-04-26T00:27:38.697Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log
14412024-04-26T00:27:38.697Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log (1435721 bytes)
14422024-04-26T00:27:39.442Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log" changed size mid upload: 1435721 -> 1457287
14432024-04-26T00:27:40.515Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log
14442024-04-26T00:27:40.515Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log (808821 bytes)
14452024-04-26T00:27:41.323Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log" changed size mid upload: 808821 -> 821977
14462024-04-26T00:27:42.395Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log
14472024-04-26T00:27:42.395Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log (1678 bytes)
14482024-04-26T00:27:43.492Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log
14492024-04-26T00:27:43.517Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log (289392 bytes)
14502024-04-26T00:27:43.704Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log" changed size mid upload: 289392 -> 295955
14512024-04-26T00:27:44.781Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log
14522024-04-26T00:27:44.781Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log (8222 bytes)
14532024-04-26T00:27:45.900Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log
14542024-04-26T00:27:45.900Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log (5393 bytes)
14552024-04-26T00:27:47.003Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log
14562024-04-26T00:27:47.003Zuploading: /pool/ext/0de19191-ee35-4d6e-ab04-5e684dafd16c/crypt/zone/oxz_crucible_0ff6eb74-346e-45ea-890b-3a8a4237dcff/root/var/svc/log/oxide-crucible-agent:default.log (22310 bytes)
14572024-04-26T00:27:48.147Zuploaded: /pool/ext/0de19191-ee35-4d6e-ab04-5e684dafd16c/crypt/zone/oxz_crucible_0ff6eb74-346e-45ea-890b-3a8a4237dcff/root/var/svc/log/oxide-crucible-agent:default.log
14582024-04-26T00:27:48.147Zuploading: /pool/ext/0de19191-ee35-4d6e-ab04-5e684dafd16c/crypt/zone/oxz_crucible_0ff6eb74-346e-45ea-890b-3a8a4237dcff/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
14592024-04-26T00:27:49.244Zuploaded: /pool/ext/0de19191-ee35-4d6e-ab04-5e684dafd16c/crypt/zone/oxz_crucible_0ff6eb74-346e-45ea-890b-3a8a4237dcff/root/var/svc/log/oxide-crucible-downstairs:default.log
14602024-04-26T00:27:49.244Zuploading: /pool/ext/0de19191-ee35-4d6e-ab04-5e684dafd16c/crypt/zone/oxz_crucible_0ff6eb74-346e-45ea-890b-3a8a4237dcff/root/var/svc/log/oxide-crucible-downstairs:downstairs-560b1ad5-c812-4d1e-a982-4d626eb49e2e.log (5694 bytes)
14612024-04-26T00:27:50.342Zuploaded: /pool/ext/0de19191-ee35-4d6e-ab04-5e684dafd16c/crypt/zone/oxz_crucible_0ff6eb74-346e-45ea-890b-3a8a4237dcff/root/var/svc/log/oxide-crucible-downstairs:downstairs-560b1ad5-c812-4d1e-a982-4d626eb49e2e.log
14622024-04-26T00:27:50.342Zuploading: /pool/ext/0de19191-ee35-4d6e-ab04-5e684dafd16c/crypt/zone/oxz_crucible_0ff6eb74-346e-45ea-890b-3a8a4237dcff/root/var/svc/log/oxide-crucible-downstairs:downstairs-80f73632-2d71-4255-a775-b1349c82a610.log (3020 bytes)
14632024-04-26T00:27:51.443Zuploaded: /pool/ext/0de19191-ee35-4d6e-ab04-5e684dafd16c/crypt/zone/oxz_crucible_0ff6eb74-346e-45ea-890b-3a8a4237dcff/root/var/svc/log/oxide-crucible-downstairs:downstairs-80f73632-2d71-4255-a775-b1349c82a610.log
14642024-04-26T00:27:51.443Zuploading: /pool/ext/0de19191-ee35-4d6e-ab04-5e684dafd16c/crypt/zone/oxz_crucible_0ff6eb74-346e-45ea-890b-3a8a4237dcff/root/var/svc/log/oxide-crucible-downstairs:snapshot-560b1ad5-c812-4d1e-a982-4d626eb49e2e-8a8cb26d-a12c-47cb-a73a-5f6e4573a290.log (5303 bytes)
14652024-04-26T00:27:52.555Zuploaded: /pool/ext/0de19191-ee35-4d6e-ab04-5e684dafd16c/crypt/zone/oxz_crucible_0ff6eb74-346e-45ea-890b-3a8a4237dcff/root/var/svc/log/oxide-crucible-downstairs:snapshot-560b1ad5-c812-4d1e-a982-4d626eb49e2e-8a8cb26d-a12c-47cb-a73a-5f6e4573a290.log
14662024-04-26T00:27:52.555Zuploading: /pool/ext/0de19191-ee35-4d6e-ab04-5e684dafd16c/crypt/zone/oxz_crucible_0ff6eb74-346e-45ea-890b-3a8a4237dcff/root/var/svc/log/oxide-zone-network-setup:default.log (1683 bytes)
14672024-04-26T00:27:53.653Zuploaded: /pool/ext/0de19191-ee35-4d6e-ab04-5e684dafd16c/crypt/zone/oxz_crucible_0ff6eb74-346e-45ea-890b-3a8a4237dcff/root/var/svc/log/oxide-zone-network-setup:default.log
14682024-04-26T00:27:53.653Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_cockroachdb_ac169011-c605-4b75-aa7b-c3144c1ddd86/root/var/svc/log/oxide-cockroachdb:default.log (4407 bytes)
14692024-04-26T00:27:54.751Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_cockroachdb_ac169011-c605-4b75-aa7b-c3144c1ddd86/root/var/svc/log/oxide-cockroachdb:default.log
14702024-04-26T00:27:54.751Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_cockroachdb_ac169011-c605-4b75-aa7b-c3144c1ddd86/root/var/svc/log/oxide-zone-network-setup:default.log (1690 bytes)
14712024-04-26T00:27:55.851Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_cockroachdb_ac169011-c605-4b75-aa7b-c3144c1ddd86/root/var/svc/log/oxide-zone-network-setup:default.log
14722024-04-26T00:27:55.876Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_a64df5e3-8d68-4581-8828-def85de45a96/root/var/svc/log/oxide-crucible-agent:default.log (1556 bytes)
14732024-04-26T00:27:56.945Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_a64df5e3-8d68-4581-8828-def85de45a96/root/var/svc/log/oxide-crucible-agent:default.log
14742024-04-26T00:27:56.945Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_a64df5e3-8d68-4581-8828-def85de45a96/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
14752024-04-26T00:27:58.041Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_a64df5e3-8d68-4581-8828-def85de45a96/root/var/svc/log/oxide-crucible-downstairs:default.log
14762024-04-26T00:27:58.041Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_a64df5e3-8d68-4581-8828-def85de45a96/root/var/svc/log/oxide-zone-network-setup:default.log (1682 bytes)
14772024-04-26T00:27:59.138Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_a64df5e3-8d68-4581-8828-def85de45a96/root/var/svc/log/oxide-zone-network-setup:default.log
14782024-04-26T00:27:59.138Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_pantry_652e3013-976a-43ba-bb68-b6f497ea4197/root/var/svc/log/oxide-crucible-pantry:default.log (783 bytes)
14792024-04-26T00:28:00.237Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_pantry_652e3013-976a-43ba-bb68-b6f497ea4197/root/var/svc/log/oxide-crucible-pantry:default.log
14802024-04-26T00:28:00.265Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_pantry_652e3013-976a-43ba-bb68-b6f497ea4197/root/var/svc/log/oxide-zone-network-setup:default.log (1725 bytes)
14812024-04-26T00:28:01.336Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_pantry_652e3013-976a-43ba-bb68-b6f497ea4197/root/var/svc/log/oxide-zone-network-setup:default.log
14822024-04-26T00:28:01.336Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_pantry_82e8a1b2-fbe0-4447-944a-27decf04f25a/root/var/svc/log/oxide-crucible-pantry:default.log (783 bytes)
14832024-04-26T00:28:02.435Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_pantry_82e8a1b2-fbe0-4447-944a-27decf04f25a/root/var/svc/log/oxide-crucible-pantry:default.log
14842024-04-26T00:28:02.435Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_pantry_82e8a1b2-fbe0-4447-944a-27decf04f25a/root/var/svc/log/oxide-zone-network-setup:default.log (1725 bytes)
14852024-04-26T00:28:03.534Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_crucible_pantry_82e8a1b2-fbe0-4447-944a-27decf04f25a/root/var/svc/log/oxide-zone-network-setup:default.log
14862024-04-26T00:28:03.560Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_external_dns_350afff8-3a4b-48f9-b954-9cd28e0a2019/root/var/svc/log/oxide-external_dns:default.log (30636 bytes)
14872024-04-26T00:28:04.701Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_external_dns_350afff8-3a4b-48f9-b954-9cd28e0a2019/root/var/svc/log/oxide-external_dns:default.log
14882024-04-26T00:28:04.701Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_external_dns_350afff8-3a4b-48f9-b954-9cd28e0a2019/root/var/svc/log/oxide-opte-interface-setup:default.log (1159 bytes)
14892024-04-26T00:28:05.803Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_external_dns_350afff8-3a4b-48f9-b954-9cd28e0a2019/root/var/svc/log/oxide-opte-interface-setup:default.log
14902024-04-26T00:28:05.803Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_external_dns_350afff8-3a4b-48f9-b954-9cd28e0a2019/root/var/svc/log/oxide-zone-network-setup:default.log (1705 bytes)
14912024-04-26T00:28:06.899Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_external_dns_350afff8-3a4b-48f9-b954-9cd28e0a2019/root/var/svc/log/oxide-zone-network-setup:default.log
14922024-04-26T00:28:06.900Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_internal_dns_ed9071c1-21f2-4f15-812c-6b6ce9f54d7b/root/var/svc/log/oxide-internal_dns:default.log (111932 bytes)
14932024-04-26T00:28:08.116Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_internal_dns_ed9071c1-21f2-4f15-812c-6b6ce9f54d7b/root/var/svc/log/oxide-internal_dns:default.log
14942024-04-26T00:28:08.117Zuploading: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_internal_dns_ed9071c1-21f2-4f15-812c-6b6ce9f54d7b/root/var/svc/log/oxide-zone-network-setup:default.log (1687 bytes)
14952024-04-26T00:28:09.210Zuploaded: /pool/ext/1962e339-1294-4041-a983-29e64bdb2ffe/crypt/zone/oxz_internal_dns_ed9071c1-21f2-4f15-812c-6b6ce9f54d7b/root/var/svc/log/oxide-zone-network-setup:default.log
14962024-04-26T00:28:09.210Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_crucible_5145b222-0ffb-403b-a919-074d655b57e3/root/var/svc/log/oxide-crucible-agent:default.log (34865 bytes)
14972024-04-26T00:28:10.380Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_crucible_5145b222-0ffb-403b-a919-074d655b57e3/root/var/svc/log/oxide-crucible-agent:default.log
14982024-04-26T00:28:10.380Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_crucible_5145b222-0ffb-403b-a919-074d655b57e3/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
14992024-04-26T00:28:11.479Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_crucible_5145b222-0ffb-403b-a919-074d655b57e3/root/var/svc/log/oxide-crucible-downstairs:default.log
15002024-04-26T00:28:11.479Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_crucible_5145b222-0ffb-403b-a919-074d655b57e3/root/var/svc/log/oxide-crucible-downstairs:downstairs-cd3940ae-6535-43d6-b671-d13b52e33b2c.log (5694 bytes)
15012024-04-26T00:28:12.575Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_crucible_5145b222-0ffb-403b-a919-074d655b57e3/root/var/svc/log/oxide-crucible-downstairs:downstairs-cd3940ae-6535-43d6-b671-d13b52e33b2c.log
15022024-04-26T00:28:12.576Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_crucible_5145b222-0ffb-403b-a919-074d655b57e3/root/var/svc/log/oxide-crucible-downstairs:downstairs-e0d3da98-9672-4c42-81cd-966008aebdb6.log (5612 bytes)
15032024-04-26T00:28:13.674Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_crucible_5145b222-0ffb-403b-a919-074d655b57e3/root/var/svc/log/oxide-crucible-downstairs:downstairs-e0d3da98-9672-4c42-81cd-966008aebdb6.log
15042024-04-26T00:28:13.699Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_crucible_5145b222-0ffb-403b-a919-074d655b57e3/root/var/svc/log/oxide-crucible-downstairs:snapshot-cd3940ae-6535-43d6-b671-d13b52e33b2c-8a8cb26d-a12c-47cb-a73a-5f6e4573a290.log (5306 bytes)
15052024-04-26T00:28:14.771Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_crucible_5145b222-0ffb-403b-a919-074d655b57e3/root/var/svc/log/oxide-crucible-downstairs:snapshot-cd3940ae-6535-43d6-b671-d13b52e33b2c-8a8cb26d-a12c-47cb-a73a-5f6e4573a290.log
15062024-04-26T00:28:14.796Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_crucible_5145b222-0ffb-403b-a919-074d655b57e3/root/var/svc/log/oxide-zone-network-setup:default.log (1682 bytes)
15072024-04-26T00:28:15.868Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_crucible_5145b222-0ffb-403b-a919-074d655b57e3/root/var/svc/log/oxide-zone-network-setup:default.log
15082024-04-26T00:28:15.868Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/svc/log/oxide-chrony-setup:default.log (13384 bytes)
15092024-04-26T00:28:16.989Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/svc/log/oxide-chrony-setup:default.log
15102024-04-26T00:28:16.989Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/svc/log/oxide-ntp:default.log (863 bytes)
15112024-04-26T00:28:18.087Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/svc/log/oxide-ntp:default.log
15122024-04-26T00:28:18.112Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/svc/log/oxide-opte-interface-setup:default.log (1128 bytes)
15132024-04-26T00:28:19.184Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/svc/log/oxide-opte-interface-setup:default.log
15142024-04-26T00:28:19.213Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/svc/log/oxide-zone-network-setup:default.log (1644 bytes)
15152024-04-26T00:28:20.286Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/svc/log/oxide-zone-network-setup:default.log
15162024-04-26T00:28:20.286Zuploading: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_cockroachdb_8cc9856d-c4e4-49e3-9237-1d97f538eae8/root/var/svc/log/oxide-cockroachdb:default.log (4407 bytes)
15172024-04-26T00:28:21.384Zuploaded: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_cockroachdb_8cc9856d-c4e4-49e3-9237-1d97f538eae8/root/var/svc/log/oxide-cockroachdb:default.log
15182024-04-26T00:28:21.384Zuploading: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_cockroachdb_8cc9856d-c4e4-49e3-9237-1d97f538eae8/root/var/svc/log/oxide-zone-network-setup:default.log (1690 bytes)
15192024-04-26T00:28:22.486Zuploaded: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_cockroachdb_8cc9856d-c4e4-49e3-9237-1d97f538eae8/root/var/svc/log/oxide-zone-network-setup:default.log
15202024-04-26T00:28:22.486Zuploading: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_crucible_c8b70fc0-b959-4973-889c-bcdff32c973f/root/var/svc/log/oxide-crucible-agent:default.log (24508 bytes)
15212024-04-26T00:28:23.628Zuploaded: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_crucible_c8b70fc0-b959-4973-889c-bcdff32c973f/root/var/svc/log/oxide-crucible-agent:default.log
15222024-04-26T00:28:23.653Zuploading: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_crucible_c8b70fc0-b959-4973-889c-bcdff32c973f/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
15232024-04-26T00:28:24.728Zuploaded: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_crucible_c8b70fc0-b959-4973-889c-bcdff32c973f/root/var/svc/log/oxide-crucible-downstairs:default.log
15242024-04-26T00:28:24.728Zuploading: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_crucible_c8b70fc0-b959-4973-889c-bcdff32c973f/root/var/svc/log/oxide-crucible-downstairs:downstairs-d2caec01-ace5-445d-bdf9-5d296c010f04.log (5612 bytes)
15252024-04-26T00:28:25.862Zuploaded: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_crucible_c8b70fc0-b959-4973-889c-bcdff32c973f/root/var/svc/log/oxide-crucible-downstairs:downstairs-d2caec01-ace5-445d-bdf9-5d296c010f04.log
15262024-04-26T00:28:25.862Zuploading: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_crucible_c8b70fc0-b959-4973-889c-bcdff32c973f/root/var/svc/log/oxide-crucible-downstairs:downstairs-e17dcc01-4337-4324-8e3b-a447b422d21c.log (3019 bytes)
15272024-04-26T00:28:27.506Zuploaded: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_crucible_c8b70fc0-b959-4973-889c-bcdff32c973f/root/var/svc/log/oxide-crucible-downstairs:downstairs-e17dcc01-4337-4324-8e3b-a447b422d21c.log
15282024-04-26T00:28:27.507Zuploading: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_crucible_c8b70fc0-b959-4973-889c-bcdff32c973f/root/var/svc/log/oxide-zone-network-setup:default.log (1683 bytes)
15292024-04-26T00:28:28.854Zuploaded: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_crucible_c8b70fc0-b959-4973-889c-bcdff32c973f/root/var/svc/log/oxide-zone-network-setup:default.log
15302024-04-26T00:28:28.854Zuploading: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_internal_dns_2459ecd0-1d68-43a3-8a7a-97b992a6728f/root/var/svc/log/oxide-internal_dns:default.log (111119 bytes)
15312024-04-26T00:28:29.067Zupload warning: file "/pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_internal_dns_2459ecd0-1d68-43a3-8a7a-97b992a6728f/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 111119 -> 114874
15322024-04-26T00:28:30.152Zuploaded: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_internal_dns_2459ecd0-1d68-43a3-8a7a-97b992a6728f/root/var/svc/log/oxide-internal_dns:default.log
15332024-04-26T00:28:30.152Zuploading: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_internal_dns_2459ecd0-1d68-43a3-8a7a-97b992a6728f/root/var/svc/log/oxide-zone-network-setup:default.log (1688 bytes)
15342024-04-26T00:28:31.264Zuploaded: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_internal_dns_2459ecd0-1d68-43a3-8a7a-97b992a6728f/root/var/svc/log/oxide-zone-network-setup:default.log
15352024-04-26T00:28:31.264Zuploading: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_oximeter_ea63e896-0dc6-4da7-b3d5-27de7463f89a/root/var/svc/log/oxide-oximeter:default.log (1750404 bytes)
15362024-04-26T00:28:31.634Zupload warning: file "/pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_oximeter_ea63e896-0dc6-4da7-b3d5-27de7463f89a/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 1750404 -> 1995304
15372024-04-26T00:28:33.086Zuploaded: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_oximeter_ea63e896-0dc6-4da7-b3d5-27de7463f89a/root/var/svc/log/oxide-oximeter:default.log
15382024-04-26T00:28:33.087Zuploading: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_oximeter_ea63e896-0dc6-4da7-b3d5-27de7463f89a/root/var/svc/log/oxide-zone-network-setup:default.log (1680 bytes)
15392024-04-26T00:28:34.562Zuploaded: /pool/ext/29f0e314-9945-472b-979d-b91f7fcb5170/crypt/zone/oxz_oximeter_ea63e896-0dc6-4da7-b3d5-27de7463f89a/root/var/svc/log/oxide-zone-network-setup:default.log
15402024-04-26T00:28:34.562Zuploading: /pool/ext/42895196-0869-4ea3-9c4e-285bbcea1a80/crypt/zone/oxz_crucible_d78f8b7a-b7a9-41c5-83d8-2b2ed0991813/root/var/svc/log/oxide-crucible-agent:default.log (1555 bytes)
15412024-04-26T00:28:35.659Zuploaded: /pool/ext/42895196-0869-4ea3-9c4e-285bbcea1a80/crypt/zone/oxz_crucible_d78f8b7a-b7a9-41c5-83d8-2b2ed0991813/root/var/svc/log/oxide-crucible-agent:default.log
15422024-04-26T00:28:35.659Zuploading: /pool/ext/42895196-0869-4ea3-9c4e-285bbcea1a80/crypt/zone/oxz_crucible_d78f8b7a-b7a9-41c5-83d8-2b2ed0991813/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
15432024-04-26T00:28:36.760Zuploaded: /pool/ext/42895196-0869-4ea3-9c4e-285bbcea1a80/crypt/zone/oxz_crucible_d78f8b7a-b7a9-41c5-83d8-2b2ed0991813/root/var/svc/log/oxide-crucible-downstairs:default.log
15442024-04-26T00:28:36.761Zuploading: /pool/ext/42895196-0869-4ea3-9c4e-285bbcea1a80/crypt/zone/oxz_crucible_d78f8b7a-b7a9-41c5-83d8-2b2ed0991813/root/var/svc/log/oxide-zone-network-setup:default.log (1681 bytes)
15452024-04-26T00:28:37.860Zuploaded: /pool/ext/42895196-0869-4ea3-9c4e-285bbcea1a80/crypt/zone/oxz_crucible_d78f8b7a-b7a9-41c5-83d8-2b2ed0991813/root/var/svc/log/oxide-zone-network-setup:default.log
15462024-04-26T00:28:37.860Zuploading: /pool/ext/73bca2cf-2bf8-443a-bf93-681a0fe6b326/crypt/zone/oxz_crucible_43c9638c-6930-4233-a47d-c58f94e587c5/root/var/svc/log/oxide-crucible-agent:default.log (18844 bytes)
15472024-04-26T00:28:39.088Zuploaded: /pool/ext/73bca2cf-2bf8-443a-bf93-681a0fe6b326/crypt/zone/oxz_crucible_43c9638c-6930-4233-a47d-c58f94e587c5/root/var/svc/log/oxide-crucible-agent:default.log
15482024-04-26T00:28:39.088Zuploading: /pool/ext/73bca2cf-2bf8-443a-bf93-681a0fe6b326/crypt/zone/oxz_crucible_43c9638c-6930-4233-a47d-c58f94e587c5/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
15492024-04-26T00:28:40.694Zuploaded: /pool/ext/73bca2cf-2bf8-443a-bf93-681a0fe6b326/crypt/zone/oxz_crucible_43c9638c-6930-4233-a47d-c58f94e587c5/root/var/svc/log/oxide-crucible-downstairs:default.log
15502024-04-26T00:28:40.694Zuploading: /pool/ext/73bca2cf-2bf8-443a-bf93-681a0fe6b326/crypt/zone/oxz_crucible_43c9638c-6930-4233-a47d-c58f94e587c5/root/var/svc/log/oxide-crucible-downstairs:downstairs-b77a18b4-2d7a-440b-af34-14c007849a99.log (5695 bytes)
15512024-04-26T00:28:42.288Zuploaded: /pool/ext/73bca2cf-2bf8-443a-bf93-681a0fe6b326/crypt/zone/oxz_crucible_43c9638c-6930-4233-a47d-c58f94e587c5/root/var/svc/log/oxide-crucible-downstairs:downstairs-b77a18b4-2d7a-440b-af34-14c007849a99.log
15522024-04-26T00:28:42.288Zuploading: /pool/ext/73bca2cf-2bf8-443a-bf93-681a0fe6b326/crypt/zone/oxz_crucible_43c9638c-6930-4233-a47d-c58f94e587c5/root/var/svc/log/oxide-crucible-downstairs:snapshot-b77a18b4-2d7a-440b-af34-14c007849a99-8a8cb26d-a12c-47cb-a73a-5f6e4573a290.log (5306 bytes)
15532024-04-26T00:28:43.995Zuploaded: /pool/ext/73bca2cf-2bf8-443a-bf93-681a0fe6b326/crypt/zone/oxz_crucible_43c9638c-6930-4233-a47d-c58f94e587c5/root/var/svc/log/oxide-crucible-downstairs:snapshot-b77a18b4-2d7a-440b-af34-14c007849a99-8a8cb26d-a12c-47cb-a73a-5f6e4573a290.log
15542024-04-26T00:28:44.021Zuploading: /pool/ext/73bca2cf-2bf8-443a-bf93-681a0fe6b326/crypt/zone/oxz_crucible_43c9638c-6930-4233-a47d-c58f94e587c5/root/var/svc/log/oxide-zone-network-setup:default.log (1683 bytes)
15552024-04-26T00:28:45.098Zuploaded: /pool/ext/73bca2cf-2bf8-443a-bf93-681a0fe6b326/crypt/zone/oxz_crucible_43c9638c-6930-4233-a47d-c58f94e587c5/root/var/svc/log/oxide-zone-network-setup:default.log
15562024-04-26T00:28:45.124Zuploading: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_cockroachdb_e9e24743-a540-4b93-9133-4eae210ed8f3/root/var/svc/log/oxide-cockroachdb:default.log (4406 bytes)
15572024-04-26T00:28:46.349Zuploaded: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_cockroachdb_e9e24743-a540-4b93-9133-4eae210ed8f3/root/var/svc/log/oxide-cockroachdb:default.log
15582024-04-26T00:28:46.350Zuploading: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_cockroachdb_e9e24743-a540-4b93-9133-4eae210ed8f3/root/var/svc/log/oxide-zone-network-setup:default.log (1690 bytes)
15592024-04-26T00:28:47.864Zuploaded: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_cockroachdb_e9e24743-a540-4b93-9133-4eae210ed8f3/root/var/svc/log/oxide-zone-network-setup:default.log
15602024-04-26T00:28:47.864Zuploading: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_crucible_34579d2e-b8f0-4cf3-bcf0-ba9d8c19e7fc/root/var/svc/log/oxide-crucible-agent:default.log (23192 bytes)
15612024-04-26T00:28:49.127Zuploaded: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_crucible_34579d2e-b8f0-4cf3-bcf0-ba9d8c19e7fc/root/var/svc/log/oxide-crucible-agent:default.log
15622024-04-26T00:28:49.127Zuploading: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_crucible_34579d2e-b8f0-4cf3-bcf0-ba9d8c19e7fc/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
15632024-04-26T00:28:50.227Zuploaded: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_crucible_34579d2e-b8f0-4cf3-bcf0-ba9d8c19e7fc/root/var/svc/log/oxide-crucible-downstairs:default.log
15642024-04-26T00:28:50.228Zuploading: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_crucible_34579d2e-b8f0-4cf3-bcf0-ba9d8c19e7fc/root/var/svc/log/oxide-crucible-downstairs:downstairs-086b81fc-7eec-4807-ae6d-1ba23fdb350a.log (5612 bytes)
15652024-04-26T00:28:51.322Zuploaded: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_crucible_34579d2e-b8f0-4cf3-bcf0-ba9d8c19e7fc/root/var/svc/log/oxide-crucible-downstairs:downstairs-086b81fc-7eec-4807-ae6d-1ba23fdb350a.log
15662024-04-26T00:28:51.323Zuploading: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_crucible_34579d2e-b8f0-4cf3-bcf0-ba9d8c19e7fc/root/var/svc/log/oxide-crucible-downstairs:downstairs-9f5676f9-ad5c-4b50-b56d-2946270a001d.log (3122 bytes)
15672024-04-26T00:28:52.583Zuploaded: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_crucible_34579d2e-b8f0-4cf3-bcf0-ba9d8c19e7fc/root/var/svc/log/oxide-crucible-downstairs:downstairs-9f5676f9-ad5c-4b50-b56d-2946270a001d.log
15682024-04-26T00:28:52.628Zuploading: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_crucible_34579d2e-b8f0-4cf3-bcf0-ba9d8c19e7fc/root/var/svc/log/oxide-zone-network-setup:default.log (1682 bytes)
15692024-04-26T00:28:53.763Zuploaded: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_crucible_34579d2e-b8f0-4cf3-bcf0-ba9d8c19e7fc/root/var/svc/log/oxide-zone-network-setup:default.log
15702024-04-26T00:28:53.763Zuploading: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_nexus_2d696101-87c5-4c96-b623-2e415a336b1d/root/var/svc/log/oxide-nexus:default.log (3183157 bytes)
15712024-04-26T00:28:55.009Zupload warning: file "/pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_nexus_2d696101-87c5-4c96-b623-2e415a336b1d/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 3183157 -> 3476421
15722024-04-26T00:28:56.082Zuploaded: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_nexus_2d696101-87c5-4c96-b623-2e415a336b1d/root/var/svc/log/oxide-nexus:default.log
15732024-04-26T00:28:56.083Zuploading: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_nexus_2d696101-87c5-4c96-b623-2e415a336b1d/root/var/svc/log/oxide-opte-interface-setup:default.log (1138 bytes)
15742024-04-26T00:28:57.186Zuploaded: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_nexus_2d696101-87c5-4c96-b623-2e415a336b1d/root/var/svc/log/oxide-opte-interface-setup:default.log
15752024-04-26T00:28:57.186Zuploading: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_nexus_2d696101-87c5-4c96-b623-2e415a336b1d/root/var/svc/log/oxide-zone-network-setup:default.log (1663 bytes)
15762024-04-26T00:28:58.284Zuploaded: /pool/ext/b4f1d1aa-0185-4f19-9f65-a972831259c3/crypt/zone/oxz_nexus_2d696101-87c5-4c96-b623-2e415a336b1d/root/var/svc/log/oxide-zone-network-setup:default.log
15772024-04-26T00:28:58.284Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_clickhouse_ff8af3cb-0e96-47e9-b05c-afa245f1dff5/root/var/svc/log/oxide-clickhouse:default.log (1974966 bytes)
15782024-04-26T00:28:59.475Zupload warning: file "/pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_clickhouse_ff8af3cb-0e96-47e9-b05c-afa245f1dff5/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 1974966 -> 2324630
15792024-04-26T00:29:00.547Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_clickhouse_ff8af3cb-0e96-47e9-b05c-afa245f1dff5/root/var/svc/log/oxide-clickhouse:default.log
15802024-04-26T00:29:00.547Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_clickhouse_ff8af3cb-0e96-47e9-b05c-afa245f1dff5/root/var/svc/log/oxide-zone-network-setup:default.log (1692 bytes)
15812024-04-26T00:29:01.647Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_clickhouse_ff8af3cb-0e96-47e9-b05c-afa245f1dff5/root/var/svc/log/oxide-zone-network-setup:default.log
15822024-04-26T00:29:01.647Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_cockroachdb_5e3ff898-c77b-42a1-9764-187791149401/root/var/svc/log/oxide-cockroachdb:default.log (4380 bytes)
15832024-04-26T00:29:02.749Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_cockroachdb_5e3ff898-c77b-42a1-9764-187791149401/root/var/svc/log/oxide-cockroachdb:default.log
15842024-04-26T00:29:02.749Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_cockroachdb_5e3ff898-c77b-42a1-9764-187791149401/root/var/svc/log/oxide-zone-network-setup:default.log (1690 bytes)
15852024-04-26T00:29:03.851Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_cockroachdb_5e3ff898-c77b-42a1-9764-187791149401/root/var/svc/log/oxide-zone-network-setup:default.log
15862024-04-26T00:29:03.851Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_crucible_4bd02937-0d24-4783-b773-973d42c4eef8/root/var/svc/log/oxide-crucible-agent:default.log (20195 bytes)
15872024-04-26T00:29:04.996Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_crucible_4bd02937-0d24-4783-b773-973d42c4eef8/root/var/svc/log/oxide-crucible-agent:default.log
15882024-04-26T00:29:04.996Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_crucible_4bd02937-0d24-4783-b773-973d42c4eef8/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
15892024-04-26T00:29:06.118Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_crucible_4bd02937-0d24-4783-b773-973d42c4eef8/root/var/svc/log/oxide-crucible-downstairs:default.log
15902024-04-26T00:29:06.118Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_crucible_4bd02937-0d24-4783-b773-973d42c4eef8/root/var/svc/log/oxide-crucible-downstairs:downstairs-e1b2e74d-8fa9-432e-b95d-6fd854f6cf1c.log (3124 bytes)
15912024-04-26T00:29:07.195Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_crucible_4bd02937-0d24-4783-b773-973d42c4eef8/root/var/svc/log/oxide-crucible-downstairs:downstairs-e1b2e74d-8fa9-432e-b95d-6fd854f6cf1c.log
15922024-04-26T00:29:07.195Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_crucible_4bd02937-0d24-4783-b773-973d42c4eef8/root/var/svc/log/oxide-crucible-downstairs:downstairs-f2cd00e7-8d1a-483c-870b-8f5ae88bd72b.log (3020 bytes)
15932024-04-26T00:29:08.292Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_crucible_4bd02937-0d24-4783-b773-973d42c4eef8/root/var/svc/log/oxide-crucible-downstairs:downstairs-f2cd00e7-8d1a-483c-870b-8f5ae88bd72b.log
15942024-04-26T00:29:08.292Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_crucible_4bd02937-0d24-4783-b773-973d42c4eef8/root/var/svc/log/oxide-zone-network-setup:default.log (1683 bytes)
15952024-04-26T00:29:09.389Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_crucible_4bd02937-0d24-4783-b773-973d42c4eef8/root/var/svc/log/oxide-zone-network-setup:default.log
15962024-04-26T00:29:09.389Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_external_dns_41ef6d1d-d04a-4944-8844-640cc5b2f887/root/var/svc/log/oxide-external_dns:default.log (30644 bytes)
15972024-04-26T00:29:09.505Zupload warning: file "/pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_external_dns_41ef6d1d-d04a-4944-8844-640cc5b2f887/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 30644 -> 34419
15982024-04-26T00:29:10.578Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_external_dns_41ef6d1d-d04a-4944-8844-640cc5b2f887/root/var/svc/log/oxide-external_dns:default.log
15992024-04-26T00:29:10.578Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_external_dns_41ef6d1d-d04a-4944-8844-640cc5b2f887/root/var/svc/log/oxide-opte-interface-setup:default.log (1159 bytes)
16002024-04-26T00:29:11.674Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_external_dns_41ef6d1d-d04a-4944-8844-640cc5b2f887/root/var/svc/log/oxide-opte-interface-setup:default.log
16012024-04-26T00:29:11.674Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_external_dns_41ef6d1d-d04a-4944-8844-640cc5b2f887/root/var/svc/log/oxide-zone-network-setup:default.log (1705 bytes)
16022024-04-26T00:29:12.772Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_external_dns_41ef6d1d-d04a-4944-8844-640cc5b2f887/root/var/svc/log/oxide-zone-network-setup:default.log
16032024-04-26T00:29:12.772Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_internal_dns_dcbedba4-6a83-41c6-9a2a-514fb195d796/root/var/svc/log/oxide-internal_dns:default.log (111125 bytes)
16042024-04-26T00:29:12.935Zupload warning: file "/pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_internal_dns_dcbedba4-6a83-41c6-9a2a-514fb195d796/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 111125 -> 114879
16052024-04-26T00:29:14.008Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_internal_dns_dcbedba4-6a83-41c6-9a2a-514fb195d796/root/var/svc/log/oxide-internal_dns:default.log
16062024-04-26T00:29:14.008Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_internal_dns_dcbedba4-6a83-41c6-9a2a-514fb195d796/root/var/svc/log/oxide-zone-network-setup:default.log (1688 bytes)
16072024-04-26T00:29:15.106Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_internal_dns_dcbedba4-6a83-41c6-9a2a-514fb195d796/root/var/svc/log/oxide-zone-network-setup:default.log
16082024-04-26T00:29:15.107Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_a1c22271-87f5-44b0-b506-06dff6539283/root/var/svc/log/oxide-nexus:default.log (36502438 bytes)
16092024-04-26T00:29:25.615Zupload warning: file "/pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_a1c22271-87f5-44b0-b506-06dff6539283/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 36502438 -> 37144948
16102024-04-26T00:29:26.687Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_a1c22271-87f5-44b0-b506-06dff6539283/root/var/svc/log/oxide-nexus:default.log
16112024-04-26T00:29:26.687Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_a1c22271-87f5-44b0-b506-06dff6539283/root/var/svc/log/oxide-opte-interface-setup:default.log (1138 bytes)
16122024-04-26T00:29:27.784Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_a1c22271-87f5-44b0-b506-06dff6539283/root/var/svc/log/oxide-opte-interface-setup:default.log
16132024-04-26T00:29:27.784Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_a1c22271-87f5-44b0-b506-06dff6539283/root/var/svc/log/oxide-zone-network-setup:default.log (1663 bytes)
16142024-04-26T00:29:28.884Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_a1c22271-87f5-44b0-b506-06dff6539283/root/var/svc/log/oxide-zone-network-setup:default.log
16152024-04-26T00:29:28.884Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_bb2cdb6e-aac0-4124-a13e-9c3e19120ba0/root/var/svc/log/oxide-nexus:default.log (2024912 bytes)
16162024-04-26T00:29:29.651Zupload warning: file "/pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_bb2cdb6e-aac0-4124-a13e-9c3e19120ba0/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 2024912 -> 2455410
16172024-04-26T00:29:30.724Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_bb2cdb6e-aac0-4124-a13e-9c3e19120ba0/root/var/svc/log/oxide-nexus:default.log
16182024-04-26T00:29:30.725Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_bb2cdb6e-aac0-4124-a13e-9c3e19120ba0/root/var/svc/log/oxide-opte-interface-setup:default.log (1138 bytes)
16192024-04-26T00:29:31.823Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_bb2cdb6e-aac0-4124-a13e-9c3e19120ba0/root/var/svc/log/oxide-opte-interface-setup:default.log
16202024-04-26T00:29:31.823Zuploading: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_bb2cdb6e-aac0-4124-a13e-9c3e19120ba0/root/var/svc/log/oxide-zone-network-setup:default.log (1661 bytes)
16212024-04-26T00:29:32.920Zuploaded: /pool/ext/d264dd09-dd1e-4b44-ac20-70d4e8728824/crypt/zone/oxz_nexus_bb2cdb6e-aac0-4124-a13e-9c3e19120ba0/root/var/svc/log/oxide-zone-network-setup:default.log
16222024-04-26T00:29:32.921Zuploading: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_cockroachdb_6a642efd-fc7d-4434-bb44-b7f02c3b3137/root/var/svc/log/oxide-cockroachdb:default.log (4407 bytes)
16232024-04-26T00:29:34.024Zuploaded: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_cockroachdb_6a642efd-fc7d-4434-bb44-b7f02c3b3137/root/var/svc/log/oxide-cockroachdb:default.log
16242024-04-26T00:29:34.024Zuploading: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_cockroachdb_6a642efd-fc7d-4434-bb44-b7f02c3b3137/root/var/svc/log/oxide-zone-network-setup:default.log (1689 bytes)
16252024-04-26T00:29:35.122Zuploaded: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_cockroachdb_6a642efd-fc7d-4434-bb44-b7f02c3b3137/root/var/svc/log/oxide-zone-network-setup:default.log
16262024-04-26T00:29:35.122Zuploading: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_crucible_14af2360-4bb4-4f0c-88c4-c36ebd6f02c7/root/var/svc/log/oxide-crucible-agent:default.log (12375 bytes)
16272024-04-26T00:29:36.246Zuploaded: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_crucible_14af2360-4bb4-4f0c-88c4-c36ebd6f02c7/root/var/svc/log/oxide-crucible-agent:default.log
16282024-04-26T00:29:36.246Zuploading: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_crucible_14af2360-4bb4-4f0c-88c4-c36ebd6f02c7/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
16292024-04-26T00:29:37.343Zuploaded: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_crucible_14af2360-4bb4-4f0c-88c4-c36ebd6f02c7/root/var/svc/log/oxide-crucible-downstairs:default.log
16302024-04-26T00:29:37.343Zuploading: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_crucible_14af2360-4bb4-4f0c-88c4-c36ebd6f02c7/root/var/svc/log/oxide-crucible-downstairs:downstairs-9a563efc-4a7f-4f5e-8f9c-42562686239d.log (3126 bytes)
16312024-04-26T00:29:38.441Zuploaded: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_crucible_14af2360-4bb4-4f0c-88c4-c36ebd6f02c7/root/var/svc/log/oxide-crucible-downstairs:downstairs-9a563efc-4a7f-4f5e-8f9c-42562686239d.log
16322024-04-26T00:29:38.441Zuploading: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_crucible_14af2360-4bb4-4f0c-88c4-c36ebd6f02c7/root/var/svc/log/oxide-zone-network-setup:default.log (1682 bytes)
16332024-04-26T00:29:39.540Zuploaded: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_crucible_14af2360-4bb4-4f0c-88c4-c36ebd6f02c7/root/var/svc/log/oxide-zone-network-setup:default.log
16342024-04-26T00:29:39.566Zuploading: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_crucible_pantry_4dfbb131-ca76-448d-8328-3ae1e5fa72b9/root/var/svc/log/oxide-crucible-pantry:default.log (1223763 bytes)
16352024-04-26T00:29:41.322Zuploaded: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_crucible_pantry_4dfbb131-ca76-448d-8328-3ae1e5fa72b9/root/var/svc/log/oxide-crucible-pantry:default.log
16362024-04-26T00:29:41.322Zuploading: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_crucible_pantry_4dfbb131-ca76-448d-8328-3ae1e5fa72b9/root/var/svc/log/oxide-zone-network-setup:default.log (1722 bytes)
16372024-04-26T00:29:42.419Zuploaded: /pool/ext/fa79b281-89d1-42f4-b871-62a5aaaf9922/crypt/zone/oxz_crucible_pantry_4dfbb131-ca76-448d-8328-3ae1e5fa72b9/root/var/svc/log/oxide-zone-network-setup:default.log
16382024-04-26T00:29:42.444Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/log/chrony/measurements.log (23564 bytes)
16392024-04-26T00:29:42.525Zupload warning: file "/pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/log/chrony/measurements.log" changed size mid upload: 23564 -> 28085
16402024-04-26T00:29:43.597Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/log/chrony/measurements.log
16412024-04-26T00:29:43.622Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/log/chrony/statistics.log (15912 bytes)
16422024-04-26T00:29:43.691Zupload warning: file "/pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/log/chrony/statistics.log" changed size mid upload: 15912 -> 19773
16432024-04-26T00:29:44.764Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/log/chrony/statistics.log
16442024-04-26T00:29:44.788Zuploading: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/log/chrony/tracking.log (1986 bytes)
16452024-04-26T00:29:44.813Zupload warning: file "/pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/log/chrony/tracking.log" changed size mid upload: 1986 -> 2382
16462024-04-26T00:29:45.885Zuploaded: /pool/ext/1af63a9d-d9ad-43e0-9ba8-d7354e1b6a8f/crypt/zone/oxz_ntp_0755d222-4cb0-4d69-ba56-feb7502debc0/root/var/log/chrony/tracking.log