01HSXZSY4PE9J5PANYXM704JXK: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log*",
        "%/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.28",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01HSXZTGV7SQPHE1ZKZ2SHEXC6

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12024-03-26T19:02:11.315Zjob dependencies complete; ready to run (waiting for 42 m 32 s)
22024-03-26T19:32:33.912Zjob assigned to worker 01HSY3RV1SVK3NMFV5FYNQ9XTT (queued for 30 m 22 s)
32024-03-26T19:32:33.983Zhttp://172.20.0.5:9969/ipxe/sock... ok
42024-03-26T19:32:34.030Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/kernel/amd64/unix... ok
52024-03-26T19:32:34.069Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/amd64/boot_archive... ok
62024-03-26T19:32:34.649Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/amd64/boot_archive.hash... ok
72024-03-26T19:32:34.690Zhttp://172.20.0.5:9969/postboot/sock... ok
82024-03-26T19:32:34.728ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
92024-03-26T19:32:34.774Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d1
102024-03-26T19:32:34.819ZOxide Helios Version helios-2.0.22550 64-bit
112024-03-26T19:32:34.869ZConfiguring devices.
122024-03-26T19:32:34.946Z##################################################
132024-03-26T19:32:35.005Z#### ############# ##
142024-03-26T19:32:35.050Z### ### ############ ##
152024-03-26T19:32:35.089Z## ### # ## ### ## Oxide Computer Company ##
162024-03-26T19:32:35.127Z## ## ## ### # ### ##
172024-03-26T19:32:35.173Z## # ### #### #### This Station Under ##
182024-03-26T19:32:35.217Z### ### #### # ### Computer Control ##
192024-03-26T19:32:35.270Z#### #### ### ## ##
202024-03-26T19:32:35.317Z##################################################
212024-03-26T19:32:35.380ZHostname: sock
222024-03-26T19:32:35.439ZLoading smf(7) service descriptions:
232024-03-26T19:32:36.717Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap-3cb81999e7440661.gz
242024-03-26T19:32:46.072Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap-3cb81999e7440661.gz
252024-03-26T19:32:46.103Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
262024-03-26T19:33:30.601Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
272024-03-26T19:33:30.602Zdownloading input: /input/ci-tools/work/end-to-end-tests/commtest-e25b00f6d79ff279.gz
282024-03-26T19:33:32.041Zdownloaded input: /input/ci-tools/work/end-to-end-tests/commtest-e25b00f6d79ff279.gz
292024-03-26T19:33:32.067Zdownloading input: /input/ci-tools/work/end-to-end-tests/dhcp_server-cdb0b653f05d35f0.gz
302024-03-26T19:33:34.529Zdownloaded input: /input/ci-tools/work/end-to-end-tests/dhcp_server-cdb0b653f05d35f0.gz
312024-03-26T19:33:34.557Zdownloading input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-92ca3720317208b2.gz
322024-03-26T19:34:06.519Zdownloaded input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-92ca3720317208b2.gz
332024-03-26T19:34:06.519Zdownloading input: /input/ci-tools/work/caboose-util.gz
342024-03-26T19:34:07.359Zdownloaded input: /input/ci-tools/work/caboose-util.gz
352024-03-26T19:34:07.359Zdownloading input: /input/ci-tools/work/tufaceous.gz
362024-03-26T19:34:10.075Zdownloaded input: /input/ci-tools/work/tufaceous.gz
372024-03-26T19:34:10.075Zdownloading input: /input/ci-tools/work/commtest
382024-03-26T19:35:38.408Zdownloaded input: /input/ci-tools/work/commtest
392024-03-26T19:35:38.434Zdownloading input: /input/package/work/version.txt
402024-03-26T19:35:38.478Zdownloaded input: /input/package/work/version.txt
412024-03-26T19:35:38.507Zdownloading input: /input/package/work/package.tar.gz
422024-03-26T19:35:54.388Zdownloaded input: /input/package/work/package.tar.gz
432024-03-26T19:35:54.419Zdownloading input: /input/package/work/global-zone-packages.tar.gz
442024-03-26T19:36:27.065Zdownloaded input: /input/package/work/global-zone-packages.tar.gz
452024-03-26T19:36:27.065Zdownloading input: /input/package/work/trampoline-global-zone-packages.tar.gz
462024-03-26T19:36:34.664Zdownloaded input: /input/package/work/trampoline-global-zone-packages.tar.gz
472024-03-26T19:36:34.705Zdownloading input: /input/package/work/zones/clickhouse.tar.gz
482024-03-26T19:37:28.051Zdownloaded input: /input/package/work/zones/clickhouse.tar.gz
492024-03-26T19:37:28.081Zdownloading input: /input/package/work/zones/clickhouse_keeper.tar.gz
502024-03-26T19:38:44.616Zdownloaded input: /input/package/work/zones/clickhouse_keeper.tar.gz
512024-03-26T19:38:44.643Zdownloading input: /input/package/work/zones/cockroachdb.tar.gz
522024-03-26T19:39:19.292Zdownloaded input: /input/package/work/zones/cockroachdb.tar.gz
532024-03-26T19:39:19.292Zdownloading input: /input/package/work/zones/crucible-pantry-zone.tar.gz
542024-03-26T19:39:26.898Zdownloaded input: /input/package/work/zones/crucible-pantry-zone.tar.gz
552024-03-26T19:39:26.932Zdownloading input: /input/package/work/zones/crucible-zone.tar.gz
562024-03-26T19:39:42.757Zdownloaded input: /input/package/work/zones/crucible-zone.tar.gz
572024-03-26T19:39:42.818Zdownloading input: /input/package/work/zones/external-dns.tar.gz
582024-03-26T19:39:49.789Zdownloaded input: /input/package/work/zones/external-dns.tar.gz
592024-03-26T19:39:49.789Zdownloading input: /input/package/work/zones/internal-dns.tar.gz
602024-03-26T19:39:56.571Zdownloaded input: /input/package/work/zones/internal-dns.tar.gz
612024-03-26T19:39:56.595Zdownloading input: /input/package/work/zones/nexus-single-sled.tar.gz
622024-03-26T19:40:18.707Zdownloaded input: /input/package/work/zones/nexus-single-sled.tar.gz
632024-03-26T19:40:18.733Zdownloading input: /input/package/work/zones/nexus.tar.gz
642024-03-26T19:40:45.272Zdownloaded input: /input/package/work/zones/nexus.tar.gz
652024-03-26T19:40:45.272Zdownloading input: /input/package/work/zones/ntp.tar.gz
662024-03-26T19:40:49.503Zdownloaded input: /input/package/work/zones/ntp.tar.gz
672024-03-26T19:40:49.504Zdownloading input: /input/package/work/zones/omicron-gateway-asic.tar.gz
682024-03-26T19:40:56.393Zdownloaded input: /input/package/work/zones/omicron-gateway-asic.tar.gz
692024-03-26T19:40:56.421Zdownloading input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
702024-03-26T19:41:01.846Zdownloaded input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
712024-03-26T19:41:01.877Zdownloading input: /input/package/work/zones/overlay.tar.gz
722024-03-26T19:41:01.911Zdownloaded input: /input/package/work/zones/overlay.tar.gz
732024-03-26T19:41:01.912Zdownloading input: /input/package/work/zones/oximeter.tar.gz
742024-03-26T19:41:18.195Zdownloaded input: /input/package/work/zones/oximeter.tar.gz
752024-03-26T19:41:18.195Zdownloading input: /input/package/work/zones/probe.tar.gz
762024-03-26T19:41:19.295Zdownloaded input: /input/package/work/zones/probe.tar.gz
772024-03-26T19:41:19.295Zdownloading input: /input/package/work/zones/propolis-server.tar.gz
782024-03-26T19:41:38.744Zdownloaded input: /input/package/work/zones/propolis-server.tar.gz
792024-03-26T19:41:38.745Zdownloading input: /input/package/work/zones/switch-asic.tar.gz
802024-03-26T19:42:55.239Zdownloaded input: /input/package/work/zones/switch-asic.tar.gz
812024-03-26T19:42:55.713Zdownloading input: /input/package/work/zones/switch-softnpu.tar.gz
822024-03-26T19:43:48.772Zdownloaded input: /input/package/work/zones/switch-softnpu.tar.gz
 
832024-03-26T19:43:48.801Zstarting task 0: "setup"
842024-03-26T19:43:48.801Z++ uname -s
852024-03-26T19:43:48.801Z+ kern=SunOS
862024-03-26T19:43:48.802Z+ case "$kern" in
872024-03-26T19:43:48.802Z+ groupadd -g 12345 build
882024-03-26T19:43:48.802Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
892024-03-26T19:43:50.749Z+ zfs create -o mountpoint=/work rpool/work
902024-03-26T19:43:50.796Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
912024-03-26T19:43:50.797Z+ home_fs=zfs
922024-03-26T19:43:50.797Z+ [[ zfs == autofs ]]
932024-03-26T19:43:50.797Z+ mkdir -p /home/build
942024-03-26T19:43:50.797Z+ chown build:build /home/build /work
952024-03-26T19:43:53.481Z+ chmod 0700 /home/build /work
962024-03-26T19:43:53.481Zprocess exited: duration 4050 ms, exit code 0
 
972024-03-26T19:43:53.557Zstarting task 1: "authentication"
982024-03-26T19:43:53.603Zprocess exited: duration 45 ms, exit code 0
 
992024-03-26T19:43:53.661Zstarting task 2: "build"
1002024-03-26T19:43:53.693Z+ trap _exit_trap EXIT
1012024-03-26T19:43:53.694Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
1022024-03-26T19:43:53.780Zsecpolicy_fs_linkdir: 0x55 = 0x55
1032024-03-26T19:43:53.780Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
1042024-03-26T19:43:53.780Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
1052024-03-26T19:43:53.780Zsecpolicy_fs_linkdir+3: 0 = 0xe5
1062024-03-26T19:43:53.780Zsecpolicy_fs_linkdir+4: 0 = 0xb8
1072024-03-26T19:43:53.781Zsecpolicy_fs_linkdir+5: 0 = 0x1
1082024-03-26T19:43:53.781Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
1092024-03-26T19:43:53.781Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
1102024-03-26T19:43:53.781Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
1112024-03-26T19:43:53.781Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
1122024-03-26T19:43:53.833Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
1132024-03-26T19:43:53.833Z+ [[ -d /opt/oxide ]]
1142024-03-26T19:43:53.834Z+ pfexec mv /opt/oxide /opt/oxide-underneath
1152024-03-26T19:43:53.834Z+ pfexec mkdir /opt/oxide
1162024-03-26T19:43:53.834Z+ pfexec mount -F tmpfs -O swap /opt/oxide
1172024-03-26T19:43:53.834Z+ [[ -d /opt/oxide-underneath ]]
1182024-03-26T19:43:53.834Z+ cd /opt/oxide-underneath
1192024-03-26T19:43:53.835Z+ pfexec tar ceEp@/f - .
1202024-03-26T19:43:53.835Z+ cd /opt/oxide
1212024-03-26T19:43:53.835Z+ pfexec tar xveEp@/f -
1222024-03-26T19:43:53.889Zx ., 0 bytes, 0 tape blocks
1232024-03-26T19:43:53.890Zx ./opte, 0 bytes, 0 tape blocks
1242024-03-26T19:43:53.890Zx ./opte/bin, 0 bytes, 0 tape blocks
1252024-03-26T19:43:53.890Zx ./opte/bin/opteadm, 43221608 bytes, 84418 tape blocks
1262024-03-26T19:43:53.947Z+ pfexec rm -rf /opt/oxide-underneath
1272024-03-26T19:43:53.947Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
1282024-03-26T19:43:53.947Z+ pfexec mkdir /opt/oxide/work
1292024-03-26T19:43:54.447Z+ pfexec chown build:build /opt/oxide/work
1302024-03-26T19:43:54.447Z+ cd /opt/oxide/work
1312024-03-26T19:43:54.447Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
1322024-03-26T19:43:55.569ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
1332024-03-26T19:43:55.569Zx out/mg-ddm-gz.tar, 75039232 bytes, 146561 tape blocks
1342024-03-26T19:43:55.596Zx out/omicron-sled-agent.tar, 114106880 bytes, 222865 tape blocks
1352024-03-26T19:43:55.596Zx out/oxlog.tar, 6244864 bytes, 12197 tape blocks
1362024-03-26T19:43:55.596Zx out/target/test, 75 bytes, 1 tape blocks
1372024-03-26T19:43:55.596Zx out/npuzone/npuzone, 37184104 bytes, 72626 tape blocks
1382024-03-26T19:43:55.596Zx package-manifest.toml, 26511 bytes, 52 tape blocks
1392024-03-26T19:43:55.596Zx smf/sled-agent/non-gimlet/config.toml, 3448 bytes, 7 tape blocks
1402024-03-26T19:43:55.596Zx target/release/omicron-package, 40142976 bytes, 78405 tape blocks
1412024-03-26T19:43:55.597Zx tools/create_virtual_hardware.sh, 3184 bytes, 7 tape blocks
1422024-03-26T19:43:55.597Zx tools/virtual_hardware.sh, 3429 bytes, 7 tape blocks
1432024-03-26T19:43:55.597Zx tools/scrimlet/create-softnpu-zone.sh, 627 bytes, 2 tape blocks
1442024-03-26T19:43:55.597Zx tools/scrimlet/destroy-softnpu-zone.sh, 203 bytes, 1 tape blocks
1452024-03-26T19:43:55.597Zx tools/scrimlet/softnpu-init.sh, 2157 bytes, 5 tape blocks
1462024-03-26T19:43:55.597Zx tools/scrimlet/softnpu-zone.txt, 330 bytes, 1 tape blocks
1472024-03-26T19:43:55.597Zx tools/scrimlet/softnpu.toml, 171 bytes, 1 tape blocks
1482024-03-26T19:43:55.597Z
1492024-03-26T19:43:55.597Zreal 1.643917017
1502024-03-26T19:43:55.597Zuser 1.481771255
1512024-03-26T19:43:55.597Zsys 0.285512255
1522024-03-26T19:43:55.598Ztrap 0.000049001
1532024-03-26T19:43:55.598Ztflt 0.000000000
1542024-03-26T19:43:55.598Zdflt 0.000000000
1552024-03-26T19:43:55.598Zkflt 0.000000000
1562024-03-26T19:43:55.598Zlock 0.000000000
1572024-03-26T19:43:55.598Zslp 1.481028247
1582024-03-26T19:43:55.598Zlat 0.037542896
1592024-03-26T19:43:55.598Zstop 0.000121953
1602024-03-26T19:43:55.598Z+ cp /input/package/work/zones/clickhouse_keeper.tar.gz /input/package/work/zones/clickhouse.tar.gz /input/package/work/zones/cockroachdb.tar.gz /input/package/work/zones/crucible-pantry-zone.tar.gz /input/package/work/zones/crucible-zone.tar.gz /input/package/work/zones/external-dns.tar.gz /input/package/work/zones/internal-dns.tar.gz /input/package/work/zones/nexus-single-sled.tar.gz /input/package/work/zones/nexus.tar.gz /input/package/work/zones/ntp.tar.gz /input/package/work/zones/omicron-gateway-asic.tar.gz /input/package/work/zones/omicron-gateway-softnpu.tar.gz /input/package/work/zones/overlay.tar.gz /input/package/work/zones/oximeter.tar.gz /input/package/work/zones/probe.tar.gz /input/package/work/zones/propolis-server.tar.gz /input/package/work/zones/switch-asic.tar.gz /input/package/work/zones/switch-softnpu.tar.gz out/
1612024-03-26T19:43:56.917Z+ mv out/nexus-single-sled.tar.gz out/nexus.tar.gz
1622024-03-26T19:43:56.947Z+ mkdir tests
1632024-03-26T19:43:56.947Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1642024-03-26T19:43:56.948Z+ ptime -m gunzip
1652024-03-26T19:43:56.948Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-3cb81999e7440661
1662024-03-26T19:43:57.139Z
1672024-03-26T19:43:57.163Zreal 0.204151618
1682024-03-26T19:43:57.163Zuser 0.190266751
1692024-03-26T19:43:57.163Zsys 0.013632778
1702024-03-26T19:43:57.163Ztrap 0.000016050
1712024-03-26T19:43:57.163Ztflt 0.000000000
1722024-03-26T19:43:57.163Zdflt 0.000000000
1732024-03-26T19:43:57.163Zkflt 0.000000000
1742024-03-26T19:43:57.163Zlock 0.000000000
1752024-03-26T19:43:57.163Zslp 0.000019990
1762024-03-26T19:43:57.163Zlat 0.000023880
1772024-03-26T19:43:57.164Zstop 0.000128863
1782024-03-26T19:43:57.164Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-3cb81999e7440661
1792024-03-26T19:43:57.164Z+ chmod a+x tests/bootstrap-3cb81999e7440661
1802024-03-26T19:43:57.164Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1812024-03-26T19:43:57.164Z+ ptime -m gunzip
1822024-03-26T19:43:57.164Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1832024-03-26T19:43:59.382Z
1842024-03-26T19:43:59.415Zreal 2.227327901
1852024-03-26T19:43:59.415Zuser 2.096329698
1862024-03-26T19:43:59.415Zsys 0.130611412
1872024-03-26T19:43:59.415Ztrap 0.000151854
1882024-03-26T19:43:59.416Ztflt 0.000000000
1892024-03-26T19:43:59.416Zdflt 0.000000000
1902024-03-26T19:43:59.416Zkflt 0.000000000
1912024-03-26T19:43:59.416Zlock 0.000000000
1922024-03-26T19:43:59.416Zslp 0.000023930
1932024-03-26T19:43:59.416Zlat 0.000023870
1942024-03-26T19:43:59.416Zstop 0.000127193
1952024-03-26T19:43:59.416Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1962024-03-26T19:43:59.416Z+ chmod a+x tests/bootstrap
1972024-03-26T19:43:59.416Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1982024-03-26T19:43:59.417Z+ ptime -m gunzip
1992024-03-26T19:43:59.417Z++ basename /input/ci-tools/work/end-to-end-tests/commtest-e25b00f6d79ff279
2002024-03-26T19:43:59.595Z
2012024-03-26T19:43:59.619Zreal 0.205797418
2022024-03-26T19:43:59.619Zuser 0.190313469
2032024-03-26T19:43:59.619Zsys 0.015210859
2042024-03-26T19:43:59.619Ztrap 0.000022291
2052024-03-26T19:43:59.619Ztflt 0.000000000
2062024-03-26T19:43:59.619Zdflt 0.000000000
2072024-03-26T19:43:59.619Zkflt 0.000000000
2082024-03-26T19:43:59.619Zlock 0.000000000
2092024-03-26T19:43:59.620Zslp 0.000024150
2102024-03-26T19:43:59.620Zlat 0.000022910
2112024-03-26T19:43:59.620Zstop 0.000132044
2122024-03-26T19:43:59.620Z++ basename /input/ci-tools/work/end-to-end-tests/commtest-e25b00f6d79ff279
2132024-03-26T19:43:59.620Z+ chmod a+x tests/commtest-e25b00f6d79ff279
2142024-03-26T19:43:59.620Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
2152024-03-26T19:43:59.620Z+ ptime -m gunzip
2162024-03-26T19:43:59.620Z++ basename /input/ci-tools/work/end-to-end-tests/dhcp_server-cdb0b653f05d35f0
2172024-03-26T19:43:59.810Z
2182024-03-26T19:43:59.834Zreal 0.204004483
2192024-03-26T19:43:59.834Zuser 0.190244249
2202024-03-26T19:43:59.834Zsys 0.013507486
2212024-03-26T19:43:59.834Ztrap 0.000016300
2222024-03-26T19:43:59.835Ztflt 0.000000000
2232024-03-26T19:43:59.835Zdflt 0.000000000
2242024-03-26T19:43:59.835Zkflt 0.000000000
2252024-03-26T19:43:59.835Zlock 0.000000000
2262024-03-26T19:43:59.835Zslp 0.000024320
2272024-03-26T19:43:59.835Zlat 0.000022941
2282024-03-26T19:43:59.835Zstop 0.000125353
2292024-03-26T19:43:59.835Z++ basename /input/ci-tools/work/end-to-end-tests/dhcp_server-cdb0b653f05d35f0
2302024-03-26T19:43:59.835Z+ chmod a+x tests/dhcp_server-cdb0b653f05d35f0
2312024-03-26T19:43:59.835Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
2322024-03-26T19:43:59.835Z+ ptime -m gunzip
2332024-03-26T19:43:59.835Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-92ca3720317208b2
2342024-03-26T19:44:02.091Z
2352024-03-26T19:44:02.124Zreal 2.268185298
2362024-03-26T19:44:02.125Zuser 2.141735252
2372024-03-26T19:44:02.125Zsys 0.126032774
2382024-03-26T19:44:02.125Ztrap 0.000183625
2392024-03-26T19:44:02.125Ztflt 0.000000000
2402024-03-26T19:44:02.125Zdflt 0.000000000
2412024-03-26T19:44:02.125Zkflt 0.000000000
2422024-03-26T19:44:02.125Zlock 0.000000000
2432024-03-26T19:44:02.125Zslp 0.000023610
2442024-03-26T19:44:02.125Zlat 0.000023190
2452024-03-26T19:44:02.125Zstop 0.000127783
2462024-03-26T19:44:02.125Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-92ca3720317208b2
2472024-03-26T19:44:02.125Z+ chmod a+x tests/end_to_end_tests-92ca3720317208b2
2482024-03-26T19:44:02.125Z+ bmat address ls
2492024-03-26T19:44:02.125ZNAME CIDR FIRST LAST COUNT
2502024-03-26T19:44:02.125Zextra 10.151.2.0/24 10.151.2.164 10.151.2.227 64
2512024-03-26T19:44:02.125Z++ bmat address ls -f extra -Ho first,last,count
2522024-03-26T19:44:02.125Z+ set -- 10.151.2.164 10.151.2.227 64
2532024-03-26T19:44:02.125Z+ EXTRA_IP_START=10.151.2.164
2542024-03-26T19:44:02.125Z+ EXTRA_IP_END=10.151.2.227
2552024-03-26T19:44:02.125Z+ EXTRA_IP_COUNT=64
2562024-03-26T19:44:02.126Z+ (( EXTRA_IP_COUNT >= 32 ))
2572024-03-26T19:44:02.126Z+ EXTRA_IP_BASE=10.151.2
2582024-03-26T19:44:02.126Z+ EXTRA_IP_FOCTET=164
2592024-03-26T19:44:02.126Z+ EXTRA_IP_LOCTET=227
2602024-03-26T19:44:02.126Z+ SERVICE_IP_POOL_START=10.151.2.164
2612024-03-26T19:44:02.126Z+ SERVICE_IP_POOL_END=10.151.2.173
2622024-03-26T19:44:02.126Z+ DNS_IP1=10.151.2.164
2632024-03-26T19:44:02.126Z+ DNS_IP2=10.151.2.165
2642024-03-26T19:44:02.126Z+ UPLINK_IP=10.151.2.174
2652024-03-26T19:44:02.126Z+ IPPOOL_START=10.151.2.175
2662024-03-26T19:44:02.126Z+ IPPOOL_END=10.151.2.227
2672024-03-26T19:44:02.154Z++ ipadm show-addr -po type,addr
2682024-03-26T19:44:02.154Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
2692024-03-26T19:44:02.154Z+ GATEWAY_IP=10.151.2.100
2702024-03-26T19:44:02.154Z+ [[ -n 10.151.2.100 ]]
2712024-03-26T19:44:02.154Z+ ping -s 10.151.2.100 56 1
2722024-03-26T19:44:02.154ZPING 10.151.2.100: 56 data bytes
2732024-03-26T19:44:02.154Z64 bytes from 10.151.2.100: icmp_seq=0. time=0.010 ms
2742024-03-26T19:44:02.154Z
2752024-03-26T19:44:02.154Z----10.151.2.100 PING Statistics----
2762024-03-26T19:44:02.155Z1 packets transmitted, 1 packets received, 0% packet loss
2772024-03-26T19:44:02.155Zround-trip (ms) min/avg/max/stddev = 0.010/0.010/0.010/-nan
2782024-03-26T19:44:02.155Z++ arp -an
2792024-03-26T19:44:02.155Z++ awk -vgw=10.151.2.100 '$2 == gw {print $NF}'
2802024-03-26T19:44:02.155Z+ GATEWAY_MAC=18:c0:4d:0d:a0:2a
2812024-03-26T19:44:02.155Z+ routeadm -e ipv4-forwarding -u
2822024-03-26T19:44:02.252Z+ PXA_START=10.151.2.164
2832024-03-26T19:44:02.253Z+ PXA_END=10.151.2.227
2842024-03-26T19:44:02.278Z+ export GATEWAY_IP GATEWAY_MAC PXA_START PXA_END
2852024-03-26T19:44:02.278Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
2862024-03-26T19:44:02.387Z+ ZPOOL_VDEV_DIR=/scratch
2872024-03-26T19:44:02.387Z+ ptime -m pfexec ./tools/create_virtual_hardware.sh
2882024-03-26T19:44:02.416Z+++ dirname ./tools/create_virtual_hardware.sh
2892024-03-26T19:44:02.416Z++ cd ./tools
2902024-03-26T19:44:02.416Z++ pwd
2912024-03-26T19:44:02.416Z+ SOURCE_DIR=/opt/oxide/work/tools
2922024-03-26T19:44:02.417Z+ OMICRON_TOP=/opt/oxide/work/tools/..
2932024-03-26T19:44:02.417Z+ SOFTNPU_MODE=zone
2942024-03-26T19:44:02.417Z+ . /opt/oxide/work/tools/virtual_hardware.sh
2952024-03-26T19:44:02.417Z++ MARKER=/etc/opt/oxide/NO_INSTALL
2962024-03-26T19:44:02.417Z++ [[ -f /etc/opt/oxide/NO_INSTALL ]]
2972024-03-26T19:44:02.417Z++ dladm show-phys -p -o LINK
2982024-03-26T19:44:02.417Z++ head -1
2992024-03-26T19:44:02.479Z+ PHYSICAL_LINK=igb0
3002024-03-26T19:44:02.509Z+ echo 'Using igb0 as physical link'
3012024-03-26T19:44:02.510ZUsing igb0 as physical link
3022024-03-26T19:44:02.510Z+ ensure_run_as_root
3032024-03-26T19:44:02.510Z++ id -u
3042024-03-26T19:44:02.510Z+ [[ 0 -ne 0 ]]
3052024-03-26T19:44:02.510Z+ ensure_zpools
3062024-03-26T19:44:02.510Z+ ZPOOL_TYPES=('oxp_' 'oxi_')
3072024-03-26T19:44:02.511Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
3082024-03-26T19:44:02.511Z+ readarray -t ZPOOLS
3092024-03-26T19:44:02.511Z++ grep '"oxp_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
3102024-03-26T19:44:02.511Z++ sed 's/[ ",]//g'
3112024-03-26T19:44:02.511Z+ for ZPOOL in "${ZPOOLS[@]}"
3122024-03-26T19:44:02.511Z+ echo 'Zpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]'
3132024-03-26T19:44:02.512ZZpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]
3142024-03-26T19:44:02.512Z+ VDEV_PATH=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
3152024-03-26T19:44:02.512Z+ [[ -f /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
3162024-03-26T19:44:02.512Z+ dd if=/dev/zero of=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
3172024-03-26T19:44:02.512Z0+0 records in
3182024-03-26T19:44:02.513Z0+0 records out
3192024-03-26T19:44:02.513Z0 bytes (0 iB) transferred in 0.000007 secs (0 iB/sec)
3202024-03-26T19:44:02.513Z+ success 'ZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
3212024-03-26T19:44:02.513Z+ set +x
3222024-03-26T19:44:02.513ZZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
3232024-03-26T19:44:02.541Z++ zpool list -o name
3242024-03-26T19:44:02.541Z++ grep oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
3252024-03-26T19:44:02.541Z+ [[ -z '' ]]
3262024-03-26T19:44:02.541Z+ zpool create -o ashift=12 -f oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
3272024-03-26T19:44:02.567Z+ success 'ZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
3282024-03-26T19:44:02.567Z+ set +x
3292024-03-26T19:44:02.567ZZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists
3302024-03-26T19:44:02.567Z+ for ZPOOL in "${ZPOOLS[@]}"
3312024-03-26T19:44:02.567Z+ echo 'Zpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3322024-03-26T19:44:02.567ZZpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]
3332024-03-26T19:44:02.567Z+ VDEV_PATH=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3342024-03-26T19:44:02.567Z+ [[ -f /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3352024-03-26T19:44:02.567Z+ dd if=/dev/zero of=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3362024-03-26T19:44:02.568Z0+0 records in
3372024-03-26T19:44:02.568Z0+0 records out
3382024-03-26T19:44:02.568Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3392024-03-26T19:44:02.568Z+ success 'ZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3402024-03-26T19:44:02.568Z+ set +x
3412024-03-26T19:44:02.568ZZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3422024-03-26T19:44:02.568Z++ zpool list -o name
3432024-03-26T19:44:02.568Z++ grep oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
3442024-03-26T19:44:02.568Z+ [[ -z '' ]]
3452024-03-26T19:44:02.568Z+ zpool create -o ashift=12 -f oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3462024-03-26T19:44:02.595Z+ success 'ZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3472024-03-26T19:44:02.595Z+ set +x
3482024-03-26T19:44:02.630ZZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3492024-03-26T19:44:02.630Z+ for ZPOOL in "${ZPOOLS[@]}"
3502024-03-26T19:44:02.630Z+ echo 'Zpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3512024-03-26T19:44:02.630ZZpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]
3522024-03-26T19:44:02.630Z+ VDEV_PATH=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3532024-03-26T19:44:02.630Z+ [[ -f /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3542024-03-26T19:44:02.630Z+ dd if=/dev/zero of=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3552024-03-26T19:44:02.630Z0+0 records in
3562024-03-26T19:44:02.630Z0+0 records out
3572024-03-26T19:44:02.631Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3582024-03-26T19:44:02.631Z+ success 'ZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3592024-03-26T19:44:02.631Z+ set +x
3602024-03-26T19:44:02.631ZZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3612024-03-26T19:44:02.631Z++ zpool list -o name
3622024-03-26T19:44:02.631Z++ grep oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
3632024-03-26T19:44:02.631Z+ [[ -z '' ]]
3642024-03-26T19:44:02.631Z+ zpool create -o ashift=12 -f oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3652024-03-26T19:44:02.658Z+ success 'ZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3662024-03-26T19:44:02.658Z+ set +x
3672024-03-26T19:44:02.658ZZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3682024-03-26T19:44:02.658Z+ for ZPOOL in "${ZPOOLS[@]}"
3692024-03-26T19:44:02.658Z+ echo 'Zpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3702024-03-26T19:44:02.658ZZpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]
3712024-03-26T19:44:02.658Z+ VDEV_PATH=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3722024-03-26T19:44:02.658Z+ [[ -f /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3732024-03-26T19:44:02.659Z+ dd if=/dev/zero of=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3742024-03-26T19:44:02.659Z0+0 records in
3752024-03-26T19:44:02.659Z0+0 records out
3762024-03-26T19:44:02.659Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3772024-03-26T19:44:02.659Z+ success 'ZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3782024-03-26T19:44:02.659Z+ set +x
3792024-03-26T19:44:02.659ZZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3802024-03-26T19:44:02.659Z++ zpool list -o name
3812024-03-26T19:44:02.659Z++ grep oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03
3822024-03-26T19:44:02.659Z+ [[ -z '' ]]
3832024-03-26T19:44:02.659Z+ zpool create -o ashift=12 -f oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3842024-03-26T19:44:02.689Z+ success 'ZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3852024-03-26T19:44:02.689Z+ set +x
3862024-03-26T19:44:02.689ZZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3872024-03-26T19:44:02.689Z+ for ZPOOL in "${ZPOOLS[@]}"
3882024-03-26T19:44:02.689Z+ echo 'Zpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3892024-03-26T19:44:02.689ZZpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]
3902024-03-26T19:44:02.689Z+ VDEV_PATH=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3912024-03-26T19:44:02.690Z+ [[ -f /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3922024-03-26T19:44:02.690Z+ dd if=/dev/zero of=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3932024-03-26T19:44:02.690Z0+0 records in
3942024-03-26T19:44:02.690Z0+0 records out
3952024-03-26T19:44:02.690Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3962024-03-26T19:44:02.690Z+ success 'ZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3972024-03-26T19:44:02.690Z+ set +x
3982024-03-26T19:44:02.690ZZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3992024-03-26T19:44:02.720Z++ zpool list -o name
4002024-03-26T19:44:02.720Z++ grep oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03
4012024-03-26T19:44:02.721Z+ [[ -z '' ]]
4022024-03-26T19:44:02.721Z+ zpool create -o ashift=12 -f oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
4032024-03-26T19:44:02.747Z+ success 'ZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
4042024-03-26T19:44:02.747Z+ set +x
4052024-03-26T19:44:02.747ZZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
4062024-03-26T19:44:02.747Z+ for ZPOOL in "${ZPOOLS[@]}"
4072024-03-26T19:44:02.747Z+ echo 'Zpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]'
4082024-03-26T19:44:02.747ZZpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]
4092024-03-26T19:44:02.747Z+ VDEV_PATH=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
4102024-03-26T19:44:02.747Z+ [[ -f /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev ]]
4112024-03-26T19:44:02.748Z+ dd if=/dev/zero of=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev bs=1 count=0 seek=20G
4122024-03-26T19:44:02.748Z0+0 records in
4132024-03-26T19:44:02.748Z0+0 records out
4142024-03-26T19:44:02.748Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4152024-03-26T19:44:02.748Z+ success 'ZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists'
4162024-03-26T19:44:02.748Z+ set +x
4172024-03-26T19:44:02.748ZZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists
4182024-03-26T19:44:02.748Z++ zpool list -o name
4192024-03-26T19:44:02.748Z++ grep oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930
4202024-03-26T19:44:02.748Z+ [[ -z '' ]]
4212024-03-26T19:44:02.748Z+ zpool create -o ashift=12 -f oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
4222024-03-26T19:44:02.781Z+ success 'ZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists'
4232024-03-26T19:44:02.781Z+ set +x
4242024-03-26T19:44:02.781ZZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists
4252024-03-26T19:44:02.781ZZpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]
4262024-03-26T19:44:02.781Z+ for ZPOOL in "${ZPOOLS[@]}"
4272024-03-26T19:44:02.781Z+ echo 'Zpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]'
4282024-03-26T19:44:02.782Z+ VDEV_PATH=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
4292024-03-26T19:44:02.782Z+ [[ -f /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev ]]
4302024-03-26T19:44:02.782Z+ dd if=/dev/zero of=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev bs=1 count=0 seek=20G
4312024-03-26T19:44:02.782Z0+0 records in
4322024-03-26T19:44:02.782Z0+0 records out
4332024-03-26T19:44:02.782Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4342024-03-26T19:44:02.782Z+ success 'ZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists'
4352024-03-26T19:44:02.782Z+ set +x
4362024-03-26T19:44:02.782ZZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists
4372024-03-26T19:44:02.807Z++ zpool list -o name
4382024-03-26T19:44:02.808Z++ grep oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60
4392024-03-26T19:44:02.808Z+ [[ -z '' ]]
4402024-03-26T19:44:02.808Z+ zpool create -o ashift=12 -f oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
4412024-03-26T19:44:02.836Z+ success 'ZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists'
4422024-03-26T19:44:02.836Z+ set +x
4432024-03-26T19:44:02.836ZZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists
4442024-03-26T19:44:02.836Z+ for ZPOOL in "${ZPOOLS[@]}"
4452024-03-26T19:44:02.836Z+ echo 'Zpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]'
4462024-03-26T19:44:02.836ZZpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]
4472024-03-26T19:44:02.836Z+ VDEV_PATH=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4482024-03-26T19:44:02.836Z+ [[ -f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev ]]
4492024-03-26T19:44:02.836Z+ dd if=/dev/zero of=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev bs=1 count=0 seek=20G
4502024-03-26T19:44:02.836Z0+0 records in
4512024-03-26T19:44:02.836Z0+0 records out
4522024-03-26T19:44:02.836Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4532024-03-26T19:44:02.836Z+ success 'ZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists'
4542024-03-26T19:44:02.836Z+ set +x
4552024-03-26T19:44:02.837ZZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists
4562024-03-26T19:44:02.837Z++ zpool list -o name
4572024-03-26T19:44:02.837Z++ grep oxp_31bd71cd-4736-4a12-a387-9b74b050396f
4582024-03-26T19:44:02.837Z+ [[ -z '' ]]
4592024-03-26T19:44:02.837Z+ zpool create -o ashift=12 -f oxp_31bd71cd-4736-4a12-a387-9b74b050396f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4602024-03-26T19:44:02.862Z+ success 'ZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists'
4612024-03-26T19:44:02.862ZZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists
4622024-03-26T19:44:02.862Z+ set +x
4632024-03-26T19:44:02.862Z+ for ZPOOL in "${ZPOOLS[@]}"
4642024-03-26T19:44:02.862Z+ echo 'Zpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]'
4652024-03-26T19:44:02.862ZZpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]
4662024-03-26T19:44:02.862Z+ VDEV_PATH=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4672024-03-26T19:44:02.862Z+ [[ -f /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev ]]
4682024-03-26T19:44:02.862Z+ dd if=/dev/zero of=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev bs=1 count=0 seek=20G
4692024-03-26T19:44:02.862Z0+0 records in
4702024-03-26T19:44:02.862Z0+0 records out
4712024-03-26T19:44:02.863Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4722024-03-26T19:44:02.863Z+ success 'ZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists'
4732024-03-26T19:44:02.863Z+ set +x
4742024-03-26T19:44:02.863ZZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists
4752024-03-26T19:44:03.299Z++ zpool list -o name
4762024-03-26T19:44:03.299Z++ grep oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7
4772024-03-26T19:44:03.299Z+ [[ -z '' ]]
4782024-03-26T19:44:03.300Z+ zpool create -o ashift=12 -f oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4792024-03-26T19:44:03.348Z+ success 'ZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists'
4802024-03-26T19:44:03.348Z+ set +x
4812024-03-26T19:44:03.348ZZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists
4822024-03-26T19:44:03.348Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
4832024-03-26T19:44:03.348Z+ readarray -t ZPOOLS
4842024-03-26T19:44:03.348Z++ grep '"oxi_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
4852024-03-26T19:44:03.348Z++ sed 's/[ ",]//g'
4862024-03-26T19:44:03.348Z+ for ZPOOL in "${ZPOOLS[@]}"
4872024-03-26T19:44:03.348Z+ echo 'Zpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4882024-03-26T19:44:03.348ZZpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]
4892024-03-26T19:44:03.349Z+ VDEV_PATH=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4902024-03-26T19:44:03.349Z+ [[ -f /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4912024-03-26T19:44:03.349Z+ dd if=/dev/zero of=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4922024-03-26T19:44:03.349Z0+0 records in
4932024-03-26T19:44:03.349Z0+0 records out
4942024-03-26T19:44:03.349Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4952024-03-26T19:44:03.349Z+ success 'ZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4962024-03-26T19:44:03.349Z+ set +x
4972024-03-26T19:44:03.349ZZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4982024-03-26T19:44:03.349Z++ zpool list -o name
4992024-03-26T19:44:03.349Z++ grep oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b
5002024-03-26T19:44:03.349Z+ [[ -z '' ]]
5012024-03-26T19:44:03.349Z+ zpool create -o ashift=12 -f oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
5022024-03-26T19:44:03.349Z+ success 'ZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
5032024-03-26T19:44:03.349Z+ set +x
5042024-03-26T19:44:03.349ZZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists
5052024-03-26T19:44:03.349Z+ for ZPOOL in "${ZPOOLS[@]}"
5062024-03-26T19:44:03.349Z+ echo 'Zpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]'
5072024-03-26T19:44:03.349ZZpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]
5082024-03-26T19:44:03.350Z+ VDEV_PATH=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
5092024-03-26T19:44:03.350Z+ [[ -f /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
5102024-03-26T19:44:03.350Z+ dd if=/dev/zero of=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
5112024-03-26T19:44:03.350Z0+0 records in
5122024-03-26T19:44:03.350Z0+0 records out
5132024-03-26T19:44:03.350Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
5142024-03-26T19:44:03.350Z+ success 'ZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
5152024-03-26T19:44:03.350Z+ set +x
5162024-03-26T19:44:03.350ZZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
5172024-03-26T19:44:03.350Z++ zpool list -o name
5182024-03-26T19:44:03.350Z++ grep oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b
5192024-03-26T19:44:03.350Z+ [[ -z '' ]]
5202024-03-26T19:44:03.350Z+ zpool create -o ashift=12 -f oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
5212024-03-26T19:44:03.350Z+ success 'ZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
5222024-03-26T19:44:03.350Z+ set +x
5232024-03-26T19:44:03.350ZZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists
5242024-03-26T19:44:03.350Z+ [[ zone == \z\o\n\e ]]
5252024-03-26T19:44:03.350Z+ ensure_simulated_links igb0
5262024-03-26T19:44:03.350Z+ local PHYSICAL_LINK=igb0
5272024-03-26T19:44:03.350Z+ INDICES=("0" "1")
5282024-03-26T19:44:03.351Z+ for I in "${INDICES[@]}"
5292024-03-26T19:44:03.351Z++ get_simnet_name_if_exists net0
5302024-03-26T19:44:03.351Z++ dladm show-simnet -p -o LINK net0
5312024-03-26T19:44:03.351Z++ echo ''
5322024-03-26T19:44:03.351Z+ [[ -z '' ]]
5332024-03-26T19:44:03.351Z+ dladm create-simnet -t net0
5342024-03-26T19:44:03.351Z+ dladm create-simnet -t sc0_0
5352024-03-26T19:44:03.351Z+ dladm modify-simnet -t -p net0 sc0_0
5362024-03-26T19:44:03.351Z+ dladm set-linkprop -p mtu=9000 sc0_0
5372024-03-26T19:44:03.351Z+ success 'Simnet net0/sc0_0 exists'
5382024-03-26T19:44:03.351Z+ set +x
5392024-03-26T19:44:03.351ZSimnet net0/sc0_0 exists
5402024-03-26T19:44:03.351Z+ for I in "${INDICES[@]}"
5412024-03-26T19:44:03.351Z++ get_simnet_name_if_exists net1
5422024-03-26T19:44:03.351Z++ dladm show-simnet -p -o LINK net1
5432024-03-26T19:44:03.351Z++ echo ''
5442024-03-26T19:44:03.351Z+ [[ -z '' ]]
5452024-03-26T19:44:03.351Z+ dladm create-simnet -t net1
5462024-03-26T19:44:03.351Z+ dladm create-simnet -t sc1_0
5472024-03-26T19:44:03.351Z+ dladm modify-simnet -t -p net1 sc1_0
5482024-03-26T19:44:03.351Z+ dladm set-linkprop -p mtu=9000 sc1_0
5492024-03-26T19:44:03.351Z+ success 'Simnet net1/sc1_0 exists'
5502024-03-26T19:44:03.352Z+ set +x
5512024-03-26T19:44:03.352ZSimnet net1/sc1_0 exists
5522024-03-26T19:44:03.352Z++ get_vnic_name_if_exists sc0_1
5532024-03-26T19:44:03.352Z++ dladm show-vnic -p -o LINK sc0_1
5542024-03-26T19:44:03.352Z++ echo ''
5552024-03-26T19:44:03.352Z+ [[ -z '' ]]
5562024-03-26T19:44:03.352Z+ dladm create-vnic -t sc0_1 -l igb0 -m a8:e1:de:01:70:1d
5572024-03-26T19:44:03.352Z+ [[ -v PROMISC_FILT_OFF ]]
5582024-03-26T19:44:03.352Z+ success 'Vnic sc0_1 exists'
5592024-03-26T19:44:03.352Z+ set +x
5602024-03-26T19:44:03.352ZVnic sc0_1 exists
5612024-03-26T19:44:03.352Z+ warn_if_no_proxy_arp
5622024-03-26T19:44:03.352Z+ [[ -v PXA_START ]]
5632024-03-26T19:44:03.352Z+ [[ -v PXA_END ]]
5642024-03-26T19:44:03.352Z+ ensure_softnpu_zone
5652024-03-26T19:44:03.352Z+ zoneadm list
5662024-03-26T19:44:03.352Z+ grep -q sidecar_softnpu
5672024-03-26T19:44:03.352Z+ [[ -f out/npuzone/npuzone ]]
5682024-03-26T19:44:03.352Z+ out/npuzone/npuzone create sidecar --omicron-zone --ports sc0_0,tfportrear0_0 --ports sc0_1,tfportqsfp0_0 --sidecar-lite-commit e3ea4b495ba0a71801ded0776ae4bbd31df57e26 --softnpu-commit dbab082dfa89da5db5ca2325c257089d2f130092
5692024-03-26T19:44:03.352Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/softnpu/image/dbab082dfa89da5db5ca2325c257089d2f130092/softnpu.sha256.txt
5702024-03-26T19:44:14.005Zdownloading softnpu ...done
5712024-03-26T19:44:14.030Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/libsidecar_lite.so.sha256.txt
5722024-03-26T19:44:22.087Zdownloading asic_program.so ...done
5732024-03-26T19:44:22.111Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/scadm.sha256.txt
5742024-03-26T19:44:32.720Zdownloading scadm ...done
5752024-03-26T19:44:32.744Zcreating port tfportrear0_0 for sc0_0
5762024-03-26T19:44:32.745Zcreating port tfportqsfp0_0 for sc0_1
5772024-03-26T19:44:32.769Zsoftnpu zone setup
5782024-03-26T19:44:32.769Zconfigure zone
5792024-03-26T19:44:32.769Zinstall zone
5802024-03-26T19:44:33.804Zboot zone
5812024-03-26T19:44:36.909Z[sidecar_softnpu] svcs milestone/network
5822024-03-26T19:44:36.936ZSTATE STIME FMRI
5832024-03-26T19:44:36.936Zonline 19:44:35 svc:/milestone/network:default
5842024-03-26T19:44:36.936Z[sidecar_softnpu] /softnpu/npu start
5852024-03-26T19:44:36.966Z
5862024-03-26T19:44:36.994Z+ /opt/oxide/work/tools/scrimlet/softnpu-init.sh
5872024-03-26T19:44:36.994Z+ GATEWAY_IP=10.151.2.100
5882024-03-26T19:44:36.994Z+ echo 'Using 10.151.2.100 as gateway ip'
5892024-03-26T19:44:36.994ZUsing 10.151.2.100 as gateway ip
5902024-03-26T19:44:36.994Z+ [[ ! -v GATEWAY_MAC ]]
5912024-03-26T19:44:36.994Z+ GATEWAY_MAC=18:c0:4d:0d:a0:2a
5922024-03-26T19:44:36.994Z++ grep -c -E '^([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}$' /dev/fd/63
5932024-03-26T19:44:36.995Z+++ echo 18:c0:4d:0d:a0:2a
5942024-03-26T19:44:36.995Z+ COUNT=1
5952024-03-26T19:44:36.995Z+ [[ 1 -ne 1 ]]
5962024-03-26T19:44:36.995Z+ echo 'Using 18:c0:4d:0d:a0:2a as gateway mac'
5972024-03-26T19:44:36.995ZUsing 18:c0:4d:0d:a0:2a as gateway mac
5982024-03-26T19:44:36.995Z+ z_scadm add-arp-entry 10.151.2.100 18:c0:4d:0d:a0:2a
5992024-03-26T19:44:36.995Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-arp-entry 10.151.2.100 18:c0:4d:0d:a0:2a
6002024-03-26T19:44:37.020Z+ PXA_MAC=a8:e1:de:01:70:1d
6012024-03-26T19:44:37.021Z+ [[ -v PXA_START ]]
6022024-03-26T19:44:37.021Z+ z_scadm add-proxy-arp 10.151.2.164 10.151.2.227 a8:e1:de:01:70:1d
6032024-03-26T19:44:37.021Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-proxy-arp 10.151.2.164 10.151.2.227 a8:e1:de:01:70:1d
6042024-03-26T19:44:37.046Z+ z_scadm dump-state
6052024-03-26T19:44:37.046Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
6062024-03-26T19:44:37.071Zlocal v6:
6072024-03-26T19:44:37.071Zlocal v4:
6082024-03-26T19:44:37.071Zrouter v6:
6092024-03-26T19:44:37.072Zrouter v4_idx:
6102024-03-26T19:44:37.072Zrouter v4_routes:
6112024-03-26T19:44:37.072Zresolver v4:
6122024-03-26T19:44:37.072Z10.151.2.100 -> 18:c0:4d:0d:a0:2a
6132024-03-26T19:44:37.072Zresolver v6:
6142024-03-26T19:44:37.072Znat v4:
6152024-03-26T19:44:37.072Znat_v6:
6162024-03-26T19:44:37.072Zport_mac:
6172024-03-26T19:44:37.072Zproxy arp:
6182024-03-26T19:44:37.072Z10.151.2.164/10.151.2.227: a8:e1:de:01:70:1d
6192024-03-26T19:44:37.072Z+ success 'softnpu zone exists'
6202024-03-26T19:44:37.072Z+ set +x
6212024-03-26T19:44:37.072Zsoftnpu zone exists
6222024-03-26T19:44:37.072Z
6232024-03-26T19:44:37.072Zreal 34.674890458
6242024-03-26T19:44:37.073Zuser 0.702337250
6252024-03-26T19:44:37.073Zsys 1.562247782
6262024-03-26T19:44:37.073Ztrap 0.000100626
6272024-03-26T19:44:37.073Ztflt 0.000146526
6282024-03-26T19:44:37.073Zdflt 0.001079158
6292024-03-26T19:44:37.073Zkflt 0.000000000
6302024-03-26T19:44:37.073Zlock 27:05.049739052
6312024-03-26T19:44:37.073Zslp 1:44.008144809
6322024-03-26T19:44:37.073Zlat 0.057327063
6332024-03-26T19:44:37.073Zstop 0.339842983
6342024-03-26T19:44:37.073Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml
6352024-03-26T19:44:37.099Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
6362024-03-26T19:44:39.064Z+ SILO_NAME=recovery
6372024-03-26T19:44:39.064Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
6382024-03-26T19:44:39.064Z+ EXTERNAL_DNS_DOMAIN=oxide.test
6392024-03-26T19:44:39.064Z+ sed -i~ '
6402024-03-26T19:44:39.064Z /^external_dns_ips/c\
6412024-03-26T19:44:39.064Zexternal_dns_ips = [ "10.151.2.164", "10.151.2.165" ]
6422024-03-26T19:44:39.064Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
6432024-03-26T19:44:39.065Z /^first/c\
6442024-03-26T19:44:39.065Zfirst = "10.151.2.164"
6452024-03-26T19:44:39.065Z /^last/c\
6462024-03-26T19:44:39.065Zlast = "10.151.2.173"
6472024-03-26T19:44:39.065Z }
6482024-03-26T19:44:39.065Z /^infra_ip_first/c\
6492024-03-26T19:44:39.065Zinfra_ip_first = "10.151.2.174"
6502024-03-26T19:44:39.065Z /^infra_ip_last/c\
6512024-03-26T19:44:39.065Zinfra_ip_last = "10.151.2.174"
6522024-03-26T19:44:39.065Z /^\[\[rack_network_config.ports/,/^$/ {
6532024-03-26T19:44:39.066Z /^routes/c\
6542024-03-26T19:44:39.066Zroutes = \[{nexthop = "10.151.2.100", destination = "0.0.0.0/0"}\]
6552024-03-26T19:44:39.066Z /^addresses/c\
6562024-03-26T19:44:39.066Zaddresses = \["10.151.2.174/24"\]
6572024-03-26T19:44:39.066Z }
6582024-03-26T19:44:39.066Z' pkg/config-rss.toml
6592024-03-26T19:44:39.066Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
6602024-03-26T19:44:39.066Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
6612024-03-26T19:44:39.066Z+++ pkg/config-rss.toml Tue Mar 26 19:44:36 2024
6622024-03-26T19:44:39.066Z@@ -33,7 +33,7 @@
6632024-03-26T19:44:39.066Z # the DNS domain delegated to the rack by the customer. Each of these addresses
6642024-03-26T19:44:39.066Z # must be contained in one of the "internal services" IP Pool ranges listed
6652024-03-26T19:44:39.066Z # below.
6662024-03-26T19:44:39.067Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
6672024-03-26T19:44:39.067Z+external_dns_ips = [ "10.151.2.164", "10.151.2.165" ]
6682024-03-26T19:44:39.067Z
6692024-03-26T19:44:39.067Z # Initial TLS certificates for the external API
6702024-03-26T19:44:39.067Z #
6712024-03-26T19:44:39.067Z@@ -69,8 +69,8 @@
6722024-03-26T19:44:39.067Z #
6732024-03-26T19:44:39.067Z # For more on this and what to put here, see docs/how-to-run.adoc.
6742024-03-26T19:44:39.067Z [[internal_services_ip_pool_ranges]]
6752024-03-26T19:44:39.067Z-first = "192.168.1.20"
6762024-03-26T19:44:39.067Z-last = "192.168.1.29"
6772024-03-26T19:44:39.068Z+first = "10.151.2.164"
6782024-03-26T19:44:39.068Z+last = "10.151.2.173"
6792024-03-26T19:44:39.068Z
6802024-03-26T19:44:39.068Z # TODO - this configuration is subject to change going forward. Ultimately these
6812024-03-26T19:44:39.068Z # parameters should be provided to the control plane via wicket, but we need to
6822024-03-26T19:44:39.068Z@@ -91,8 +91,8 @@
6832024-03-26T19:44:39.068Z # A range of IP addresses used by Boundary Services on the external network. In
6842024-03-26T19:44:39.068Z # a real system, these would be addresses of the uplink ports on the Sidecar.
6852024-03-26T19:44:39.068Z # With softnpu, only one address is used.
6862024-03-26T19:44:39.068Z-infra_ip_first = "192.168.1.30"
6872024-03-26T19:44:39.068Z-infra_ip_last = "192.168.1.30"
6882024-03-26T19:44:39.068Z+infra_ip_first = "10.151.2.174"
6892024-03-26T19:44:39.068Z+infra_ip_last = "10.151.2.174"
6902024-03-26T19:44:39.068Z
6912024-03-26T19:44:39.068Z # Configurations for BGP routers to run on the scrimlets.
6922024-03-26T19:44:39.068Z bgp = []
6932024-03-26T19:44:39.069Z@@ -100,9 +100,9 @@
6942024-03-26T19:44:39.069Z # You can configure multiple uplinks by repeating the following stanza
6952024-03-26T19:44:39.069Z [[rack_network_config.ports]]
6962024-03-26T19:44:39.069Z # Routes associated with this port.
6972024-03-26T19:44:39.069Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
6982024-03-26T19:44:39.069Z+routes = [{nexthop = "10.151.2.100", destination = "0.0.0.0/0"}]
6992024-03-26T19:44:39.069Z # Addresses associated with this port.
7002024-03-26T19:44:39.069Z-addresses = ["192.168.1.30/24"]
7012024-03-26T19:44:39.069Z+addresses = ["10.151.2.174/24"]
7022024-03-26T19:44:39.069Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
7032024-03-26T19:44:39.069Z port = "qsfp0"
7042024-03-26T19:44:39.069Z # The speed of this port.
7052024-03-26T19:44:39.069Z+ true
7062024-03-26T19:44:39.069Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml
7072024-03-26T19:44:39.069Za pkg/config-rss.toml 7K
7082024-03-26T19:44:39.069Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~
7092024-03-26T19:44:39.069Z+ TLS_NAME=recovery.sys.oxide.test
7102024-03-26T19:44:39.069Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
7112024-03-26T19:44:39.106Z..+............+...+..+.......+.................+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*........+.......+.....+............+.+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+....+....................+.........+...+.........+.+.........+......+..+.............+...............+...+.....+......+...+..........+..+....+.....+....+.....+............+................+...+.........+.....+......+.+.....+.+........................+.........+..............+.............+....................+...+....+...........+.......+...........................+.....+.......+..............+....+..+.+.................+...+...+.......+..................+..+.......+..+...............+.........+....+...+..+..........+..+....+............+..............+...+...+...+......................+..+.+...........+......+...+.......+.....+.........+................+.........+...+..+.........+......+....+.........+........+.........+...+...+.............+.....+......+....+......+...........+...+.+...............+.........+..+.........+...+.+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
7122024-03-26T19:44:39.106Z..+.........+.....+....+..+...+....+...........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+.........+.+...+..+....+...+.....+......+................+...+..+................+.....+...+.+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*......+...............+....+...+.................+.+...............+......+.....+.........+......+...+...+...+...............+......+....+..........................+...................+.........+...+........................+..............+.+...+...+...+.....+.......+...+..+...............+..........+..+......+....+..+...+......+................+...+..+.............+..+......+...+......+.+..+.+..+.........+......+..........+...+.....+.+...+............+......+........+.......+........+...................+..+...+.+......+.....+....+..+.............+.....+...............+.+....................+......+....+...+...+.................+.......+.....+..........+......+............+......+............+..+.+......+..................+......+..+.......+...............+.....+...+...+............+...+..........+......+...............+...+........+...+......+.+...+............+............+...........+.......+.....+............+.......+...........+....+..................+...........+....+...+..+.+.........+......+.........+......+.....+.+.....+.+........+..........+...........+...+...+...+......+.+............+..+...................+..+..........+........+........................+.+...........+.+.....+...+...............+.......+............+...............+...........+.......+..+.+...........+.+..+...+...+.........+.......+......+..............+.............+.....+.............+.........+...+..+.+..............+.+..+...............+......+....+...+....................+...............+...+..........+.....+.......+..+............+...+.....................+.+.....+.+............+......+..+.............+.....................+.....+....+......+......+.....+........................+......+...+....+...+...+..+....+.....+......+..........+........+.+.....+.+...+.................+....+......+..+......+.......................................+.+.........+.....+..........+..+...+.+.....+.+..........................+....+.........+..............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
7132024-03-26T19:44:39.106Z-----
7142024-03-26T19:44:39.106Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
7152024-03-26T19:44:39.106Za pkg/initial-tls-cert.pem 2K
7162024-03-26T19:44:39.107Za pkg/initial-tls-key.pem 4K
7172024-03-26T19:44:39.107Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
7182024-03-26T19:44:39.107Z+ rmdir pkg
7192024-03-26T19:44:39.107Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
7202024-03-26T19:44:39.107Z+ pfexec mkdir -p /usr/oxide
7212024-03-26T19:44:39.107Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0
7222024-03-26T19:44:39.660Z+ pfexec chmod +x /usr/oxide/oxide
7232024-03-26T19:44:39.686Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
7242024-03-26T19:44:57.976Z+ OMICRON_NO_UNINSTALL=1
7252024-03-26T19:44:57.976Z+ ptime -m pfexec ./target/release/omicron-package -t test install
7262024-03-26T19:44:58.010ZLogging to: /opt/oxide/work/out/LOG
7272024-03-26T19:44:58.532Z
7282024-03-26T19:44:58.564Zreal 0.550996903
7292024-03-26T19:44:58.564Zuser 0.114666416
7302024-03-26T19:44:58.564Zsys 2.211987384
7312024-03-26T19:44:58.564Ztrap 0.000007849
7322024-03-26T19:44:58.564Ztflt 0.000018850
7332024-03-26T19:44:58.565Zdflt 0.000986489
7342024-03-26T19:44:58.565Zkflt 0.000000000
7352024-03-26T19:44:58.565Zlock 50.137010672
7362024-03-26T19:44:58.565Zslp 0.769002903
7372024-03-26T19:44:58.565Zlat 0.003351579
7382024-03-26T19:44:58.565Zstop 0.214263167
7392024-03-26T19:44:58.565Z+ retry=0
7402024-03-26T19:44:58.566Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7412024-03-26T19:44:58.566Z+ [[ 0 -gt 30 ]]
7422024-03-26T19:44:58.566Z+ sleep 1
7432024-03-26T19:44:59.553Z+ retry=1
7442024-03-26T19:44:59.553Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7452024-03-26T19:44:59.593Z+ [[ 1 -gt 30 ]]
7462024-03-26T19:44:59.593Z+ sleep 1
7472024-03-26T19:45:00.556Z+ retry=2
7482024-03-26T19:45:00.557Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7492024-03-26T19:45:00.593Z+ [[ 2 -gt 30 ]]
7502024-03-26T19:45:00.593Z+ sleep 1
7512024-03-26T19:45:01.572Z+ retry=3
7522024-03-26T19:45:01.572Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7532024-03-26T19:45:01.600Z+ [[ 3 -gt 30 ]]
7542024-03-26T19:45:01.600Z+ sleep 1
7552024-03-26T19:45:02.586Z+ retry=4
7562024-03-26T19:45:02.587Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7572024-03-26T19:45:27.754Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
7582024-03-26T19:45:27.783Zlocal v6:
7592024-03-26T19:45:27.783Zfd71:c865:c5c8:ced3::1
7602024-03-26T19:45:27.783Zfe80::e460:cbff:fe52:cef9
7612024-03-26T19:45:27.783Zlocal v4:
7622024-03-26T19:45:27.783Z10.151.2.174
7632024-03-26T19:45:27.784Zrouter v6:
7642024-03-26T19:45:27.784Zfdb0:18c0:4d0d:a02a::/64 -> fe80::f88d:66ff:fea7:16e0 (1)
7652024-03-26T19:45:27.784Zfd00:1122:3344:101::/64 -> fe80::f88d:66ff:fea7:16e0 (1)
7662024-03-26T19:45:27.784Zfd00:1122:3344:3::/64 -> fe80::f88d:66ff:fea7:16e0 (1)
7672024-03-26T19:45:27.784Zfd00:1122:3344:1::/64 -> fe80::f88d:66ff:fea7:16e0 (1)
7682024-03-26T19:45:27.784Zfd00:1122:3344:2::/64 -> fe80::f88d:66ff:fea7:16e0 (1)
7692024-03-26T19:45:27.784Zrouter v4_idx:
7702024-03-26T19:45:27.784Z0.0.0.0/0 -> 0
7712024-03-26T19:45:27.784Zrouter v4_routes:
7722024-03-26T19:45:27.784Z0 -> 10.151.2.100 (2)
7732024-03-26T19:45:27.784Zresolver v4:
7742024-03-26T19:45:27.784Z10.151.2.100 -> 18:c0:4d:0d:a0:2a
7752024-03-26T19:45:27.785Z10.151.2.174 -> 7a:a9:84:30:0c:5a
7762024-03-26T19:45:27.785Zresolver v6:
7772024-03-26T19:45:27.817Zfe80::f88d:66ff:fea7:16e0 -> fa:8d:66:a7:16:e0
7782024-03-26T19:45:27.817Zfe80::e460:cbff:fe52:cef9 -> e6:60:cb:52:ce:f9
7792024-03-26T19:45:27.817Znat v4:
7802024-03-26T19:45:27.817Z10.151.2.169 0/16383 -> fd00:1122:3344:101::1 100/a8:40:25:ff:a8:93
7812024-03-26T19:45:27.818Znat_v6:
7822024-03-26T19:45:27.818Zport_mac:
7832024-03-26T19:45:27.818Z1000: a8:40:25:7a:ea:d0
7842024-03-26T19:45:27.818Z1: a8:40:25:7a:ea:d1
7852024-03-26T19:45:27.818Z2: a8:40:25:7a:ea:f1
7862024-03-26T19:45:27.818Zproxy arp:
7872024-03-26T19:45:27.818Z10.151.2.164/10.151.2.227: a8:e1:de:01:70:1d
7882024-03-26T19:45:27.818Z+ retry=0
7892024-03-26T19:45:27.819Z+++ zoneadm list -n
7902024-03-26T19:45:27.819Z+++ grep oxz_ntp
7912024-03-26T19:45:27.819Z++ pfexec svcs -z oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b -Hostate oxide/ntp
7922024-03-26T19:45:27.819Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7932024-03-26T19:45:27.819Z++ true
7942024-03-26T19:45:27.819Z+ [[ '' != online ]]
7952024-03-26T19:45:27.819Z+ [[ 0 -gt 60 ]]
7962024-03-26T19:45:27.820Z+ sleep 1
7972024-03-26T19:45:28.815Z+ retry=1
7982024-03-26T19:45:28.855Z+++ zoneadm list -n
7992024-03-26T19:45:28.856Z+++ grep oxz_ntp
8002024-03-26T19:45:28.857Z++ pfexec svcs -z oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b -Hostate oxide/ntp
8012024-03-26T19:45:28.859Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
8022024-03-26T19:45:29.611Z++ true
8032024-03-26T19:45:29.611Z+ [[ '' != online ]]
8042024-03-26T19:45:29.612Z+ [[ 1 -gt 60 ]]
8052024-03-26T19:45:29.612Z+ sleep 1
8062024-03-26T19:45:29.845Z+ retry=2
8072024-03-26T19:45:29.890Z+++ zoneadm list -n
8082024-03-26T19:45:29.890Z+++ grep oxz_ntp
8092024-03-26T19:45:29.890Z++ pfexec svcs -z oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b -Hostate oxide/ntp
8102024-03-26T19:45:29.890Z+ [[ offline != online ]]
8112024-03-26T19:45:29.890Z+ [[ 2 -gt 60 ]]
8122024-03-26T19:45:29.890Z+ sleep 1
8132024-03-26T19:45:30.872Z+ retry=3
8142024-03-26T19:45:30.916Z+++ zoneadm list -n
8152024-03-26T19:45:30.917Z+++ grep oxz_ntp
8162024-03-26T19:45:30.917Z++ pfexec svcs -z oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b -Hostate oxide/ntp
8172024-03-26T19:45:30.917Z+ [[ offline != online ]]
8182024-03-26T19:45:30.917Z+ [[ 3 -gt 60 ]]
8192024-03-26T19:45:30.917Z+ sleep 1
8202024-03-26T19:45:31.890Z+ retry=4
8212024-03-26T19:45:31.915Z+++ zoneadm list -n
8222024-03-26T19:45:31.916Z+++ grep oxz_ntp
8232024-03-26T19:45:31.916Z++ pfexec svcs -z oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b -Hostate oxide/ntp
8242024-03-26T19:45:31.916Z+ [[ offline != online ]]
8252024-03-26T19:45:31.916Z+ [[ 4 -gt 60 ]]
8262024-03-26T19:45:31.916Z+ sleep 1
8272024-03-26T19:45:32.935Z+ retry=5
8282024-03-26T19:45:34.272Z+++ zoneadm list -n
8292024-03-26T19:45:34.272Z+++ grep oxz_ntp
8302024-03-26T19:45:34.272Z++ pfexec svcs -z oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b -Hostate oxide/ntp
8312024-03-26T19:45:34.273Z+ [[ offline != online ]]
8322024-03-26T19:45:34.273Z+ [[ 5 -gt 60 ]]
8332024-03-26T19:45:34.273Z+ sleep 1
8342024-03-26T19:45:34.273Z+ retry=6
8352024-03-26T19:45:34.273Z+++ zoneadm list -n
8362024-03-26T19:45:34.273Z+++ grep oxz_ntp
8372024-03-26T19:45:34.273Z++ pfexec svcs -z oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b -Hostate oxide/ntp
8382024-03-26T19:45:34.273Z+ [[ offline != online ]]
8392024-03-26T19:45:34.273Z+ [[ 6 -gt 60 ]]
8402024-03-26T19:45:34.273Z+ sleep 1
8412024-03-26T19:45:34.957Z+ retry=7
8422024-03-26T19:45:34.983Z+++ zoneadm list -n
8432024-03-26T19:45:34.983Z+++ grep oxz_ntp
8442024-03-26T19:45:34.983Z++ pfexec svcs -z oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b -Hostate oxide/ntp
8452024-03-26T19:45:34.983Z+ [[ offline != online ]]
8462024-03-26T19:45:34.983Z+ [[ 7 -gt 60 ]]
8472024-03-26T19:45:34.983Z+ sleep 1
8482024-03-26T19:45:35.979Z+ retry=8
8492024-03-26T19:45:36.012Z+++ zoneadm list -n
8502024-03-26T19:45:36.012Z+++ grep oxz_ntp
8512024-03-26T19:45:36.012Z++ pfexec svcs -z oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b -Hostate oxide/ntp
8522024-03-26T19:45:36.012Z+ [[ offline != online ]]
8532024-03-26T19:45:36.012Z+ [[ 8 -gt 60 ]]
8542024-03-26T19:45:36.012Z+ sleep 1
8552024-03-26T19:45:37.002Z+ retry=9
 
8562024-03-26T19:45:37.008Zsock console login: Mar 26 19:45:36 sock ip: ip: joining multicasts failed (4) on vopte0 - will use link layer broadcasts for multicast
 
8572024-03-26T19:45:37.029Z+++ zoneadm list -n
8582024-03-26T19:45:37.029Z+++ grep oxz_ntp
8592024-03-26T19:45:37.030Z++ pfexec svcs -z oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b -Hostate oxide/ntp
8602024-03-26T19:45:37.030Z+ [[ online != online ]]
8612024-03-26T19:45:37.030Z+ echo 'Waited for chrony: 9s'
8622024-03-26T19:45:37.030Z+ retry=0
8632024-03-26T19:45:37.030ZWaited for chrony: 9s
8642024-03-26T19:45:37.030Z+ zoneadm list
8652024-03-26T19:45:37.030Z+ grep nexus
8662024-03-26T19:45:37.030Z+ [[ 0 -gt 300 ]]
8672024-03-26T19:45:37.030Z+ sleep 1
8682024-03-26T19:45:38.034Z+ retry=1
8692024-03-26T19:45:38.064Z+ zoneadm list
8702024-03-26T19:45:38.064Z+ grep nexus
8712024-03-26T19:45:38.064Z+ [[ 1 -gt 300 ]]
8722024-03-26T19:45:38.064Z+ sleep 1
8732024-03-26T19:45:39.041Z+ retry=2
8742024-03-26T19:45:39.066Z+ zoneadm list
8752024-03-26T19:45:39.066Z+ grep nexus
8762024-03-26T19:45:39.067Z+ [[ 2 -gt 300 ]]
8772024-03-26T19:45:39.067Z+ sleep 1
8782024-03-26T19:45:40.074Z+ retry=3
8792024-03-26T19:45:40.102Z+ zoneadm list
8802024-03-26T19:45:40.102Z+ grep nexus
8812024-03-26T19:45:40.102Z+ [[ 3 -gt 300 ]]
8822024-03-26T19:45:40.102Z+ sleep 1
8832024-03-26T19:45:41.066Z+ retry=4
8842024-03-26T19:45:41.113Z+ zoneadm list
8852024-03-26T19:45:41.113Z+ grep nexus
8862024-03-26T19:45:41.113Z+ [[ 4 -gt 300 ]]
8872024-03-26T19:45:41.113Z+ sleep 1
8882024-03-26T19:45:42.077Z+ retry=5
8892024-03-26T19:45:42.103Z+ zoneadm list
8902024-03-26T19:45:42.103Z+ grep nexus
8912024-03-26T19:45:42.103Z+ [[ 5 -gt 300 ]]
8922024-03-26T19:45:42.103Z+ sleep 1
8932024-03-26T19:45:43.084Z+ retry=6
8942024-03-26T19:45:43.108Z+ zoneadm list
8952024-03-26T19:45:43.109Z+ grep nexus
8962024-03-26T19:45:43.109Z+ [[ 6 -gt 300 ]]
8972024-03-26T19:45:43.109Z+ sleep 1
8982024-03-26T19:45:44.095Z+ retry=7
8992024-03-26T19:45:44.120Z+ zoneadm list
9002024-03-26T19:45:44.120Z+ grep nexus
9012024-03-26T19:45:44.120Z+ [[ 7 -gt 300 ]]
9022024-03-26T19:45:44.120Z+ sleep 1
9032024-03-26T19:45:45.105Z+ retry=8
9042024-03-26T19:45:45.132Z+ zoneadm list
9052024-03-26T19:45:45.132Z+ grep nexus
9062024-03-26T19:45:45.132Z+ [[ 8 -gt 300 ]]
9072024-03-26T19:45:45.133Z+ sleep 1
9082024-03-26T19:45:46.120Z+ retry=9
9092024-03-26T19:45:46.155Z+ zoneadm list
9102024-03-26T19:45:46.155Z+ grep nexus
9112024-03-26T19:45:46.155Z+ [[ 9 -gt 300 ]]
9122024-03-26T19:45:46.155Z+ sleep 1
9132024-03-26T19:45:47.129Z+ retry=10
9142024-03-26T19:45:47.162Z+ zoneadm list
9152024-03-26T19:45:47.162Z+ grep nexus
9162024-03-26T19:45:47.163Z+ [[ 10 -gt 300 ]]
9172024-03-26T19:45:47.163Z+ sleep 1
9182024-03-26T19:45:48.137Z+ retry=11
9192024-03-26T19:45:48.170Z+ zoneadm list
9202024-03-26T19:45:48.171Z+ grep nexus
9212024-03-26T19:45:48.171Z+ [[ 11 -gt 300 ]]
9222024-03-26T19:45:48.171Z+ sleep 1
9232024-03-26T19:45:49.147Z+ retry=12
9242024-03-26T19:45:49.171Z+ zoneadm list
9252024-03-26T19:45:49.171Z+ grep nexus
9262024-03-26T19:45:49.171Z+ [[ 12 -gt 300 ]]
9272024-03-26T19:45:49.171Z+ sleep 1
9282024-03-26T19:45:50.158Z+ retry=13
9292024-03-26T19:45:50.183Z+ zoneadm list
9302024-03-26T19:45:50.183Z+ grep nexus
9312024-03-26T19:45:50.183Z+ [[ 13 -gt 300 ]]
9322024-03-26T19:45:50.183Z+ sleep 1
9332024-03-26T19:45:51.169Z+ retry=14
9342024-03-26T19:45:51.193Z+ zoneadm list
9352024-03-26T19:45:51.193Z+ grep nexus
9362024-03-26T19:45:51.223Z+ [[ 14 -gt 300 ]]
9372024-03-26T19:45:51.223Z+ sleep 1
9382024-03-26T19:45:52.178Z+ retry=15
9392024-03-26T19:45:52.202Z+ zoneadm list
9402024-03-26T19:45:52.202Z+ grep nexus
9412024-03-26T19:45:52.202Z+ [[ 15 -gt 300 ]]
9422024-03-26T19:45:52.202Z+ sleep 1
9432024-03-26T19:45:53.188Z+ retry=16
9442024-03-26T19:45:53.211Z+ zoneadm list
9452024-03-26T19:45:53.212Z+ grep nexus
9462024-03-26T19:45:53.212Z+ [[ 16 -gt 300 ]]
9472024-03-26T19:45:53.212Z+ sleep 1
9482024-03-26T19:45:54.199Z+ retry=17
9492024-03-26T19:45:54.223Z+ zoneadm list
9502024-03-26T19:45:54.223Z+ grep nexus
9512024-03-26T19:45:54.223Z+ [[ 17 -gt 300 ]]
9522024-03-26T19:45:54.223Z+ sleep 1
9532024-03-26T19:45:55.211Z+ retry=18
9542024-03-26T19:45:55.248Z+ zoneadm list
9552024-03-26T19:45:55.248Z+ grep nexus
9562024-03-26T19:45:55.248Z+ [[ 18 -gt 300 ]]
9572024-03-26T19:45:55.249Z+ sleep 1
9582024-03-26T19:45:56.481Z+ retry=19
9592024-03-26T19:45:56.482Z+ zoneadm list
9602024-03-26T19:45:56.482Z+ grep nexus
9612024-03-26T19:45:56.482Z+ [[ 19 -gt 300 ]]
9622024-03-26T19:45:56.482Z+ sleep 1
9632024-03-26T19:45:58.887Z+ retry=20
9642024-03-26T19:45:58.974Z+ zoneadm list
9652024-03-26T19:45:58.974Z+ grep nexus
9662024-03-26T19:45:58.974Z+ [[ 20 -gt 300 ]]
9672024-03-26T19:45:58.974Z+ sleep 1
9682024-03-26T19:45:58.975Z+ retry=21
9692024-03-26T19:45:58.975Z+ zoneadm list
9702024-03-26T19:45:58.975Z+ grep nexus
9712024-03-26T19:45:58.975Z+ [[ 21 -gt 300 ]]
9722024-03-26T19:45:58.975Z+ sleep 1
9732024-03-26T19:45:59.251Z+ retry=22
9742024-03-26T19:45:59.276Z+ zoneadm list
9752024-03-26T19:45:59.276Z+ grep nexus
9762024-03-26T19:45:59.277Z+ [[ 22 -gt 300 ]]
9772024-03-26T19:45:59.277Z+ sleep 1
9782024-03-26T19:46:00.270Z+ retry=23
9792024-03-26T19:46:00.297Z+ zoneadm list
9802024-03-26T19:46:00.297Z+ grep nexus
9812024-03-26T19:46:00.297Z+ [[ 23 -gt 300 ]]
9822024-03-26T19:46:00.297Z+ sleep 1
9832024-03-26T19:46:01.354Z+ retry=24
9842024-03-26T19:46:01.354Z+ zoneadm list
9852024-03-26T19:46:01.354Z+ grep nexus
9862024-03-26T19:46:01.354Z+ [[ 24 -gt 300 ]]
9872024-03-26T19:46:01.354Z+ sleep 1
9882024-03-26T19:46:02.283Z+ retry=25
9892024-03-26T19:46:02.309Z+ zoneadm list
9902024-03-26T19:46:02.310Z+ grep nexus
9912024-03-26T19:46:02.310Z+ [[ 25 -gt 300 ]]
9922024-03-26T19:46:02.310Z+ sleep 1
9932024-03-26T19:46:03.297Z+ retry=26
9942024-03-26T19:46:03.491Z+ zoneadm list
9952024-03-26T19:46:03.491Z+ grep nexus
9962024-03-26T19:46:03.491Z+ [[ 26 -gt 300 ]]
9972024-03-26T19:46:03.491Z+ sleep 1
9982024-03-26T19:46:04.307Z+ retry=27
9992024-03-26T19:46:04.336Z+ zoneadm list
10002024-03-26T19:46:04.336Z+ grep nexus
10012024-03-26T19:46:04.337Z+ [[ 27 -gt 300 ]]
10022024-03-26T19:46:04.337Z+ sleep 1
10032024-03-26T19:46:05.314Z+ retry=28
10042024-03-26T19:46:05.338Z+ zoneadm list
10052024-03-26T19:46:05.338Z+ grep nexus
10062024-03-26T19:46:05.338Z+ [[ 28 -gt 300 ]]
10072024-03-26T19:46:05.338Z+ sleep 1
10082024-03-26T19:46:06.324Z+ retry=29
10092024-03-26T19:46:06.350Z+ zoneadm list
10102024-03-26T19:46:06.350Z+ grep nexus
10112024-03-26T19:46:06.350Z+ [[ 29 -gt 300 ]]
10122024-03-26T19:46:06.350Z+ sleep 1
10132024-03-26T19:46:07.338Z+ retry=30
10142024-03-26T19:46:07.371Z+ zoneadm list
10152024-03-26T19:46:07.371Z+ grep nexus
10162024-03-26T19:46:07.371Z+ [[ 30 -gt 300 ]]
10172024-03-26T19:46:07.371Z+ sleep 1
10182024-03-26T19:46:08.354Z+ retry=31
10192024-03-26T19:46:08.380Z+ zoneadm list
10202024-03-26T19:46:08.380Z+ grep nexus
10212024-03-26T19:46:08.380Z+ [[ 31 -gt 300 ]]
10222024-03-26T19:46:08.381Z+ sleep 1
10232024-03-26T19:46:09.355Z+ retry=32
10242024-03-26T19:46:09.379Z+ zoneadm list
10252024-03-26T19:46:09.379Z+ grep nexus
10262024-03-26T19:46:09.379Z+ [[ 32 -gt 300 ]]
10272024-03-26T19:46:09.379Z+ sleep 1
10282024-03-26T19:46:10.366Z+ retry=33
10292024-03-26T19:46:10.412Z+ zoneadm list
10302024-03-26T19:46:10.413Z+ grep nexus
10312024-03-26T19:46:10.413Z+ [[ 33 -gt 300 ]]
10322024-03-26T19:46:10.413Z+ sleep 1
10332024-03-26T19:46:11.376Z+ retry=34
10342024-03-26T19:46:11.400Z+ zoneadm list
10352024-03-26T19:46:11.400Z+ grep nexus
10362024-03-26T19:46:11.400Z+ [[ 34 -gt 300 ]]
10372024-03-26T19:46:11.400Z+ sleep 1
10382024-03-26T19:46:12.388Z+ retry=35
10392024-03-26T19:46:12.419Z+ zoneadm list
10402024-03-26T19:46:12.419Z+ grep nexus
10412024-03-26T19:46:12.419Z+ [[ 35 -gt 300 ]]
10422024-03-26T19:46:12.419Z+ sleep 1
10432024-03-26T19:46:13.397Z+ retry=36
10442024-03-26T19:46:13.421Z+ zoneadm list
10452024-03-26T19:46:13.421Z+ grep nexus
10462024-03-26T19:46:13.421Z+ [[ 36 -gt 300 ]]
10472024-03-26T19:46:13.422Z+ sleep 1
10482024-03-26T19:46:14.408Z+ retry=37
10492024-03-26T19:46:14.438Z+ zoneadm list
10502024-03-26T19:46:14.438Z+ grep nexus
10512024-03-26T19:46:14.438Z+ [[ 37 -gt 300 ]]
10522024-03-26T19:46:14.438Z+ sleep 1
10532024-03-26T19:46:15.417Z+ retry=38
10542024-03-26T19:46:15.440Z+ zoneadm list
10552024-03-26T19:46:15.441Z+ grep nexus
10562024-03-26T19:46:15.441Z+ [[ 38 -gt 300 ]]
10572024-03-26T19:46:15.441Z+ sleep 1
10582024-03-26T19:46:16.437Z+ retry=39
10592024-03-26T19:46:16.480Z+ zoneadm list
10602024-03-26T19:46:16.481Z+ grep nexus
10612024-03-26T19:46:16.481Z+ [[ 39 -gt 300 ]]
10622024-03-26T19:46:16.481Z+ sleep 1
10632024-03-26T19:46:17.450Z+ retry=40
10642024-03-26T19:46:17.475Z+ zoneadm list
10652024-03-26T19:46:17.475Z+ grep nexus
10662024-03-26T19:46:17.475Z+ [[ 40 -gt 300 ]]
10672024-03-26T19:46:17.475Z+ sleep 1
10682024-03-26T19:46:18.449Z+ retry=41
10692024-03-26T19:46:18.472Z+ zoneadm list
10702024-03-26T19:46:18.473Z+ grep nexus
10712024-03-26T19:46:18.473Z+ [[ 41 -gt 300 ]]
10722024-03-26T19:46:18.473Z+ sleep 1
10732024-03-26T19:46:19.459Z+ retry=42
10742024-03-26T19:46:19.898Z+ zoneadm list
10752024-03-26T19:46:19.898Z+ grep nexus
10762024-03-26T19:46:19.899Z+ [[ 42 -gt 300 ]]
10772024-03-26T19:46:19.899Z+ sleep 1
10782024-03-26T19:46:20.474Z+ retry=43
10792024-03-26T19:46:20.499Z+ zoneadm list
10802024-03-26T19:46:20.499Z+ grep nexus
10812024-03-26T19:46:20.499Z+ [[ 43 -gt 300 ]]
10822024-03-26T19:46:20.499Z+ sleep 1
10832024-03-26T19:46:21.479Z+ retry=44
10842024-03-26T19:46:21.503Z+ zoneadm list
10852024-03-26T19:46:21.504Z+ grep nexus
10862024-03-26T19:46:21.504Z+ [[ 44 -gt 300 ]]
10872024-03-26T19:46:21.504Z+ sleep 1
10882024-03-26T19:46:22.490Z+ retry=45
10892024-03-26T19:46:22.927Z+ zoneadm list
10902024-03-26T19:46:22.928Z+ grep nexus
10912024-03-26T19:46:22.928Z+ [[ 45 -gt 300 ]]
10922024-03-26T19:46:22.928Z+ sleep 1
10932024-03-26T19:46:23.502Z+ retry=46
10942024-03-26T19:46:23.530Z+ zoneadm list
10952024-03-26T19:46:23.530Z+ grep nexus
10962024-03-26T19:46:23.530Z+ [[ 46 -gt 300 ]]
10972024-03-26T19:46:23.530Z+ sleep 1
10982024-03-26T19:46:24.517Z+ retry=47
10992024-03-26T19:46:24.544Z+ zoneadm list
11002024-03-26T19:46:24.544Z+ grep nexus
11012024-03-26T19:46:24.544Z+ [[ 47 -gt 300 ]]
11022024-03-26T19:46:24.544Z+ sleep 1
11032024-03-26T19:46:25.522Z+ retry=48
11042024-03-26T19:46:25.553Z+ zoneadm list
11052024-03-26T19:46:25.553Z+ grep nexus
11062024-03-26T19:46:25.553Z+ [[ 48 -gt 300 ]]
11072024-03-26T19:46:25.554Z+ sleep 1
11082024-03-26T19:46:27.307Z+ retry=49
11092024-03-26T19:46:27.307Z+ zoneadm list
11102024-03-26T19:46:27.307Z+ grep nexus
11112024-03-26T19:46:27.307Z+ [[ 49 -gt 300 ]]
11122024-03-26T19:46:27.308Z+ sleep 1
11132024-03-26T19:46:27.551Z+ retry=50
11142024-03-26T19:46:27.581Z+ zoneadm list
11152024-03-26T19:46:27.581Z+ grep nexus
11162024-03-26T19:46:27.581Z+ [[ 50 -gt 300 ]]
11172024-03-26T19:46:27.582Z+ sleep 1
11182024-03-26T19:46:28.558Z+ retry=51
11192024-03-26T19:46:28.586Z+ zoneadm list
11202024-03-26T19:46:28.587Z+ grep nexus
11212024-03-26T19:46:28.587Z+ [[ 51 -gt 300 ]]
11222024-03-26T19:46:28.587Z+ sleep 1
11232024-03-26T19:46:29.967Z+ retry=52
11242024-03-26T19:46:30.006Z+ zoneadm list
11252024-03-26T19:46:30.006Z+ grep nexus
11262024-03-26T19:46:30.006Z+ [[ 52 -gt 300 ]]
11272024-03-26T19:46:30.006Z+ sleep 1
11282024-03-26T19:46:30.576Z+ retry=53
11292024-03-26T19:46:30.603Z+ zoneadm list
11302024-03-26T19:46:30.603Z+ grep nexus
11312024-03-26T19:46:30.603Z+ [[ 53 -gt 300 ]]
11322024-03-26T19:46:30.603Z+ sleep 1
11332024-03-26T19:46:31.629Z+ retry=54
11342024-03-26T19:46:31.629Z+ zoneadm list
11352024-03-26T19:46:31.629Z+ grep nexus
11362024-03-26T19:46:31.629Z+ [[ 54 -gt 300 ]]
11372024-03-26T19:46:31.629Z+ sleep 1
11382024-03-26T19:46:32.597Z+ retry=55
11392024-03-26T19:46:32.624Z+ zoneadm list
11402024-03-26T19:46:32.624Z+ grep nexus
11412024-03-26T19:46:32.624Z+ [[ 55 -gt 300 ]]
11422024-03-26T19:46:32.624Z+ sleep 1
11432024-03-26T19:46:33.603Z+ retry=56
11442024-03-26T19:46:33.632Z+ zoneadm list
11452024-03-26T19:46:33.632Z+ grep nexus
11462024-03-26T19:46:33.632Z+ [[ 56 -gt 300 ]]
11472024-03-26T19:46:33.633Z+ sleep 1
11482024-03-26T19:46:34.617Z+ retry=57
11492024-03-26T19:46:34.643Z+ zoneadm list
11502024-03-26T19:46:34.643Z+ grep nexus
11512024-03-26T19:46:34.643Z+ [[ 57 -gt 300 ]]
11522024-03-26T19:46:34.643Z+ sleep 1
11532024-03-26T19:46:35.624Z+ retry=58
11542024-03-26T19:46:35.649Z+ zoneadm list
11552024-03-26T19:46:35.649Z+ grep nexus
11562024-03-26T19:46:35.649Z+ [[ 58 -gt 300 ]]
11572024-03-26T19:46:35.649Z+ sleep 1
11582024-03-26T19:46:36.637Z+ retry=59
11592024-03-26T19:46:36.664Z+ zoneadm list
11602024-03-26T19:46:36.664Z+ grep nexus
11612024-03-26T19:46:36.665Z+ [[ 59 -gt 300 ]]
11622024-03-26T19:46:36.665Z+ sleep 1
11632024-03-26T19:46:37.646Z+ retry=60
11642024-03-26T19:46:37.671Z+ zoneadm list
11652024-03-26T19:46:37.671Z+ grep nexus
11662024-03-26T19:46:37.671Z+ [[ 60 -gt 300 ]]
11672024-03-26T19:46:37.671Z+ sleep 1
11682024-03-26T19:46:38.668Z+ retry=61
11692024-03-26T19:46:38.695Z+ zoneadm list
11702024-03-26T19:46:38.695Z+ grep nexus
11712024-03-26T19:46:38.695Z+ [[ 61 -gt 300 ]]
11722024-03-26T19:46:38.695Z+ sleep 1
11732024-03-26T19:46:44.001Z+ retry=62
11742024-03-26T19:46:44.194Z+ zoneadm list
11752024-03-26T19:46:44.194Z+ grep nexus
11762024-03-26T19:46:44.194Z+ [[ 62 -gt 300 ]]
11772024-03-26T19:46:44.194Z+ sleep 1
11782024-03-26T19:46:44.194Z+ retry=63
11792024-03-26T19:46:44.194Z+ zoneadm list
11802024-03-26T19:46:44.194Z+ grep nexus
11812024-03-26T19:46:44.194Z+ [[ 63 -gt 300 ]]
11822024-03-26T19:46:44.195Z+ sleep 1
11832024-03-26T19:46:44.311Z+ retry=64
11842024-03-26T19:46:44.311Z+ zoneadm list
11852024-03-26T19:46:44.311Z+ grep nexus
11862024-03-26T19:46:44.311Z+ [[ 64 -gt 300 ]]
11872024-03-26T19:46:44.311Z+ sleep 1
11882024-03-26T19:46:44.311Z+ retry=65
11892024-03-26T19:46:44.311Z+ zoneadm list
11902024-03-26T19:46:44.311Z+ grep nexus
11912024-03-26T19:46:44.312Z+ [[ 65 -gt 300 ]]
11922024-03-26T19:46:44.312Z+ sleep 1
11932024-03-26T19:46:44.312Z+ retry=66
11942024-03-26T19:46:44.312Z+ zoneadm list
11952024-03-26T19:46:44.312Z+ grep nexus
11962024-03-26T19:46:44.312Z+ [[ 66 -gt 300 ]]
11972024-03-26T19:46:44.312Z+ sleep 1
11982024-03-26T19:46:44.746Z+ retry=67
11992024-03-26T19:46:44.786Z+ zoneadm list
12002024-03-26T19:46:44.786Z+ grep nexus
12012024-03-26T19:46:44.786Z+ [[ 67 -gt 300 ]]
12022024-03-26T19:46:44.786Z+ sleep 1
12032024-03-26T19:46:45.728Z+ retry=68
12042024-03-26T19:46:45.757Z+ zoneadm list
12052024-03-26T19:46:45.757Z+ grep nexus
12062024-03-26T19:46:45.757Z+ [[ 68 -gt 300 ]]
12072024-03-26T19:46:45.757Z+ sleep 1
12082024-03-26T19:46:46.739Z+ retry=69
12092024-03-26T19:46:46.764Z+ zoneadm list
12102024-03-26T19:46:46.764Z+ grep nexus
12112024-03-26T19:46:46.764Z+ [[ 69 -gt 300 ]]
12122024-03-26T19:46:46.764Z+ sleep 1
12132024-03-26T19:46:47.755Z+ retry=70
12142024-03-26T19:46:47.783Z+ zoneadm list
12152024-03-26T19:46:47.783Z+ grep nexus
12162024-03-26T19:46:47.784Z+ [[ 70 -gt 300 ]]
12172024-03-26T19:46:47.784Z+ sleep 1
12182024-03-26T19:46:48.760Z+ retry=71
12192024-03-26T19:46:48.789Z+ zoneadm list
12202024-03-26T19:46:48.790Z+ grep nexus
12212024-03-26T19:46:48.790Z+ [[ 71 -gt 300 ]]
12222024-03-26T19:46:48.790Z+ sleep 1
12232024-03-26T19:46:49.770Z+ retry=72
12242024-03-26T19:46:49.795Z+ zoneadm list
12252024-03-26T19:46:49.795Z+ grep nexus
12262024-03-26T19:46:49.795Z+ [[ 72 -gt 300 ]]
12272024-03-26T19:46:49.795Z+ sleep 1
12282024-03-26T19:46:50.785Z+ retry=73
12292024-03-26T19:46:50.809Z+ zoneadm list
12302024-03-26T19:46:50.809Z+ grep nexus
12312024-03-26T19:46:50.809Z+ [[ 73 -gt 300 ]]
12322024-03-26T19:46:50.809Z+ sleep 1
12332024-03-26T19:46:51.795Z+ retry=74
12342024-03-26T19:46:51.820Z+ zoneadm list
12352024-03-26T19:46:51.820Z+ grep nexus
12362024-03-26T19:46:51.820Z+ [[ 74 -gt 300 ]]
12372024-03-26T19:46:51.820Z+ sleep 1
12382024-03-26T19:46:52.808Z+ retry=75
12392024-03-26T19:46:52.834Z+ zoneadm list
12402024-03-26T19:46:52.835Z+ grep nexus
12412024-03-26T19:46:52.835Z+ [[ 75 -gt 300 ]]
12422024-03-26T19:46:52.835Z+ sleep 1
12432024-03-26T19:46:53.888Z+ retry=76
12442024-03-26T19:46:53.889Z+ zoneadm list
12452024-03-26T19:46:53.889Z+ grep nexus
12462024-03-26T19:46:53.916Z+ [[ 76 -gt 300 ]]
12472024-03-26T19:46:53.916Z+ sleep 1
12482024-03-26T19:46:54.918Z+ retry=77
12492024-03-26T19:46:54.946Z+ grep nexus
12502024-03-26T19:46:54.946Z+ zoneadm list
12512024-03-26T19:46:54.971Z+ [[ 77 -gt 300 ]]
12522024-03-26T19:46:54.971Z+ sleep 1
12532024-03-26T19:46:55.971Z+ retry=78
12542024-03-26T19:46:56.002Z+ zoneadm list
12552024-03-26T19:46:56.003Z+ grep nexus
12562024-03-26T19:46:56.003Z+ [[ 78 -gt 300 ]]
12572024-03-26T19:46:56.003Z+ sleep 1
12582024-03-26T19:46:56.981Z+ retry=79
12592024-03-26T19:46:57.004Z+ zoneadm list
12602024-03-26T19:46:57.004Z+ grep nexus
12612024-03-26T19:46:57.004Z+ [[ 79 -gt 300 ]]
12622024-03-26T19:46:57.004Z+ sleep 1
12632024-03-26T19:46:57.994Z+ retry=80
12642024-03-26T19:46:58.022Z+ zoneadm list
12652024-03-26T19:46:58.022Z+ grep nexus
12662024-03-26T19:46:58.022Z+ [[ 80 -gt 300 ]]
12672024-03-26T19:46:58.023Z+ sleep 1
12682024-03-26T19:46:59.008Z+ retry=81
12692024-03-26T19:46:59.032Z+ zoneadm list
12702024-03-26T19:46:59.032Z+ grep nexus
12712024-03-26T19:46:59.032Z+ [[ 81 -gt 300 ]]
12722024-03-26T19:46:59.032Z+ sleep 1
12732024-03-26T19:47:00.049Z+ retry=82
12742024-03-26T19:47:00.081Z+ zoneadm list
12752024-03-26T19:47:00.082Z+ grep nexus
12762024-03-26T19:47:00.082Z+ [[ 82 -gt 300 ]]
12772024-03-26T19:47:00.082Z+ sleep 1
12782024-03-26T19:47:01.039Z+ retry=83
12792024-03-26T19:47:01.070Z+ zoneadm list
12802024-03-26T19:47:01.070Z+ grep nexus
12812024-03-26T19:47:01.070Z+ [[ 83 -gt 300 ]]
12822024-03-26T19:47:01.070Z+ sleep 1
12832024-03-26T19:47:02.041Z+ retry=84
12842024-03-26T19:47:02.066Z+ zoneadm list
12852024-03-26T19:47:02.066Z+ grep nexus
12862024-03-26T19:47:02.066Z+ [[ 84 -gt 300 ]]
12872024-03-26T19:47:02.066Z+ sleep 1
12882024-03-26T19:47:03.054Z+ retry=85
12892024-03-26T19:47:03.097Z+ zoneadm list
12902024-03-26T19:47:03.098Z+ grep nexus
12912024-03-26T19:47:03.098Z+ [[ 85 -gt 300 ]]
12922024-03-26T19:47:03.098Z+ sleep 1
12932024-03-26T19:47:04.079Z+ retry=86
12942024-03-26T19:47:04.581Z+ zoneadm list
12952024-03-26T19:47:04.581Z+ grep nexus
12962024-03-26T19:47:04.581Z+ [[ 86 -gt 300 ]]
12972024-03-26T19:47:04.581Z+ sleep 1
12982024-03-26T19:47:05.079Z+ retry=87
12992024-03-26T19:47:05.103Z+ zoneadm list
13002024-03-26T19:47:05.103Z+ grep nexus
13012024-03-26T19:47:05.103Z+ [[ 87 -gt 300 ]]
13022024-03-26T19:47:05.103Z+ sleep 1
13032024-03-26T19:47:06.092Z+ retry=88
13042024-03-26T19:47:06.117Z+ zoneadm list
13052024-03-26T19:47:06.117Z+ grep nexus
13062024-03-26T19:47:06.117Z+ [[ 88 -gt 300 ]]
13072024-03-26T19:47:06.117Z+ sleep 1
13082024-03-26T19:47:07.107Z+ retry=89
13092024-03-26T19:47:07.133Z+ zoneadm list
13102024-03-26T19:47:07.133Z+ grep nexus
13112024-03-26T19:47:07.133Z+ [[ 89 -gt 300 ]]
13122024-03-26T19:47:07.133Z+ sleep 1
13132024-03-26T19:47:08.120Z+ retry=90
13142024-03-26T19:47:08.146Z+ zoneadm list
13152024-03-26T19:47:08.146Z+ grep nexus
13162024-03-26T19:47:08.146Z+ [[ 90 -gt 300 ]]
13172024-03-26T19:47:08.146Z+ sleep 1
13182024-03-26T19:47:09.128Z+ retry=91
13192024-03-26T19:47:09.151Z+ zoneadm list
13202024-03-26T19:47:09.151Z+ grep nexus
13212024-03-26T19:47:09.151Z+ [[ 91 -gt 300 ]]
13222024-03-26T19:47:09.151Z+ sleep 1
13232024-03-26T19:47:10.144Z+ retry=92
13242024-03-26T19:47:10.176Z+ zoneadm list
13252024-03-26T19:47:10.176Z+ grep nexus
13262024-03-26T19:47:10.176Z+ [[ 92 -gt 300 ]]
13272024-03-26T19:47:10.176Z+ sleep 1
13282024-03-26T19:47:11.150Z+ retry=93
13292024-03-26T19:47:11.182Z+ zoneadm list
13302024-03-26T19:47:11.182Z+ grep nexus
13312024-03-26T19:47:11.183Z+ [[ 93 -gt 300 ]]
13322024-03-26T19:47:11.183Z+ sleep 1
13332024-03-26T19:47:12.164Z+ retry=94
13342024-03-26T19:47:12.207Z+ zoneadm list
13352024-03-26T19:47:12.207Z+ grep nexus
13362024-03-26T19:47:12.207Z+ [[ 94 -gt 300 ]]
13372024-03-26T19:47:12.207Z+ sleep 1
13382024-03-26T19:47:13.176Z+ retry=95
13392024-03-26T19:47:13.200Z+ zoneadm list
13402024-03-26T19:47:13.200Z+ grep nexus
13412024-03-26T19:47:13.200Z+ [[ 95 -gt 300 ]]
13422024-03-26T19:47:13.200Z+ sleep 1
13432024-03-26T19:47:15.860Z+ retry=96
13442024-03-26T19:47:15.896Z+ zoneadm list
13452024-03-26T19:47:15.896Z+ grep nexus
13462024-03-26T19:47:15.920Z+ [[ 96 -gt 300 ]]
13472024-03-26T19:47:15.920Z+ sleep 1
13482024-03-26T19:47:16.914Z+ retry=97
13492024-03-26T19:47:16.939Z+ zoneadm list
13502024-03-26T19:47:16.940Z+ grep nexus
13512024-03-26T19:47:16.940Z+ [[ 97 -gt 300 ]]
13522024-03-26T19:47:16.940Z+ sleep 1
13532024-03-26T19:47:17.933Z+ retry=98
13542024-03-26T19:47:17.958Z+ zoneadm list
13552024-03-26T19:47:17.958Z+ grep nexus
13562024-03-26T19:47:17.958Z+ [[ 98 -gt 300 ]]
13572024-03-26T19:47:17.958Z+ sleep 1
13582024-03-26T19:47:18.949Z+ retry=99
13592024-03-26T19:47:18.974Z+ zoneadm list
13602024-03-26T19:47:18.974Z+ grep nexus
13612024-03-26T19:47:18.974Z+ [[ 99 -gt 300 ]]
13622024-03-26T19:47:18.975Z+ sleep 1
13632024-03-26T19:47:20.383Z+ retry=100
13642024-03-26T19:47:20.415Z+ zoneadm list
13652024-03-26T19:47:20.415Z+ grep nexus
13662024-03-26T19:47:20.415Z+ [[ 100 -gt 300 ]]
13672024-03-26T19:47:20.415Z+ sleep 1
13682024-03-26T19:47:20.990Z+ retry=101
13692024-03-26T19:47:21.019Z+ zoneadm list
13702024-03-26T19:47:21.020Z+ grep nexus
13712024-03-26T19:47:21.020Z+ [[ 101 -gt 300 ]]
13722024-03-26T19:47:21.020Z+ sleep 1
13732024-03-26T19:47:21.995Z+ retry=102
13742024-03-26T19:47:22.021Z+ zoneadm list
13752024-03-26T19:47:22.022Z+ grep nexus
13762024-03-26T19:47:22.022Z+ [[ 102 -gt 300 ]]
13772024-03-26T19:47:22.022Z+ sleep 1
13782024-03-26T19:47:23.010Z+ retry=103
13792024-03-26T19:47:23.036Z+ zoneadm list
13802024-03-26T19:47:23.036Z+ grep nexus
13812024-03-26T19:47:23.037Z+ [[ 103 -gt 300 ]]
13822024-03-26T19:47:23.037Z+ sleep 1
13832024-03-26T19:47:24.023Z+ retry=104
13842024-03-26T19:47:24.048Z+ zoneadm list
13852024-03-26T19:47:24.049Z+ grep nexus
13862024-03-26T19:47:24.049Z+ [[ 104 -gt 300 ]]
13872024-03-26T19:47:24.049Z+ sleep 1
13882024-03-26T19:47:25.033Z+ retry=105
13892024-03-26T19:47:25.058Z+ zoneadm list
13902024-03-26T19:47:25.058Z+ grep nexus
13912024-03-26T19:47:25.058Z+ [[ 105 -gt 300 ]]
13922024-03-26T19:47:25.058Z+ sleep 1
13932024-03-26T19:47:26.046Z+ retry=106
13942024-03-26T19:47:26.070Z+ zoneadm list
13952024-03-26T19:47:26.070Z+ grep nexus
13962024-03-26T19:47:26.070Z+ [[ 106 -gt 300 ]]
13972024-03-26T19:47:26.070Z+ sleep 1
13982024-03-26T19:47:27.058Z+ retry=107
13992024-03-26T19:47:27.494Z+ zoneadm list
14002024-03-26T19:47:27.494Z+ grep nexus
14012024-03-26T19:47:27.494Z+ [[ 107 -gt 300 ]]
14022024-03-26T19:47:27.494Z+ sleep 1
14032024-03-26T19:47:28.080Z+ retry=108
14042024-03-26T19:47:28.105Z+ zoneadm list
14052024-03-26T19:47:28.105Z+ grep nexus
14062024-03-26T19:47:28.105Z+ [[ 108 -gt 300 ]]
14072024-03-26T19:47:28.105Z+ sleep 1
14082024-03-26T19:47:29.085Z+ retry=109
14092024-03-26T19:47:29.118Z+ zoneadm list
14102024-03-26T19:47:29.119Z+ grep nexus
14112024-03-26T19:47:29.119Z+ [[ 109 -gt 300 ]]
14122024-03-26T19:47:29.119Z+ sleep 1
14132024-03-26T19:47:30.101Z+ retry=110
14142024-03-26T19:47:30.136Z+ zoneadm list
14152024-03-26T19:47:30.136Z+ grep nexus
14162024-03-26T19:47:30.136Z+ [[ 110 -gt 300 ]]
14172024-03-26T19:47:30.136Z+ sleep 1
14182024-03-26T19:47:31.148Z+ retry=111
14192024-03-26T19:47:31.173Z+ zoneadm list
14202024-03-26T19:47:31.173Z+ grep nexus
14212024-03-26T19:47:31.173Z+ [[ 111 -gt 300 ]]
14222024-03-26T19:47:31.173Z+ sleep 1
14232024-03-26T19:47:42.144Z+ retry=112
14242024-03-26T19:47:42.144Z+ zoneadm list
14252024-03-26T19:47:42.609Z+ grep nexus
14262024-03-26T19:47:42.609Z+ [[ 112 -gt 300 ]]
14272024-03-26T19:47:42.610Z+ sleep 1
14282024-03-26T19:47:42.610Z+ retry=113
14292024-03-26T19:47:42.610Z+ zoneadm list
14302024-03-26T19:47:42.610Z+ grep nexus
14312024-03-26T19:47:42.610Z+ [[ 113 -gt 300 ]]
14322024-03-26T19:47:42.611Z+ sleep 1
14332024-03-26T19:47:42.611Z+ retry=114
14342024-03-26T19:47:42.611Z+ zoneadm list
14352024-03-26T19:47:42.611Z+ grep nexus
14362024-03-26T19:47:42.611Z+ [[ 114 -gt 300 ]]
14372024-03-26T19:47:42.611Z+ sleep 1
14382024-03-26T19:47:42.611Z+ retry=115
14392024-03-26T19:47:42.612Z+ zoneadm list
14402024-03-26T19:47:42.612Z+ grep nexus
14412024-03-26T19:47:42.612Z+ [[ 115 -gt 300 ]]
14422024-03-26T19:47:42.612Z+ sleep 1
14432024-03-26T19:47:42.682Z+ retry=116
14442024-03-26T19:47:42.683Z+ zoneadm list
14452024-03-26T19:47:42.683Z+ grep nexus
14462024-03-26T19:47:42.683Z+ echo 'Waited for nexus: 116s'
14472024-03-26T19:47:42.683Z+ export RUST_BACKTRACE=1
14482024-03-26T19:47:42.684Z+ RUST_BACKTRACE=1
14492024-03-26T19:47:42.684Z+ export E2E_TLS_CERT IPPOOL_START IPPOOL_END
14502024-03-26T19:47:42.684Z++ ./tests/bootstrap
14512024-03-26T19:47:43.602ZUsing external DNS server at 10.151.2.164:53
14522024-03-26T19:47:43.913Z2024-03-26 19:47:43.904160168 UTC: attempting to log into API
14532024-03-26T19:47:44.019Zoxz_nexus_9548a077-d4f7-452a-ad05-2a2815dce2ee
14542024-03-26T19:47:44.019Zoxz_nexus_95cff384-74b1-4ecb-8cbf-cab77d39b166
14552024-03-26T19:47:44.019ZWaited for nexus: 116s
14562024-03-26T19:47:59.030Z2024-03-26 19:47:58.996217907 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
14572024-03-26T19:48:00.009Z2024-03-26 19:48:00.002530680 UTC: attempting to log into API
 
14582024-03-26T19:48:04.489ZMar 26 19:48:03 sock ip: ip: joining multicasts failed (4) on vopte5 - will use link layer broadcasts for multicast
14592024-03-26T19:48:04.544ZMar 26 19:48:03 sock ip: ip: joining multicasts failed (4) on vopte3 - will use link layer broadcasts for multicast
14602024-03-26T19:48:04.587ZMar 26 19:48:03 sock ip: ip: joining multicasts failed (4) on vopte1 - will use link layer broadcasts for multicast
14612024-03-26T19:48:04.630ZMar 26 19:48:04 sock ip: ip: joining multicasts failed (4) on vopte2 - will use link layer broadcasts for multicast
14622024-03-26T19:48:04.950ZMar 26 19:48:04 sock ip: ip: joining multicasts failed (4) on vopte4 - will use link layer broadcasts for multicast
 
14632024-03-26T19:48:15.087Z2024-03-26 19:48:15.079398011 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
14642024-03-26T19:48:16.087Z2024-03-26 19:48:16.080491136 UTC: attempting to log into API
14652024-03-26T19:48:31.120Z2024-03-26 19:48:31.112574672 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
14662024-03-26T19:48:32.121Z2024-03-26 19:48:32.113767809 UTC: attempting to log into API
14672024-03-26T19:48:47.160Z2024-03-26 19:48:47.145669168 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
14682024-03-26T19:48:48.154Z2024-03-26 19:48:48.147742473 UTC: attempting to log into API
14692024-03-26T19:48:48.969Z2024-03-26 19:48:48.955061075 UTC: login succeeded
14702024-03-26T19:48:48.995Zwaiting for nexus to come up...
14712024-03-26T19:48:49.087Zcreating IP pool... 10.151.2.175 - 10.151.2.227
14722024-03-26T19:48:49.321Zsetting up quotas...
14732024-03-26T19:48:49.354Zensuring datasets are ready...
14742024-03-26T19:48:52.098Zlet's roll.
14752024-03-26T19:48:52.137Z+ eval 'OXIDE_HOST=https://recovery.sys.oxide.test
14762024-03-26T19:48:52.137ZOXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.2.166
14772024-03-26T19:48:52.137ZOXIDE_TOKEN=oxide-token-310b2195553467619555821460fe67bacc3c9e08'
14782024-03-26T19:48:52.137Z++ OXIDE_HOST=https://recovery.sys.oxide.test
14792024-03-26T19:48:52.137Z++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.2.166
14802024-03-26T19:48:52.137Z++ OXIDE_TOKEN=oxide-token-310b2195553467619555821460fe67bacc3c9e08
14812024-03-26T19:48:52.138Z+ export OXIDE_HOST OXIDE_TOKEN
14822024-03-26T19:48:52.138Z+ retry=0
14832024-03-26T19:48:52.138Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
14842024-03-26T19:48:52.165Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
14852024-03-26T19:48:52.165Z+ [[ 0 -gt 60 ]]
14862024-03-26T19:48:52.165Z+ sleep 1
14872024-03-26T19:48:53.138Z+ retry=1
14882024-03-26T19:48:53.138Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
14892024-03-26T19:48:53.162Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
14902024-03-26T19:48:53.163Z+ [[ 1 -gt 60 ]]
14912024-03-26T19:48:53.163Z+ sleep 1
14922024-03-26T19:48:54.155Z+ retry=2
14932024-03-26T19:48:54.155Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
14942024-03-26T19:48:54.179Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
14952024-03-26T19:48:54.179Z+ [[ 2 -gt 60 ]]
14962024-03-26T19:48:54.179Z+ sleep 1
14972024-03-26T19:48:55.185Z+ retry=3
14982024-03-26T19:48:55.185Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
14992024-03-26T19:48:55.211Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15002024-03-26T19:48:55.211Z+ [[ 3 -gt 60 ]]
15012024-03-26T19:48:55.211Z+ sleep 1
15022024-03-26T19:48:56.196Z+ retry=4
15032024-03-26T19:48:56.197Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15042024-03-26T19:48:56.224Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15052024-03-26T19:48:56.224Z+ [[ 4 -gt 60 ]]
15062024-03-26T19:48:56.224Z+ sleep 1
15072024-03-26T19:48:57.204Z+ retry=5
15082024-03-26T19:48:57.204Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15092024-03-26T19:48:57.228Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15102024-03-26T19:48:57.228Z+ [[ 5 -gt 60 ]]
15112024-03-26T19:48:57.228Z+ sleep 1
15122024-03-26T19:48:58.221Z+ retry=6
15132024-03-26T19:48:58.221Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15142024-03-26T19:48:58.246Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15152024-03-26T19:48:58.246Z+ [[ 6 -gt 60 ]]
15162024-03-26T19:48:58.246Z+ sleep 1
15172024-03-26T19:48:59.247Z+ retry=7
15182024-03-26T19:48:59.247Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15192024-03-26T19:48:59.271Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15202024-03-26T19:48:59.271Z+ [[ 7 -gt 60 ]]
15212024-03-26T19:48:59.271Z+ sleep 1
15222024-03-26T19:49:00.261Z+ retry=8
15232024-03-26T19:49:00.261Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15242024-03-26T19:49:00.287Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15252024-03-26T19:49:00.287Z+ [[ 8 -gt 60 ]]
15262024-03-26T19:49:00.287Z+ sleep 1
15272024-03-26T19:49:01.269Z+ retry=9
15282024-03-26T19:49:01.269Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15292024-03-26T19:49:01.301Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15302024-03-26T19:49:01.301Z+ [[ 9 -gt 60 ]]
15312024-03-26T19:49:01.302Z+ sleep 1
15322024-03-26T19:49:02.287Z+ retry=10
15332024-03-26T19:49:02.287Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15342024-03-26T19:49:02.313Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15352024-03-26T19:49:02.314Z+ [[ 10 -gt 60 ]]
15362024-03-26T19:49:02.314Z+ sleep 1
15372024-03-26T19:49:03.344Z+ retry=11
15382024-03-26T19:49:03.344Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15392024-03-26T19:49:03.381Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15402024-03-26T19:49:03.382Z+ [[ 11 -gt 60 ]]
15412024-03-26T19:49:03.382Z+ sleep 1
15422024-03-26T19:49:04.328Z+ retry=12
15432024-03-26T19:49:04.328Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15442024-03-26T19:49:04.366Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15452024-03-26T19:49:04.367Z+ [[ 12 -gt 60 ]]
15462024-03-26T19:49:04.367Z+ sleep 1
15472024-03-26T19:49:05.451Z+ retry=13
15482024-03-26T19:49:05.451Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15492024-03-26T19:49:05.515Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15502024-03-26T19:49:05.515Z+ [[ 13 -gt 60 ]]
15512024-03-26T19:49:05.516Z+ sleep 1
15522024-03-26T19:49:06.351Z+ retry=14
15532024-03-26T19:49:06.383Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15542024-03-26T19:49:06.383Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15552024-03-26T19:49:06.383Z+ [[ 14 -gt 60 ]]
15562024-03-26T19:49:06.384Z+ sleep 1
15572024-03-26T19:49:08.748Z+ retry=15
15582024-03-26T19:49:08.748Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15592024-03-26T19:49:08.808Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15602024-03-26T19:49:08.808Z+ [[ 15 -gt 60 ]]
15612024-03-26T19:49:08.808Z+ sleep 1
15622024-03-26T19:49:08.808Z+ retry=16
15632024-03-26T19:49:08.809Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15642024-03-26T19:49:08.809Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15652024-03-26T19:49:08.809Z+ [[ 16 -gt 60 ]]
15662024-03-26T19:49:08.809Z+ sleep 1
15672024-03-26T19:49:09.398Z+ retry=17
15682024-03-26T19:49:09.398Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15692024-03-26T19:49:09.422Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Couldn't connect to server
15702024-03-26T19:49:09.422Z+ [[ 17 -gt 60 ]]
15712024-03-26T19:49:09.422Z+ sleep 1
15722024-03-26T19:49:10.419Z+ retry=18
15732024-03-26T19:49:10.419Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
15742024-03-26T19:49:11.631Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images'
15752024-03-26T19:49:12.017Z{"status":"ok"}success
15762024-03-26T19:49:12.017ZProject {
15772024-03-26T19:49:12.017Z description: "some images",
15782024-03-26T19:49:12.017Z id: 92b5aa50-1fbd-4a21-a812-03164945265b,
15792024-03-26T19:49:12.017Z name: Name(
15802024-03-26T19:49:12.017Z "images",
15812024-03-26T19:49:12.017Z ),
15822024-03-26T19:49:12.067Z time_created: 2024-03-26T19:49:11.754764Z,
15832024-03-26T19:49:12.068Z time_modified: 2024-03-26T19:49:11.754764Z,
15842024-03-26T19:49:12.068Z}
15852024-03-26T19:49:12.068Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11
15862024-03-26T19:54:44.577Zdone!
15872024-03-26T19:54:45.101Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11
15882024-03-26T19:54:45.133Zsuccess
15892024-03-26T19:54:45.133ZImage {
15902024-03-26T19:54:45.133Z block_size: ByteCount(
15912024-03-26T19:54:45.133Z 512,
15922024-03-26T19:54:45.133Z ),
15932024-03-26T19:54:45.134Z description: "debian 11 original base image",
15942024-03-26T19:54:45.134Z digest: None,
15952024-03-26T19:54:45.134Z id: 26a3ed30-f8f7-480a-94fd-33e3400a2c6d,
15962024-03-26T19:54:45.134Z name: Name(
15972024-03-26T19:54:45.134Z "debian11",
15982024-03-26T19:54:45.134Z ),
15992024-03-26T19:54:45.134Z os: "debian",
16002024-03-26T19:54:45.134Z project_id: None,
16012024-03-26T19:54:45.134Z size: ByteCount(
16022024-03-26T19:54:45.134Z 2147483648,
16032024-03-26T19:54:45.134Z ),
16042024-03-26T19:54:45.135Z time_created: 2024-03-26T19:54:44.541025Z,
16052024-03-26T19:54:45.135Z time_modified: 2024-03-26T19:54:44.774522Z,
16062024-03-26T19:54:45.135Z url: None,
16072024-03-26T19:54:45.135Z version: "11",
16082024-03-26T19:54:45.135Z}
16092024-03-26T19:54:45.135Z+ rm ./tests/bootstrap
16102024-03-26T19:54:45.135Z+ for test_bin in tests/*
16112024-03-26T19:54:45.135Z+ ./tests/bootstrap-3cb81999e7440661
16122024-03-26T19:54:45.135Z
16132024-03-26T19:54:45.135Zrunning 0 tests
16142024-03-26T19:54:45.135Z
16152024-03-26T19:54:45.136Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
16162024-03-26T19:54:45.136Z
16172024-03-26T19:54:45.136Z+ for test_bin in tests/*
16182024-03-26T19:54:45.136Z+ ./tests/commtest-e25b00f6d79ff279
16192024-03-26T19:54:45.136Z
16202024-03-26T19:54:45.136Zrunning 0 tests
16212024-03-26T19:54:45.136Z
16222024-03-26T19:54:45.136Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
16232024-03-26T19:54:45.136Z
16242024-03-26T19:54:45.136Z+ for test_bin in tests/*
16252024-03-26T19:54:45.136Z+ ./tests/dhcp_server-cdb0b653f05d35f0
16262024-03-26T19:54:45.136Z
16272024-03-26T19:54:45.137Zrunning 0 tests
16282024-03-26T19:54:45.137Z
16292024-03-26T19:54:45.137Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
16302024-03-26T19:54:45.137Z
16312024-03-26T19:54:45.137Z+ for test_bin in tests/*
16322024-03-26T19:54:45.137Z+ ./tests/end_to_end_tests-92ca3720317208b2
16332024-03-26T19:54:45.362Z
16342024-03-26T19:54:45.362Zrunning 2 tests
16352024-03-26T19:54:45.634Ztest no_spoof::no_spoof_here ... ok
16362024-03-26T19:55:11.234Ztest instance_launch::instance_launch ... ok
16372024-03-26T19:55:11.234Z
16382024-03-26T19:55:11.268Ztest result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 25.87s
16392024-03-26T19:55:11.268Z
16402024-03-26T19:55:11.268Z+ _exit_trap
16412024-03-26T19:55:11.268Z+ local status=0
16422024-03-26T19:55:11.268Z+ [[ 0 -eq 0 ]]
16432024-03-26T19:55:11.268Z+ exit 0
16442024-03-26T19:55:12.016Zprocess exited: duration 678362 ms, exit code 0
 
16452024-03-26T19:55:12.218Zfound 100 output files
16462024-03-26T19:55:12.218Zuploading: /var/svc/log/oxide-sled-agent:default.log (1224162 bytes)
16472024-03-26T19:55:15.612Zuploaded: /var/svc/log/oxide-sled-agent:default.log
16482024-03-26T19:55:15.613Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log (2701952 bytes)
16492024-03-26T19:55:21.150Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log" changed size mid upload: 2701952 -> 2738775
16502024-03-26T19:55:22.240Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log
16512024-03-26T19:55:22.240Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log (52288 bytes)
16522024-03-26T19:55:23.846Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log
16532024-03-26T19:55:23.846Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log (7906 bytes)
16542024-03-26T19:55:24.564Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log
16552024-03-26T19:55:24.564Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log (57597 bytes)
16562024-03-26T19:55:32.331Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log
16572024-03-26T19:55:32.331Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log (1339909 bytes)
16582024-03-26T19:55:33.677Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log" changed size mid upload: 1339909 -> 1387813
16592024-03-26T19:55:34.751Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log
16602024-03-26T19:55:34.751Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log (761703 bytes)
16612024-03-26T19:55:35.503Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log" changed size mid upload: 761703 -> 792843
16622024-03-26T19:55:36.596Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log
16632024-03-26T19:55:36.596Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log (1388 bytes)
16642024-03-26T19:55:37.697Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log
16652024-03-26T19:55:37.697Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log (287437 bytes)
16662024-03-26T19:55:38.495Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log" changed size mid upload: 287437 -> 299548
16672024-03-26T19:55:39.991Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log
16682024-03-26T19:55:39.991Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log (7832 bytes)
16692024-03-26T19:55:40.123Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log" changed size mid upload: 7832 -> 8417
16702024-03-26T19:55:41.308Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log
16712024-03-26T19:55:41.308Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log (5393 bytes)
16722024-03-26T19:55:42.310Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log
16732024-03-26T19:55:42.336Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_d23596a2-04e5-4ac6-8b8f-4c1301643047/root/var/svc/log/oxide-clickhouse:default.log (2200992 bytes)
16742024-03-26T19:55:45.455Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_d23596a2-04e5-4ac6-8b8f-4c1301643047/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 2200992 -> 2382253
16752024-03-26T19:55:46.529Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_d23596a2-04e5-4ac6-8b8f-4c1301643047/root/var/svc/log/oxide-clickhouse:default.log
16762024-03-26T19:55:46.529Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_d23596a2-04e5-4ac6-8b8f-4c1301643047/root/var/svc/log/oxide-zone-network-setup:default.log (1714 bytes)
16772024-03-26T19:55:47.627Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_d23596a2-04e5-4ac6-8b8f-4c1301643047/root/var/svc/log/oxide-zone-network-setup:default.log
16782024-03-26T19:55:47.651Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_90b22052-d746-485e-980b-ea6c77d9d3ef/root/var/svc/log/oxide-cockroachdb:default.log (4405 bytes)
16792024-03-26T19:55:48.741Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_90b22052-d746-485e-980b-ea6c77d9d3ef/root/var/svc/log/oxide-cockroachdb:default.log
16802024-03-26T19:55:48.741Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_90b22052-d746-485e-980b-ea6c77d9d3ef/root/var/svc/log/oxide-zone-network-setup:default.log (1716 bytes)
16812024-03-26T19:55:49.840Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_90b22052-d746-485e-980b-ea6c77d9d3ef/root/var/svc/log/oxide-zone-network-setup:default.log
16822024-03-26T19:55:49.840Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c1dcb7b7-fa22-4118-a989-ee55a72ed12c/root/var/svc/log/oxide-crucible-agent:default.log (23602 bytes)
16832024-03-26T19:55:50.981Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c1dcb7b7-fa22-4118-a989-ee55a72ed12c/root/var/svc/log/oxide-crucible-agent:default.log
16842024-03-26T19:55:50.982Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c1dcb7b7-fa22-4118-a989-ee55a72ed12c/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
16852024-03-26T19:55:52.082Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c1dcb7b7-fa22-4118-a989-ee55a72ed12c/root/var/svc/log/oxide-crucible-downstairs:default.log
16862024-03-26T19:55:52.082Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c1dcb7b7-fa22-4118-a989-ee55a72ed12c/root/var/svc/log/oxide-crucible-downstairs:downstairs-a3278a85-6d25-4b97-b07e-461b2764ef24.log (5613 bytes)
16872024-03-26T19:55:53.238Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c1dcb7b7-fa22-4118-a989-ee55a72ed12c/root/var/svc/log/oxide-crucible-downstairs:downstairs-a3278a85-6d25-4b97-b07e-461b2764ef24.log
16882024-03-26T19:55:53.238Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c1dcb7b7-fa22-4118-a989-ee55a72ed12c/root/var/svc/log/oxide-crucible-downstairs:downstairs-e80d0166-7d07-411e-8fcd-ef84e168afd3.log (3020 bytes)
16892024-03-26T19:55:54.349Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c1dcb7b7-fa22-4118-a989-ee55a72ed12c/root/var/svc/log/oxide-crucible-downstairs:downstairs-e80d0166-7d07-411e-8fcd-ef84e168afd3.log
16902024-03-26T19:55:54.350Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c1dcb7b7-fa22-4118-a989-ee55a72ed12c/root/var/svc/log/oxide-zone-network-setup:default.log (1702 bytes)
16912024-03-26T19:55:55.459Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c1dcb7b7-fa22-4118-a989-ee55a72ed12c/root/var/svc/log/oxide-zone-network-setup:default.log
16922024-03-26T19:55:55.459Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_ff566e8b-48e5-4cc2-b7f3-7fb1ceff09a0/root/var/svc/log/oxide-crucible-pantry:default.log (782 bytes)
16932024-03-26T19:55:56.556Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_ff566e8b-48e5-4cc2-b7f3-7fb1ceff09a0/root/var/svc/log/oxide-crucible-pantry:default.log
16942024-03-26T19:55:56.556Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_ff566e8b-48e5-4cc2-b7f3-7fb1ceff09a0/root/var/svc/log/oxide-zone-network-setup:default.log (1745 bytes)
16952024-03-26T19:55:57.665Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_ff566e8b-48e5-4cc2-b7f3-7fb1ceff09a0/root/var/svc/log/oxide-zone-network-setup:default.log
16962024-03-26T19:55:57.665Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_47dd69da-c146-4f42-bbf7-29108a61e2c4/root/var/svc/log/oxide-external_dns:default.log (30649 bytes)
16972024-03-26T19:55:58.872Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_47dd69da-c146-4f42-bbf7-29108a61e2c4/root/var/svc/log/oxide-external_dns:default.log
16982024-03-26T19:55:58.872Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_47dd69da-c146-4f42-bbf7-29108a61e2c4/root/var/svc/log/oxide-opte-interface-setup:default.log (1187 bytes)
16992024-03-26T19:55:59.993Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_47dd69da-c146-4f42-bbf7-29108a61e2c4/root/var/svc/log/oxide-opte-interface-setup:default.log
17002024-03-26T19:55:59.994Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_47dd69da-c146-4f42-bbf7-29108a61e2c4/root/var/svc/log/oxide-zone-network-setup:default.log (1725 bytes)
17012024-03-26T19:56:01.124Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_47dd69da-c146-4f42-bbf7-29108a61e2c4/root/var/svc/log/oxide-zone-network-setup:default.log
17022024-03-26T19:56:01.124Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_9cc58510-30a1-40d4-8d78-e7ba6c2132c4/root/var/svc/log/oxide-internal_dns:default.log (111939 bytes)
17032024-03-26T19:56:02.516Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_9cc58510-30a1-40d4-8d78-e7ba6c2132c4/root/var/svc/log/oxide-internal_dns:default.log
17042024-03-26T19:56:02.516Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_9cc58510-30a1-40d4-8d78-e7ba6c2132c4/root/var/svc/log/oxide-zone-network-setup:default.log (1709 bytes)
17052024-03-26T19:56:04.042Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_9cc58510-30a1-40d4-8d78-e7ba6c2132c4/root/var/svc/log/oxide-zone-network-setup:default.log
17062024-03-26T19:56:04.043Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_7af4d714-fae5-48c3-b46a-5589fa68f9c9/root/var/svc/log/oxide-nexus:default.log (2951896 bytes)
17072024-03-26T19:56:06.987Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_7af4d714-fae5-48c3-b46a-5589fa68f9c9/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 2951896 -> 3041035
17082024-03-26T19:56:08.064Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_7af4d714-fae5-48c3-b46a-5589fa68f9c9/root/var/svc/log/oxide-nexus:default.log
17092024-03-26T19:56:08.064Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_7af4d714-fae5-48c3-b46a-5589fa68f9c9/root/var/svc/log/oxide-opte-interface-setup:default.log (1166 bytes)
17102024-03-26T19:56:09.611Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_7af4d714-fae5-48c3-b46a-5589fa68f9c9/root/var/svc/log/oxide-opte-interface-setup:default.log
17112024-03-26T19:56:09.611Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_7af4d714-fae5-48c3-b46a-5589fa68f9c9/root/var/svc/log/oxide-zone-network-setup:default.log (1684 bytes)
17122024-03-26T19:56:10.729Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_7af4d714-fae5-48c3-b46a-5589fa68f9c9/root/var/svc/log/oxide-zone-network-setup:default.log
17132024-03-26T19:56:10.730Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_8585c804-b81f-4145-bd5c-9d3c50973495/root/var/svc/log/oxide-cockroachdb:default.log (4407 bytes)
17142024-03-26T19:56:11.848Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_8585c804-b81f-4145-bd5c-9d3c50973495/root/var/svc/log/oxide-cockroachdb:default.log
17152024-03-26T19:56:11.848Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_8585c804-b81f-4145-bd5c-9d3c50973495/root/var/svc/log/oxide-zone-network-setup:default.log (1715 bytes)
17162024-03-26T19:56:12.956Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_8585c804-b81f-4145-bd5c-9d3c50973495/root/var/svc/log/oxide-zone-network-setup:default.log
17172024-03-26T19:56:12.956Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f59cec3a-78b0-4985-b562-ff3326daae44/root/var/svc/log/oxide-crucible-agent:default.log (12471 bytes)
17182024-03-26T19:56:14.123Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f59cec3a-78b0-4985-b562-ff3326daae44/root/var/svc/log/oxide-crucible-agent:default.log
17192024-03-26T19:56:14.123Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f59cec3a-78b0-4985-b562-ff3326daae44/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
17202024-03-26T19:56:15.248Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f59cec3a-78b0-4985-b562-ff3326daae44/root/var/svc/log/oxide-crucible-downstairs:default.log
17212024-03-26T19:56:15.249Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f59cec3a-78b0-4985-b562-ff3326daae44/root/var/svc/log/oxide-crucible-downstairs:downstairs-080af97e-499c-49d0-b6b1-4e52ca119ff2.log (3125 bytes)
17222024-03-26T19:56:16.808Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f59cec3a-78b0-4985-b562-ff3326daae44/root/var/svc/log/oxide-crucible-downstairs:downstairs-080af97e-499c-49d0-b6b1-4e52ca119ff2.log
17232024-03-26T19:56:16.809Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f59cec3a-78b0-4985-b562-ff3326daae44/root/var/svc/log/oxide-zone-network-setup:default.log (1703 bytes)
17242024-03-26T19:56:17.908Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f59cec3a-78b0-4985-b562-ff3326daae44/root/var/svc/log/oxide-zone-network-setup:default.log
17252024-03-26T19:56:17.908Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_4396f691-f346-451b-9d59-4a8874557f63/root/var/svc/log/oxide-external_dns:default.log (30643 bytes)
17262024-03-26T19:56:18.492Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_4396f691-f346-451b-9d59-4a8874557f63/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 30643 -> 34415
17272024-03-26T19:56:19.572Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_4396f691-f346-451b-9d59-4a8874557f63/root/var/svc/log/oxide-external_dns:default.log
17282024-03-26T19:56:19.572Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_4396f691-f346-451b-9d59-4a8874557f63/root/var/svc/log/oxide-opte-interface-setup:default.log (1188 bytes)
17292024-03-26T19:56:20.682Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_4396f691-f346-451b-9d59-4a8874557f63/root/var/svc/log/oxide-opte-interface-setup:default.log
17302024-03-26T19:56:20.682Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_4396f691-f346-451b-9d59-4a8874557f63/root/var/svc/log/oxide-zone-network-setup:default.log (1726 bytes)
17312024-03-26T19:56:22.243Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_4396f691-f346-451b-9d59-4a8874557f63/root/var/svc/log/oxide-zone-network-setup:default.log
17322024-03-26T19:56:22.243Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_c28a66dd-7207-4ab8-975b-d4e7904e4ebe/root/var/svc/log/oxide-internal_dns:default.log (111117 bytes)
17332024-03-26T19:56:22.473Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_c28a66dd-7207-4ab8-975b-d4e7904e4ebe/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 111117 -> 114873
17342024-03-26T19:56:27.828Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_c28a66dd-7207-4ab8-975b-d4e7904e4ebe/root/var/svc/log/oxide-internal_dns:default.log
17352024-03-26T19:56:27.828Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_c28a66dd-7207-4ab8-975b-d4e7904e4ebe/root/var/svc/log/oxide-zone-network-setup:default.log (1708 bytes)
17362024-03-26T19:56:29.734Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_c28a66dd-7207-4ab8-975b-d4e7904e4ebe/root/var/svc/log/oxide-zone-network-setup:default.log
17372024-03-26T19:56:29.734Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_88cd0976-db6c-4237-b0a8-a341936a58e2/root/var/svc/log/oxide-cockroachdb:default.log (4408 bytes)
17382024-03-26T19:56:31.412Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_88cd0976-db6c-4237-b0a8-a341936a58e2/root/var/svc/log/oxide-cockroachdb:default.log
17392024-03-26T19:56:31.412Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_88cd0976-db6c-4237-b0a8-a341936a58e2/root/var/svc/log/oxide-zone-network-setup:default.log (1710 bytes)
17402024-03-26T19:56:32.517Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_88cd0976-db6c-4237-b0a8-a341936a58e2/root/var/svc/log/oxide-zone-network-setup:default.log
17412024-03-26T19:56:32.517Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_354c619c-ce37-4dd7-902a-4858618387d6/root/var/svc/log/oxide-crucible-agent:default.log (23185 bytes)
17422024-03-26T19:56:33.682Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_354c619c-ce37-4dd7-902a-4858618387d6/root/var/svc/log/oxide-crucible-agent:default.log
17432024-03-26T19:56:33.711Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_354c619c-ce37-4dd7-902a-4858618387d6/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
17442024-03-26T19:56:34.799Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_354c619c-ce37-4dd7-902a-4858618387d6/root/var/svc/log/oxide-crucible-downstairs:default.log
17452024-03-26T19:56:34.799Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_354c619c-ce37-4dd7-902a-4858618387d6/root/var/svc/log/oxide-crucible-downstairs:downstairs-2f07026e-dc67-40a0-a57e-ae23b3553f40.log (3123 bytes)
17462024-03-26T19:56:35.911Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_354c619c-ce37-4dd7-902a-4858618387d6/root/var/svc/log/oxide-crucible-downstairs:downstairs-2f07026e-dc67-40a0-a57e-ae23b3553f40.log
17472024-03-26T19:56:35.911Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_354c619c-ce37-4dd7-902a-4858618387d6/root/var/svc/log/oxide-crucible-downstairs:downstairs-c4590200-fc6b-42c7-9ce6-e403edcc43c5.log (5614 bytes)
17482024-03-26T19:56:37.031Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_354c619c-ce37-4dd7-902a-4858618387d6/root/var/svc/log/oxide-crucible-downstairs:downstairs-c4590200-fc6b-42c7-9ce6-e403edcc43c5.log
17492024-03-26T19:56:37.031Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_354c619c-ce37-4dd7-902a-4858618387d6/root/var/svc/log/oxide-zone-network-setup:default.log (1704 bytes)
17502024-03-26T19:56:38.149Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_354c619c-ce37-4dd7-902a-4858618387d6/root/var/svc/log/oxide-zone-network-setup:default.log
17512024-03-26T19:56:38.149Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_620c922e-160b-4c61-ab27-e652f2ed073d/root/var/svc/log/oxide-internal_dns:default.log (111116 bytes)
17522024-03-26T19:56:38.545Zupload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_620c922e-160b-4c61-ab27-e652f2ed073d/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 111116 -> 114872
17532024-03-26T19:56:39.617Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_620c922e-160b-4c61-ab27-e652f2ed073d/root/var/svc/log/oxide-internal_dns:default.log
17542024-03-26T19:56:39.617Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_620c922e-160b-4c61-ab27-e652f2ed073d/root/var/svc/log/oxide-zone-network-setup:default.log (1709 bytes)
17552024-03-26T19:56:40.750Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_620c922e-160b-4c61-ab27-e652f2ed073d/root/var/svc/log/oxide-zone-network-setup:default.log
17562024-03-26T19:56:40.750Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_95cff384-74b1-4ecb-8cbf-cab77d39b166/root/var/svc/log/oxide-nexus:default.log (1790950 bytes)
17572024-03-26T19:56:44.648Zupload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_95cff384-74b1-4ecb-8cbf-cab77d39b166/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 1790950 -> 2063273
17582024-03-26T19:56:45.732Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_95cff384-74b1-4ecb-8cbf-cab77d39b166/root/var/svc/log/oxide-nexus:default.log
17592024-03-26T19:56:45.732Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_95cff384-74b1-4ecb-8cbf-cab77d39b166/root/var/svc/log/oxide-opte-interface-setup:default.log (1167 bytes)
17602024-03-26T19:56:46.840Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_95cff384-74b1-4ecb-8cbf-cab77d39b166/root/var/svc/log/oxide-opte-interface-setup:default.log
17612024-03-26T19:56:46.841Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_95cff384-74b1-4ecb-8cbf-cab77d39b166/root/var/svc/log/oxide-zone-network-setup:default.log (1684 bytes)
17622024-03-26T19:56:47.982Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_95cff384-74b1-4ecb-8cbf-cab77d39b166/root/var/svc/log/oxide-zone-network-setup:default.log
17632024-03-26T19:56:47.982Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_9a917780-f7a6-4e99-aa42-3f4eafaa01ed/root/var/svc/log/oxide-cockroachdb:default.log (4407 bytes)
17642024-03-26T19:56:49.101Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_9a917780-f7a6-4e99-aa42-3f4eafaa01ed/root/var/svc/log/oxide-cockroachdb:default.log
17652024-03-26T19:56:49.101Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_9a917780-f7a6-4e99-aa42-3f4eafaa01ed/root/var/svc/log/oxide-zone-network-setup:default.log (1716 bytes)
17662024-03-26T19:56:50.226Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_9a917780-f7a6-4e99-aa42-3f4eafaa01ed/root/var/svc/log/oxide-zone-network-setup:default.log
17672024-03-26T19:56:50.227Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_70b73a9e-271c-4f3e-971e-e939f4b92221/root/var/svc/log/oxide-crucible-agent:default.log (1327 bytes)
17682024-03-26T19:56:51.348Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_70b73a9e-271c-4f3e-971e-e939f4b92221/root/var/svc/log/oxide-crucible-agent:default.log
17692024-03-26T19:56:51.348Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_70b73a9e-271c-4f3e-971e-e939f4b92221/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
17702024-03-26T19:56:52.468Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_70b73a9e-271c-4f3e-971e-e939f4b92221/root/var/svc/log/oxide-crucible-downstairs:default.log
17712024-03-26T19:56:52.468Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_70b73a9e-271c-4f3e-971e-e939f4b92221/root/var/svc/log/oxide-zone-network-setup:default.log (1703 bytes)
17722024-03-26T19:56:53.992Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_70b73a9e-271c-4f3e-971e-e939f4b92221/root/var/svc/log/oxide-zone-network-setup:default.log
17732024-03-26T19:56:53.992Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_pantry_b3c4dfbc-9048-4010-bd53-9f26dc7478c4/root/var/svc/log/oxide-crucible-pantry:default.log (783 bytes)
17742024-03-26T19:56:55.088Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_pantry_b3c4dfbc-9048-4010-bd53-9f26dc7478c4/root/var/svc/log/oxide-crucible-pantry:default.log
17752024-03-26T19:56:55.088Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_pantry_b3c4dfbc-9048-4010-bd53-9f26dc7478c4/root/var/svc/log/oxide-zone-network-setup:default.log (1743 bytes)
17762024-03-26T19:56:56.223Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_pantry_b3c4dfbc-9048-4010-bd53-9f26dc7478c4/root/var/svc/log/oxide-zone-network-setup:default.log
17772024-03-26T19:56:56.223Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/svc/log/oxide-ntp:default.log (905 bytes)
17782024-03-26T19:56:57.309Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/svc/log/oxide-ntp:default.log
17792024-03-26T19:56:57.309Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/svc/log/oxide-opte-interface-setup:default.log (1158 bytes)
17802024-03-26T19:56:58.415Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/svc/log/oxide-opte-interface-setup:default.log
17812024-03-26T19:56:58.415Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/svc/log/oxide-zone-network-setup:default.log (1665 bytes)
17822024-03-26T19:56:59.522Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/svc/log/oxide-zone-network-setup:default.log
17832024-03-26T19:56:59.522Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_03d60141-91a3-464d-8e19-ffdc8db5422d/root/var/svc/log/oxide-cockroachdb:default.log (4381 bytes)
17842024-03-26T19:57:00.648Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_03d60141-91a3-464d-8e19-ffdc8db5422d/root/var/svc/log/oxide-cockroachdb:default.log
17852024-03-26T19:57:00.648Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_03d60141-91a3-464d-8e19-ffdc8db5422d/root/var/svc/log/oxide-zone-network-setup:default.log (1710 bytes)
17862024-03-26T19:57:01.755Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_03d60141-91a3-464d-8e19-ffdc8db5422d/root/var/svc/log/oxide-zone-network-setup:default.log
17872024-03-26T19:57:01.755Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_e4699323-7a5d-42c6-91f7-710c83f45cf3/root/var/svc/log/oxide-crucible-agent:default.log (1327 bytes)
17882024-03-26T19:57:03.268Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_e4699323-7a5d-42c6-91f7-710c83f45cf3/root/var/svc/log/oxide-crucible-agent:default.log
17892024-03-26T19:57:03.268Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_e4699323-7a5d-42c6-91f7-710c83f45cf3/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
17902024-03-26T19:57:04.383Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_e4699323-7a5d-42c6-91f7-710c83f45cf3/root/var/svc/log/oxide-crucible-downstairs:default.log
17912024-03-26T19:57:04.383Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_e4699323-7a5d-42c6-91f7-710c83f45cf3/root/var/svc/log/oxide-zone-network-setup:default.log (1704 bytes)
17922024-03-26T19:57:05.483Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_e4699323-7a5d-42c6-91f7-710c83f45cf3/root/var/svc/log/oxide-zone-network-setup:default.log
17932024-03-26T19:57:05.483Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_nexus_9548a077-d4f7-452a-ad05-2a2815dce2ee/root/var/svc/log/oxide-nexus:default.log (36383372 bytes)
17942024-03-26T19:57:22.376Zupload warning: file "/pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_nexus_9548a077-d4f7-452a-ad05-2a2815dce2ee/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 36383372 -> 37033255
17952024-03-26T19:57:23.472Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_nexus_9548a077-d4f7-452a-ad05-2a2815dce2ee/root/var/svc/log/oxide-nexus:default.log
17962024-03-26T19:57:23.472Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_nexus_9548a077-d4f7-452a-ad05-2a2815dce2ee/root/var/svc/log/oxide-opte-interface-setup:default.log (1166 bytes)
17972024-03-26T19:57:24.569Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_nexus_9548a077-d4f7-452a-ad05-2a2815dce2ee/root/var/svc/log/oxide-opte-interface-setup:default.log
17982024-03-26T19:57:24.569Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_nexus_9548a077-d4f7-452a-ad05-2a2815dce2ee/root/var/svc/log/oxide-zone-network-setup:default.log (1684 bytes)
17992024-03-26T19:57:25.676Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_nexus_9548a077-d4f7-452a-ad05-2a2815dce2ee/root/var/svc/log/oxide-zone-network-setup:default.log
18002024-03-26T19:57:25.676Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_db275601-0503-4d31-afe4-9fd31f11bac8/root/var/svc/log/oxide-crucible-agent:default.log (25690 bytes)
18012024-03-26T19:57:27.211Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_db275601-0503-4d31-afe4-9fd31f11bac8/root/var/svc/log/oxide-crucible-agent:default.log
18022024-03-26T19:57:27.211Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_db275601-0503-4d31-afe4-9fd31f11bac8/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
18032024-03-26T19:57:28.705Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_db275601-0503-4d31-afe4-9fd31f11bac8/root/var/svc/log/oxide-crucible-downstairs:default.log
18042024-03-26T19:57:28.705Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_db275601-0503-4d31-afe4-9fd31f11bac8/root/var/svc/log/oxide-crucible-downstairs:downstairs-643ab76f-b963-473a-8a83-4878b0c10079.log (5694 bytes)
18052024-03-26T19:57:29.827Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_db275601-0503-4d31-afe4-9fd31f11bac8/root/var/svc/log/oxide-crucible-downstairs:downstairs-643ab76f-b963-473a-8a83-4878b0c10079.log
18062024-03-26T19:57:29.827Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_db275601-0503-4d31-afe4-9fd31f11bac8/root/var/svc/log/oxide-crucible-downstairs:downstairs-b697dcd7-a359-4c8b-9ba8-fb637f30be39.log (3126 bytes)
18072024-03-26T19:57:31.372Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_db275601-0503-4d31-afe4-9fd31f11bac8/root/var/svc/log/oxide-crucible-downstairs:downstairs-b697dcd7-a359-4c8b-9ba8-fb637f30be39.log
18082024-03-26T19:57:31.372Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_db275601-0503-4d31-afe4-9fd31f11bac8/root/var/svc/log/oxide-crucible-downstairs:snapshot-643ab76f-b963-473a-8a83-4878b0c10079-b602aa42-71bd-47aa-96a7-d6be5868de11.log (5305 bytes)
18092024-03-26T19:57:32.536Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_db275601-0503-4d31-afe4-9fd31f11bac8/root/var/svc/log/oxide-crucible-downstairs:snapshot-643ab76f-b963-473a-8a83-4878b0c10079-b602aa42-71bd-47aa-96a7-d6be5868de11.log
18102024-03-26T19:57:32.536Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_db275601-0503-4d31-afe4-9fd31f11bac8/root/var/svc/log/oxide-zone-network-setup:default.log (1704 bytes)
18112024-03-26T19:57:33.650Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_db275601-0503-4d31-afe4-9fd31f11bac8/root/var/svc/log/oxide-zone-network-setup:default.log
18122024-03-26T19:57:33.650Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_3eacd805-52c4-427b-a906-dd1b74c60f25/root/var/svc/log/oxide-crucible-agent:default.log (1326 bytes)
18132024-03-26T19:57:42.647Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_3eacd805-52c4-427b-a906-dd1b74c60f25/root/var/svc/log/oxide-crucible-agent:default.log
18142024-03-26T19:57:42.647Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_3eacd805-52c4-427b-a906-dd1b74c60f25/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
18152024-03-26T19:57:43.099Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_3eacd805-52c4-427b-a906-dd1b74c60f25/root/var/svc/log/oxide-crucible-downstairs:default.log
18162024-03-26T19:57:43.099Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_3eacd805-52c4-427b-a906-dd1b74c60f25/root/var/svc/log/oxide-zone-network-setup:default.log (1704 bytes)
18172024-03-26T19:57:44.317Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_3eacd805-52c4-427b-a906-dd1b74c60f25/root/var/svc/log/oxide-zone-network-setup:default.log
18182024-03-26T19:57:44.317Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_2cac9179-a451-44f0-8e8a-fcbb6bc2fb93/root/var/svc/log/oxide-crucible-agent:default.log (26562 bytes)
18192024-03-26T19:57:45.625Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_2cac9179-a451-44f0-8e8a-fcbb6bc2fb93/root/var/svc/log/oxide-crucible-agent:default.log
18202024-03-26T19:57:45.625Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_2cac9179-a451-44f0-8e8a-fcbb6bc2fb93/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
18212024-03-26T19:57:46.741Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_2cac9179-a451-44f0-8e8a-fcbb6bc2fb93/root/var/svc/log/oxide-crucible-downstairs:default.log
18222024-03-26T19:57:46.741Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_2cac9179-a451-44f0-8e8a-fcbb6bc2fb93/root/var/svc/log/oxide-crucible-downstairs:downstairs-70630c85-1f34-41f1-9e6d-85a187376eec.log (5692 bytes)
18232024-03-26T19:57:48.744Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_2cac9179-a451-44f0-8e8a-fcbb6bc2fb93/root/var/svc/log/oxide-crucible-downstairs:downstairs-70630c85-1f34-41f1-9e6d-85a187376eec.log
18242024-03-26T19:57:48.744Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_2cac9179-a451-44f0-8e8a-fcbb6bc2fb93/root/var/svc/log/oxide-crucible-downstairs:downstairs-eee3be5c-efbd-4392-b273-66d6ae4fe3c8.log (3019 bytes)
18252024-03-26T19:57:49.445Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_2cac9179-a451-44f0-8e8a-fcbb6bc2fb93/root/var/svc/log/oxide-crucible-downstairs:downstairs-eee3be5c-efbd-4392-b273-66d6ae4fe3c8.log
18262024-03-26T19:57:49.445Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_2cac9179-a451-44f0-8e8a-fcbb6bc2fb93/root/var/svc/log/oxide-crucible-downstairs:snapshot-70630c85-1f34-41f1-9e6d-85a187376eec-b602aa42-71bd-47aa-96a7-d6be5868de11.log (5303 bytes)
18272024-03-26T19:57:50.562Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_2cac9179-a451-44f0-8e8a-fcbb6bc2fb93/root/var/svc/log/oxide-crucible-downstairs:snapshot-70630c85-1f34-41f1-9e6d-85a187376eec-b602aa42-71bd-47aa-96a7-d6be5868de11.log
18282024-03-26T19:57:50.562Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_2cac9179-a451-44f0-8e8a-fcbb6bc2fb93/root/var/svc/log/oxide-zone-network-setup:default.log (1703 bytes)
18292024-03-26T19:57:51.724Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_2cac9179-a451-44f0-8e8a-fcbb6bc2fb93/root/var/svc/log/oxide-zone-network-setup:default.log
18302024-03-26T19:57:51.724Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-crucible-agent:default.log (41597 bytes)
18312024-03-26T19:57:53.487Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-crucible-agent:default.log
18322024-03-26T19:57:53.489Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
18332024-03-26T19:57:54.689Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-crucible-downstairs:default.log
18342024-03-26T19:57:54.690Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-crucible-downstairs:downstairs-18d63532-1365-4420-a814-e188d0028204.log (3019 bytes)
18352024-03-26T19:57:55.851Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-crucible-downstairs:downstairs-18d63532-1365-4420-a814-e188d0028204.log
18362024-03-26T19:57:55.852Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-crucible-downstairs:downstairs-30a7be61-a003-45b5-b0f8-d02a0911e50d.log (5696 bytes)
18372024-03-26T19:57:56.984Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-crucible-downstairs:downstairs-30a7be61-a003-45b5-b0f8-d02a0911e50d.log
18382024-03-26T19:57:56.984Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-crucible-downstairs:downstairs-61dc6897-9a36-47fe-9a7d-cba7e2565801.log (5610 bytes)
18392024-03-26T19:57:58.125Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-crucible-downstairs:downstairs-61dc6897-9a36-47fe-9a7d-cba7e2565801.log
18402024-03-26T19:57:58.125Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-crucible-downstairs:snapshot-30a7be61-a003-45b5-b0f8-d02a0911e50d-b602aa42-71bd-47aa-96a7-d6be5868de11.log (5303 bytes)
18412024-03-26T19:57:59.232Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-crucible-downstairs:snapshot-30a7be61-a003-45b5-b0f8-d02a0911e50d-b602aa42-71bd-47aa-96a7-d6be5868de11.log
18422024-03-26T19:57:59.233Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-zone-network-setup:default.log (1702 bytes)
18432024-03-26T19:58:00.366Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_246336c3-791c-4c31-9083-b8a00e2ce292/root/var/svc/log/oxide-zone-network-setup:default.log
18442024-03-26T19:58:00.366Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_10275ccc-917a-4c8b-be46-7e77d0aa1b4d/root/var/svc/log/oxide-crucible-pantry:default.log (1222733 bytes)
18452024-03-26T19:58:03.134Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_10275ccc-917a-4c8b-be46-7e77d0aa1b4d/root/var/svc/log/oxide-crucible-pantry:default.log
18462024-03-26T19:58:03.134Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_10275ccc-917a-4c8b-be46-7e77d0aa1b4d/root/var/svc/log/oxide-zone-network-setup:default.log (1744 bytes)
18472024-03-26T19:58:04.496Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_10275ccc-917a-4c8b-be46-7e77d0aa1b4d/root/var/svc/log/oxide-zone-network-setup:default.log
18482024-03-26T19:58:04.496Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_dc320043-cdd8-4913-8282-580d4a933c11/root/var/svc/log/oxide-oximeter:default.log (720926 bytes)
18492024-03-26T19:58:06.141Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_dc320043-cdd8-4913-8282-580d4a933c11/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 720926 -> 1008950
18502024-03-26T19:58:07.231Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_dc320043-cdd8-4913-8282-580d4a933c11/root/var/svc/log/oxide-oximeter:default.log
18512024-03-26T19:58:07.231Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_dc320043-cdd8-4913-8282-580d4a933c11/root/var/svc/log/oxide-zone-network-setup:default.log (1701 bytes)
18522024-03-26T19:58:08.325Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_dc320043-cdd8-4913-8282-580d4a933c11/root/var/svc/log/oxide-zone-network-setup:default.log
18532024-03-26T19:58:08.325Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/log/chrony/measurements.log (15207 bytes)
18542024-03-26T19:58:08.416Zupload warning: file "/pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/log/chrony/measurements.log" changed size mid upload: 15207 -> 19865
18552024-03-26T19:58:09.497Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/log/chrony/measurements.log
18562024-03-26T19:58:09.497Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/log/chrony/statistics.log (10062 bytes)
18572024-03-26T19:58:09.573Zupload warning: file "/pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/log/chrony/statistics.log" changed size mid upload: 10062 -> 14157
18582024-03-26T19:58:10.698Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/log/chrony/statistics.log
18592024-03-26T19:58:10.698Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/log/chrony/tracking.log (1851 bytes)
18602024-03-26T19:58:10.775Zupload warning: file "/pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/log/chrony/tracking.log" changed size mid upload: 1851 -> 2247
18612024-03-26T19:58:11.851Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_ntp_9285e95c-9d34-4e60-a223-7336a874579b/root/var/log/chrony/tracking.log