01HNKKEMKXG49TFS1XYRHF1JME: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log*",
        "%/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.28",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01HNKKG78NVDX24E2T434BWWKD

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12024-02-02T02:23:08.525Zjob dependencies complete; ready to run (waiting for 1 h 54 m 1 s)
22024-02-02T03:01:23.348Zjob assigned to worker 01HNKVXTWH8PSNZZCTD19FDWBM (queued for 38 m 14 s)
32024-02-02T03:01:23.578Zhttp://172.20.0.5:9969/ipxe/buskin... ok
42024-02-02T03:01:23.630Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/kernel/amd64/unix... ok
52024-02-02T03:01:23.670Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive... ok
62024-02-02T03:01:23.709Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive.hash... ok
72024-02-02T03:01:23.748Zhttp://172.20.0.5:9969/postboot/buskin... ok
82024-02-02T03:01:23.788ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
92024-02-02T03:01:23.827Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d1
102024-02-02T03:01:23.868ZOxide Helios Version helios-2.0.22418 64-bit
112024-02-02T03:01:24.342ZConfiguring devices.
122024-02-02T03:01:24.392Z##################################################
132024-02-02T03:01:24.441Z#### ############# ##
142024-02-02T03:01:24.479Z### ### ############ ##
152024-02-02T03:01:24.518Z## ### # ## ### ## Oxide Computer Company ##
162024-02-02T03:01:24.561Z## ## ## ### # ### ##
172024-02-02T03:01:24.594Z## # ### #### #### This Station Under ##
182024-02-02T03:01:24.635Z### ### #### # ### Computer Control ##
192024-02-02T03:01:24.683Z#### #### ### ## ##
202024-02-02T03:01:24.720Z##################################################
212024-02-02T03:01:24.757ZHostname: buskin
222024-02-02T03:01:24.804ZLoading smf(7) service descriptions:
232024-02-02T03:01:25.231Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap-4fa0aa7f5e8d8c20.gz
242024-02-02T03:01:28.435Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap-4fa0aa7f5e8d8c20.gz
252024-02-02T03:01:28.435Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
262024-02-02T03:02:27.134Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
272024-02-02T03:02:27.135Zdownloading input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-378ef3b1593728ce.gz
282024-02-02T03:03:11.300Zdownloaded input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-378ef3b1593728ce.gz
292024-02-02T03:03:11.300Zdownloading input: /input/ci-tools/work/caboose-util.gz
302024-02-02T03:03:15.231Zdownloaded input: /input/ci-tools/work/caboose-util.gz
312024-02-02T03:03:15.269Zdownloading input: /input/ci-tools/work/tufaceous.gz
322024-02-02T03:03:20.857Zdownloaded input: /input/ci-tools/work/tufaceous.gz
332024-02-02T03:03:20.893Zdownloading input: /input/package/work/version.txt
342024-02-02T03:03:20.983Zdownloaded input: /input/package/work/version.txt
352024-02-02T03:03:20.983Zdownloading input: /input/package/work/package.tar.gz
362024-02-02T03:03:47.578Zdownloaded input: /input/package/work/package.tar.gz
372024-02-02T03:03:47.579Zdownloading input: /input/package/work/global-zone-packages.tar.gz
382024-02-02T03:04:36.434Zdownloaded input: /input/package/work/global-zone-packages.tar.gz
392024-02-02T03:04:36.434Zdownloading input: /input/package/work/trampoline-global-zone-packages.tar.gz
402024-02-02T03:05:07.627Zdownloaded input: /input/package/work/trampoline-global-zone-packages.tar.gz
412024-02-02T03:05:07.653Zdownloading input: /input/package/work/zones/clickhouse.tar.gz
422024-02-02T03:07:23.573Zdownloaded input: /input/package/work/zones/clickhouse.tar.gz
432024-02-02T03:07:23.573Zdownloading input: /input/package/work/zones/clickhouse_keeper.tar.gz
442024-02-02T03:08:03.328Zdownloaded input: /input/package/work/zones/clickhouse_keeper.tar.gz
452024-02-02T03:08:03.329Zdownloading input: /input/package/work/zones/cockroachdb.tar.gz
462024-02-02T03:08:26.701Zdownloaded input: /input/package/work/zones/cockroachdb.tar.gz
472024-02-02T03:08:26.701Zdownloading input: /input/package/work/zones/crucible-pantry.tar.gz
482024-02-02T03:08:29.918Zdownloaded input: /input/package/work/zones/crucible-pantry.tar.gz
492024-02-02T03:08:29.919Zdownloading input: /input/package/work/zones/crucible.tar.gz
502024-02-02T03:08:36.370Zdownloaded input: /input/package/work/zones/crucible.tar.gz
512024-02-02T03:08:36.371Zdownloading input: /input/package/work/zones/external-dns.tar.gz
522024-02-02T03:08:40.906Zdownloaded input: /input/package/work/zones/external-dns.tar.gz
532024-02-02T03:08:40.932Zdownloading input: /input/package/work/zones/internal-dns.tar.gz
542024-02-02T03:08:45.578Zdownloaded input: /input/package/work/zones/internal-dns.tar.gz
552024-02-02T03:08:45.578Zdownloading input: /input/package/work/zones/ntp.tar.gz
562024-02-02T03:08:45.615Zdownloaded input: /input/package/work/zones/ntp.tar.gz
572024-02-02T03:08:45.615Zdownloading input: /input/package/work/zones/omicron-gateway-asic.tar.gz
582024-02-02T03:08:50.216Zdownloaded input: /input/package/work/zones/omicron-gateway-asic.tar.gz
592024-02-02T03:08:50.216Zdownloading input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
602024-02-02T03:08:54.558Zdownloaded input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
612024-02-02T03:08:54.583Zdownloading input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
622024-02-02T03:09:09.950Zdownloaded input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
632024-02-02T03:09:09.950Zdownloading input: /input/package/work/zones/omicron-nexus.tar.gz
642024-02-02T03:09:37.689Zdownloaded input: /input/package/work/zones/omicron-nexus.tar.gz
652024-02-02T03:09:37.737Zdownloading input: /input/package/work/zones/overlay.tar.gz
662024-02-02T03:09:37.773Zdownloaded input: /input/package/work/zones/overlay.tar.gz
672024-02-02T03:09:37.773Zdownloading input: /input/package/work/zones/oximeter.tar.gz
682024-02-02T03:09:52.510Zdownloaded input: /input/package/work/zones/oximeter.tar.gz
692024-02-02T03:09:52.510Zdownloading input: /input/package/work/zones/propolis-server.tar.gz
702024-02-02T03:10:00.352Zbuskin console login: Feb 2 03:10:00 buskin cron[810]: Solaris_audit getaddrinfo(buskin) failed[node name or service name not known]: Error 0
712024-02-02T03:10:00.400ZFeb 2 03:10:00 buskin cron[810]: Solaris_audit mapping buskin to 10.151.1.100: Error 0
722024-02-02T03:10:07.245Zdownloaded input: /input/package/work/zones/propolis-server.tar.gz
732024-02-02T03:10:07.681Zdownloading input: /input/package/work/zones/switch-asic.tar.gz
742024-02-02T03:11:00.585Zdownloaded input: /input/package/work/zones/switch-asic.tar.gz
752024-02-02T03:11:00.585Zdownloading input: /input/package/work/zones/switch-softnpu.tar.gz
762024-02-02T03:11:41.402Zdownloaded input: /input/package/work/zones/switch-softnpu.tar.gz
 
772024-02-02T03:11:41.594Zstarting task 0: "setup"
782024-02-02T03:11:41.595Z++ uname -s
792024-02-02T03:11:41.595Z+ kern=SunOS
802024-02-02T03:11:41.595Z+ case "$kern" in
812024-02-02T03:11:41.595Z+ groupadd -g 12345 build
822024-02-02T03:11:41.595Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
832024-02-02T03:11:41.740Z+ zfs create -o mountpoint=/work rpool/work
842024-02-02T03:11:41.740Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
852024-02-02T03:11:41.740Z+ home_fs=zfs
862024-02-02T03:11:41.740Z+ [[ zfs == autofs ]]
872024-02-02T03:11:41.740Z+ mkdir -p /home/build
882024-02-02T03:11:41.740Z+ chown build:build /home/build /work
892024-02-02T03:11:43.069Z+ chmod 0700 /home/build /work
902024-02-02T03:11:43.097Zprocess exited: duration 4060 ms, exit code 0
 
912024-02-02T03:11:43.234Zstarting task 1: "authentication"
922024-02-02T03:11:43.291Zprocess exited: duration 57 ms, exit code 0
 
932024-02-02T03:11:43.368Zstarting task 2: "build"
942024-02-02T03:11:43.457Z+ trap _exit_trap EXIT
952024-02-02T03:11:43.457Z+ set +x
962024-02-02T03:12:26.027Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
972024-02-02T03:12:26.264Zsecpolicy_fs_linkdir: 0x55 = 0x55
982024-02-02T03:12:26.264Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
992024-02-02T03:12:26.264Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
1002024-02-02T03:12:26.264Zsecpolicy_fs_linkdir+3: 0 = 0xe5
1012024-02-02T03:12:26.291Zsecpolicy_fs_linkdir+4: 0 = 0xb8
1022024-02-02T03:12:26.291Zsecpolicy_fs_linkdir+5: 0 = 0x1
1032024-02-02T03:12:26.291Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
1042024-02-02T03:12:26.291Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
1052024-02-02T03:12:26.292Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
1062024-02-02T03:12:26.292Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
1072024-02-02T03:12:26.292Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
1082024-02-02T03:12:26.292Z+ [[ -d /opt/oxide ]]
1092024-02-02T03:12:26.292Z+ pfexec mv /opt/oxide /opt/oxide-underneath
1102024-02-02T03:12:26.292Z+ pfexec mkdir /opt/oxide
1112024-02-02T03:12:26.292Z+ pfexec mount -F tmpfs -O swap /opt/oxide
1122024-02-02T03:12:26.292Z+ [[ -d /opt/oxide-underneath ]]
1132024-02-02T03:12:26.292Z+ cd /opt/oxide-underneath
1142024-02-02T03:12:26.292Z+ pfexec tar ceEp@/f - .
1152024-02-02T03:12:26.292Z+ cd /opt/oxide
1162024-02-02T03:12:26.292Z+ pfexec tar xveEp@/f -
1172024-02-02T03:12:26.326Zx ., 0 bytes, 0 tape blocks
1182024-02-02T03:12:26.326Zx ./opte, 0 bytes, 0 tape blocks
1192024-02-02T03:12:26.326Zx ./opte/bin, 0 bytes, 0 tape blocks
1202024-02-02T03:12:26.326Zx ./opte/bin/opteadm, 51799224 bytes, 101171 tape blocks
1212024-02-02T03:12:26.359Z+ pfexec rm -rf /opt/oxide-underneath
1222024-02-02T03:12:26.359Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
1232024-02-02T03:12:26.385Z+ pfexec mkdir /opt/oxide/work
1242024-02-02T03:12:26.385Z+ pfexec chown build:build /opt/oxide/work
1252024-02-02T03:12:26.385Z+ cd /opt/oxide/work
1262024-02-02T03:12:26.385Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
1272024-02-02T03:12:28.041ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
1282024-02-02T03:12:28.041Zx out/mg-ddm-gz.tar, 63076864 bytes, 123197 tape blocks
1292024-02-02T03:12:28.041Zx out/omicron-sled-agent.tar, 122424320 bytes, 239110 tape blocks
1302024-02-02T03:12:28.041Zx out/oxlog.tar, 8382464 bytes, 16372 tape blocks
1312024-02-02T03:12:28.041Zx out/target/test, 75 bytes, 1 tape blocks
1322024-02-02T03:12:28.041Zx out/npuzone/npuzone, 37184104 bytes, 72626 tape blocks
1332024-02-02T03:12:28.041Zx package-manifest.toml, 23732 bytes, 47 tape blocks
1342024-02-02T03:12:28.041Zx smf/sled-agent/non-gimlet/config.toml, 3448 bytes, 7 tape blocks
1352024-02-02T03:12:28.041Zx target/release/omicron-package, 45618880 bytes, 89100 tape blocks
1362024-02-02T03:12:28.041Zx tools/create_virtual_hardware.sh, 3184 bytes, 7 tape blocks
1372024-02-02T03:12:28.041Zx tools/virtual_hardware.sh, 3416 bytes, 7 tape blocks
1382024-02-02T03:12:28.041Zx tools/scrimlet/create-softnpu-zone.sh, 627 bytes, 2 tape blocks
1392024-02-02T03:12:28.042Zx tools/scrimlet/destroy-softnpu-zone.sh, 203 bytes, 1 tape blocks
1402024-02-02T03:12:28.042Zx tools/scrimlet/softnpu-init.sh, 2157 bytes, 5 tape blocks
1412024-02-02T03:12:28.042Zx tools/scrimlet/softnpu-zone.txt, 330 bytes, 1 tape blocks
1422024-02-02T03:12:28.042Zx tools/scrimlet/softnpu.toml, 171 bytes, 1 tape blocks
1432024-02-02T03:12:28.068Z
1442024-02-02T03:12:28.068Zreal 1.674082626
1452024-02-02T03:12:28.068Zuser 1.504502846
1462024-02-02T03:12:28.068Zsys 0.301831195
1472024-02-02T03:12:28.068Ztrap 0.000071300
1482024-02-02T03:12:28.068Ztflt 0.000000000
1492024-02-02T03:12:28.068Zdflt 0.000000000
1502024-02-02T03:12:28.068Zkflt 0.000000000
1512024-02-02T03:12:28.068Zlock 0.000000000
1522024-02-02T03:12:28.068Zslp 1.501885474
1532024-02-02T03:12:28.068Zlat 0.037656227
1542024-02-02T03:12:28.068Zstop 0.000128961
1552024-02-02T03:12:28.068Z+ cp /input/package/work/zones/clickhouse_keeper.tar.gz /input/package/work/zones/clickhouse.tar.gz /input/package/work/zones/cockroachdb.tar.gz /input/package/work/zones/crucible-pantry.tar.gz /input/package/work/zones/crucible.tar.gz /input/package/work/zones/external-dns.tar.gz /input/package/work/zones/internal-dns.tar.gz /input/package/work/zones/ntp.tar.gz /input/package/work/zones/omicron-gateway-asic.tar.gz /input/package/work/zones/omicron-gateway-softnpu.tar.gz /input/package/work/zones/omicron-nexus-single-sled.tar.gz /input/package/work/zones/omicron-nexus.tar.gz /input/package/work/zones/overlay.tar.gz /input/package/work/zones/oximeter.tar.gz /input/package/work/zones/propolis-server.tar.gz /input/package/work/zones/switch-asic.tar.gz /input/package/work/zones/switch-softnpu.tar.gz out/
1562024-02-02T03:12:29.275Z+ mv out/omicron-nexus-single-sled.tar.gz out/omicron-nexus.tar.gz
1572024-02-02T03:12:29.731Z+ mkdir tests
1582024-02-02T03:12:29.731Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1592024-02-02T03:12:29.731Z+ ptime -m gunzip
1602024-02-02T03:12:29.731Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-4fa0aa7f5e8d8c20
1612024-02-02T03:12:29.772Z
1622024-02-02T03:12:29.772Zreal 0.236333740
1632024-02-02T03:12:29.772Zuser 0.217409093
1642024-02-02T03:12:29.772Zsys 0.018680455
1652024-02-02T03:12:29.772Ztrap 0.000018949
1662024-02-02T03:12:29.772Ztflt 0.000000000
1672024-02-02T03:12:29.772Zdflt 0.000000000
1682024-02-02T03:12:29.772Zkflt 0.000000000
1692024-02-02T03:12:29.772Zlock 0.000000000
1702024-02-02T03:12:29.773Zslp 0.000020040
1712024-02-02T03:12:29.773Zlat 0.000022840
1722024-02-02T03:12:29.773Zstop 0.000129400
1732024-02-02T03:12:29.773Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-4fa0aa7f5e8d8c20
1742024-02-02T03:12:29.773Z+ chmod a+x tests/bootstrap-4fa0aa7f5e8d8c20
1752024-02-02T03:12:29.773Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1762024-02-02T03:12:29.773Z+ ptime -m gunzip
1772024-02-02T03:12:29.773Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1782024-02-02T03:12:32.094Z
1792024-02-02T03:12:32.128Zreal 2.539276494
1802024-02-02T03:12:32.128Zuser 2.369716427
1812024-02-02T03:12:32.129Zsys 0.169110405
1822024-02-02T03:12:32.129Ztrap 0.000209461
1832024-02-02T03:12:32.129Ztflt 0.000000000
1842024-02-02T03:12:32.130Zdflt 0.000000000
1852024-02-02T03:12:32.130Zkflt 0.000000000
1862024-02-02T03:12:32.131Zlock 0.000000000
1872024-02-02T03:12:32.131Zslp 0.000014850
1882024-02-02T03:12:32.131Zlat 0.000024779
1892024-02-02T03:12:32.132Zstop 0.000131961
1902024-02-02T03:12:32.132Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1912024-02-02T03:12:32.132Z+ chmod a+x tests/bootstrap
1922024-02-02T03:12:32.132Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1932024-02-02T03:12:32.133Z+ ptime -m gunzip
1942024-02-02T03:12:32.133Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-378ef3b1593728ce
1952024-02-02T03:12:34.655Z
1962024-02-02T03:12:34.681Zreal 2.558150775
1972024-02-02T03:12:34.682Zuser 2.404184906
1982024-02-02T03:12:34.682Zsys 0.153493784
1992024-02-02T03:12:34.682Ztrap 0.000208393
2002024-02-02T03:12:34.682Ztflt 0.000000000
2012024-02-02T03:12:34.682Zdflt 0.000000000
2022024-02-02T03:12:34.682Zkflt 0.000000000
2032024-02-02T03:12:34.683Zlock 0.000000000
2042024-02-02T03:12:34.683Zslp 0.000044320
2052024-02-02T03:12:34.683Zlat 0.000025280
2062024-02-02T03:12:34.683Zstop 0.000131141
2072024-02-02T03:12:34.683Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-378ef3b1593728ce
2082024-02-02T03:12:34.683Z+ chmod a+x tests/end_to_end_tests-378ef3b1593728ce
2092024-02-02T03:12:34.683Z+ bmat address ls
2102024-02-02T03:12:34.683ZNAME CIDR FIRST LAST COUNT
2112024-02-02T03:12:34.683Zextra 10.151.1.0/24 10.151.1.164 10.151.1.227 64
2122024-02-02T03:12:34.684Z++ bmat address ls -f extra -Ho first,last,count
2132024-02-02T03:12:34.711Z+ set -- 10.151.1.164 10.151.1.227 64
2142024-02-02T03:12:34.711Z+ EXTRA_IP_START=10.151.1.164
2152024-02-02T03:12:34.712Z+ EXTRA_IP_END=10.151.1.227
2162024-02-02T03:12:34.712Z+ EXTRA_IP_COUNT=64
2172024-02-02T03:12:34.712Z+ (( EXTRA_IP_COUNT >= 32 ))
2182024-02-02T03:12:34.712Z+ EXTRA_IP_BASE=10.151.1
2192024-02-02T03:12:34.712Z+ EXTRA_IP_FOCTET=164
2202024-02-02T03:12:34.713Z+ EXTRA_IP_LOCTET=227
2212024-02-02T03:12:34.713Z+ SERVICE_IP_POOL_START=10.151.1.164
2222024-02-02T03:12:34.713Z+ SERVICE_IP_POOL_END=10.151.1.173
2232024-02-02T03:12:34.713Z+ DNS_IP1=10.151.1.164
2242024-02-02T03:12:34.714Z+ DNS_IP2=10.151.1.165
2252024-02-02T03:12:34.714Z+ UPLINK_IP=10.151.1.174
2262024-02-02T03:12:34.714Z+ IPPOOL_START=10.151.1.175
2272024-02-02T03:12:34.714Z+ IPPOOL_END=10.151.1.227
2282024-02-02T03:12:34.714Z++ ipadm show-addr -po type,addr
2292024-02-02T03:12:34.714Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
2302024-02-02T03:12:34.715Z+ GATEWAY_IP=10.151.1.100
2312024-02-02T03:12:34.715Z+ [[ -n 10.151.1.100 ]]
2322024-02-02T03:12:34.715Z+ ping -s 10.151.1.100 56 1
2332024-02-02T03:12:34.715ZPING 10.151.1.100: 56 data bytes
2342024-02-02T03:12:34.715Z64 bytes from 10.151.1.100: icmp_seq=0. time=0.012 ms
2352024-02-02T03:12:34.716Z
2362024-02-02T03:12:34.716Z----10.151.1.100 PING Statistics----
2372024-02-02T03:12:34.716Z1 packets transmitted, 1 packets received, 0% packet loss
2382024-02-02T03:12:34.716Zround-trip (ms) min/avg/max/stddev = 0.012/0.012/0.012/-nan
2392024-02-02T03:12:34.716Z++ arp -an
2402024-02-02T03:12:34.717Z++ awk -vgw=10.151.1.100 '$2 == gw {print $NF}'
2412024-02-02T03:12:34.717Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
2422024-02-02T03:12:34.717Z+ routeadm -e ipv4-forwarding -u
2432024-02-02T03:12:34.824Z+ PXA_START=10.151.1.164
2442024-02-02T03:12:34.825Z+ PXA_END=10.151.1.227
2452024-02-02T03:12:34.862Z+ export GATEWAY_IP GATEWAY_MAC PXA_START PXA_END
2462024-02-02T03:12:34.863Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
2472024-02-02T03:12:36.364Z+ ZPOOL_VDEV_DIR=/scratch
2482024-02-02T03:12:36.364Z+ ptime -m pfexec ./tools/create_virtual_hardware.sh
2492024-02-02T03:12:36.434Z+++ dirname ./tools/create_virtual_hardware.sh
2502024-02-02T03:12:36.435Z++ cd ./tools
2512024-02-02T03:12:36.435Z++ pwd
2522024-02-02T03:12:36.435Z+ SOURCE_DIR=/opt/oxide/work/tools
2532024-02-02T03:12:36.435Z+ OMICRON_TOP=/opt/oxide/work/tools/..
2542024-02-02T03:12:36.435Z+ SOFTNPU_MODE=zone
2552024-02-02T03:12:36.435Z+ . /opt/oxide/work/tools/virtual_hardware.sh
2562024-02-02T03:12:36.435Z++ MARKER=/etc/opt/oxide/NO_INSTALL
2572024-02-02T03:12:36.436Z++ [[ -f /etc/opt/oxide/NO_INSTALL ]]
2582024-02-02T03:12:36.436Z++ dladm show-phys -p -o LINK
2592024-02-02T03:12:36.436Z++ head -1
2602024-02-02T03:12:36.436Z+ PHYSICAL_LINK=igb0
2612024-02-02T03:12:36.436Z+ echo 'Using igb0 as physical link'
2622024-02-02T03:12:36.436ZUsing igb0 as physical link
2632024-02-02T03:12:36.437Z+ ensure_run_as_root
2642024-02-02T03:12:36.437Z++ id -u
2652024-02-02T03:12:36.437Z+ [[ 0 -ne 0 ]]
2662024-02-02T03:12:36.437Z+ ensure_zpools
2672024-02-02T03:12:36.437Z+ ZPOOL_TYPES=('oxp_' 'oxi_')
2682024-02-02T03:12:36.437Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
2692024-02-02T03:12:36.437Z+ readarray -t ZPOOLS
2702024-02-02T03:12:36.437Z++ grep '"oxp_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
2712024-02-02T03:12:36.438Z++ sed 's/[ ",]//g'
2722024-02-02T03:12:36.438Z+ for ZPOOL in "${ZPOOLS[@]}"
2732024-02-02T03:12:36.438Z+ echo 'Zpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]'
2742024-02-02T03:12:36.438ZZpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]
2752024-02-02T03:12:36.438Z+ VDEV_PATH=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2762024-02-02T03:12:36.438Z+ [[ -f /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
2772024-02-02T03:12:36.438Z+ dd if=/dev/zero of=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
2782024-02-02T03:12:36.438Z0+0 records in
2792024-02-02T03:12:36.438Z0+0 records out
2802024-02-02T03:12:36.438Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
2812024-02-02T03:12:36.438Z+ success 'ZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
2822024-02-02T03:12:36.439Z+ set +x
2832024-02-02T03:12:36.439ZZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
2842024-02-02T03:12:36.439Z++ zpool list -o name
2852024-02-02T03:12:36.439Z++ grep oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
2862024-02-02T03:12:36.439Z+ [[ -z '' ]]
2872024-02-02T03:12:36.439Z+ zpool create -f oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2882024-02-02T03:12:36.439Z+ success 'ZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
2892024-02-02T03:12:36.439Z+ set +x
2902024-02-02T03:12:36.439ZZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists
2912024-02-02T03:12:36.440Z+ for ZPOOL in "${ZPOOLS[@]}"
2922024-02-02T03:12:36.440Z+ echo 'Zpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
2932024-02-02T03:12:36.440Z+ VDEV_PATH=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
2942024-02-02T03:12:36.440ZZpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]
2952024-02-02T03:12:36.440Z+ [[ -f /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
2962024-02-02T03:12:36.440Z+ dd if=/dev/zero of=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
2972024-02-02T03:12:36.440Z0+0 records in
2982024-02-02T03:12:36.440Z0+0 records out
2992024-02-02T03:12:36.440Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3002024-02-02T03:12:36.440Z+ success 'ZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3012024-02-02T03:12:36.441Z+ set +x
3022024-02-02T03:12:36.441ZZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3032024-02-02T03:12:36.441Z++ zpool list -o name
3042024-02-02T03:12:36.441Z++ grep oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
3052024-02-02T03:12:36.441Z+ [[ -z '' ]]
3062024-02-02T03:12:36.441Z+ zpool create -f oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3072024-02-02T03:12:36.441Z+ success 'ZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3082024-02-02T03:12:36.441Z+ set +x
3092024-02-02T03:12:36.441ZZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3102024-02-02T03:12:36.441Z+ for ZPOOL in "${ZPOOLS[@]}"
3112024-02-02T03:12:36.441Z+ echo 'Zpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3122024-02-02T03:12:36.442ZZpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]
3132024-02-02T03:12:36.442Z+ VDEV_PATH=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3142024-02-02T03:12:36.442Z+ [[ -f /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3152024-02-02T03:12:36.442Z+ dd if=/dev/zero of=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3162024-02-02T03:12:36.442Z0+0 records in
3172024-02-02T03:12:36.442Z0+0 records out
3182024-02-02T03:12:36.442Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3192024-02-02T03:12:36.442Z+ success 'ZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3202024-02-02T03:12:36.442Z+ set +x
3212024-02-02T03:12:36.442ZZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3222024-02-02T03:12:36.442Z++ zpool list -o name
3232024-02-02T03:12:36.442Z++ grep oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
3242024-02-02T03:12:36.443Z+ [[ -z '' ]]
3252024-02-02T03:12:36.443Z+ zpool create -f oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3262024-02-02T03:12:36.443Z+ success 'ZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3272024-02-02T03:12:36.443Z+ set +x
3282024-02-02T03:12:36.443ZZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3292024-02-02T03:12:36.443Z+ for ZPOOL in "${ZPOOLS[@]}"
3302024-02-02T03:12:36.443Z+ echo 'Zpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3312024-02-02T03:12:36.443ZZpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]
3322024-02-02T03:12:36.443Z+ VDEV_PATH=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3332024-02-02T03:12:36.443Z+ [[ -f /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3342024-02-02T03:12:36.443Z+ dd if=/dev/zero of=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3352024-02-02T03:12:36.443Z0+0 records in
3362024-02-02T03:12:36.444Z0+0 records out
3372024-02-02T03:12:36.444Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3382024-02-02T03:12:36.444Z+ success 'ZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3392024-02-02T03:12:36.444Z+ set +x
3402024-02-02T03:12:36.444ZZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3412024-02-02T03:12:36.444Z++ zpool list -o name
3422024-02-02T03:12:36.444Z++ grep oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03
3432024-02-02T03:12:36.444Z+ [[ -z '' ]]
3442024-02-02T03:12:36.444Z+ zpool create -f oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3452024-02-02T03:12:36.444Z+ success 'ZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3462024-02-02T03:12:36.445Z+ set +x
3472024-02-02T03:12:36.445ZZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3482024-02-02T03:12:36.445Z+ for ZPOOL in "${ZPOOLS[@]}"
3492024-02-02T03:12:36.496Z+ echo 'Zpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3502024-02-02T03:12:36.496ZZpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]
3512024-02-02T03:12:36.496Z+ VDEV_PATH=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3522024-02-02T03:12:36.496Z+ [[ -f /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3532024-02-02T03:12:36.496Z+ dd if=/dev/zero of=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3542024-02-02T03:12:36.496Z0+0 records in
3552024-02-02T03:12:36.496Z0+0 records out
3562024-02-02T03:12:36.497Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3572024-02-02T03:12:36.497Z+ success 'ZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3582024-02-02T03:12:36.497Z+ set +x
3592024-02-02T03:12:36.497ZZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3602024-02-02T03:12:36.497Z++ zpool list -o name
3612024-02-02T03:12:36.497Z++ grep oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03
3622024-02-02T03:12:36.497Z+ [[ -z '' ]]
3632024-02-02T03:12:36.497Z+ zpool create -f oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3642024-02-02T03:12:36.498Z+ success 'ZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3652024-02-02T03:12:36.498Z+ set +x
3662024-02-02T03:12:36.498ZZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3672024-02-02T03:12:36.498Z+ for ZPOOL in "${ZPOOLS[@]}"
3682024-02-02T03:12:36.498Z+ echo 'Zpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]'
3692024-02-02T03:12:36.498ZZpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]
3702024-02-02T03:12:36.498Z+ VDEV_PATH=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3712024-02-02T03:12:36.499Z+ [[ -f /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev ]]
3722024-02-02T03:12:36.499Z+ dd if=/dev/zero of=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev bs=1 count=0 seek=20G
3732024-02-02T03:12:36.499Z0+0 records in
3742024-02-02T03:12:36.499Z0+0 records out
3752024-02-02T03:12:36.499Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3762024-02-02T03:12:36.499Z+ success 'ZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists'
3772024-02-02T03:12:36.499Z+ set +x
3782024-02-02T03:12:36.499ZZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists
3792024-02-02T03:12:36.499Z++ zpool list -o name
3802024-02-02T03:12:36.499Z++ grep oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930
3812024-02-02T03:12:36.499Z+ [[ -z '' ]]
3822024-02-02T03:12:36.500Z+ zpool create -f oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3832024-02-02T03:12:36.500Z+ success 'ZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists'
3842024-02-02T03:12:36.500Z+ set +x
3852024-02-02T03:12:36.500ZZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists
3862024-02-02T03:12:36.500Z+ for ZPOOL in "${ZPOOLS[@]}"
3872024-02-02T03:12:36.500Z+ echo 'Zpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]'
3882024-02-02T03:12:36.500ZZpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]
3892024-02-02T03:12:36.500Z+ VDEV_PATH=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
3902024-02-02T03:12:36.501Z+ [[ -f /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev ]]
3912024-02-02T03:12:36.501Z+ dd if=/dev/zero of=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev bs=1 count=0 seek=20G
3922024-02-02T03:12:36.501Z0+0 records in
3932024-02-02T03:12:36.501Z0+0 records out
3942024-02-02T03:12:36.501Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3952024-02-02T03:12:36.501Z+ success 'ZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists'
3962024-02-02T03:12:36.501Z+ set +x
3972024-02-02T03:12:36.501ZZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists
3982024-02-02T03:12:36.501Z++ zpool list -o name
3992024-02-02T03:12:36.501Z++ grep oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60
4002024-02-02T03:12:36.501Z+ [[ -z '' ]]
4012024-02-02T03:12:36.501Z+ zpool create -f oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
4022024-02-02T03:12:36.501Z+ success 'ZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists'
4032024-02-02T03:12:36.502Z+ set +x
4042024-02-02T03:12:36.502ZZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists
4052024-02-02T03:12:36.502Z+ for ZPOOL in "${ZPOOLS[@]}"
4062024-02-02T03:12:36.502Z+ echo 'Zpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]'
4072024-02-02T03:12:36.502ZZpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]
4082024-02-02T03:12:36.502Z+ VDEV_PATH=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4092024-02-02T03:12:36.502Z+ [[ -f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev ]]
4102024-02-02T03:12:36.502Z+ dd if=/dev/zero of=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev bs=1 count=0 seek=20G
4112024-02-02T03:12:36.502Z0+0 records in
4122024-02-02T03:12:36.502Z0+0 records out
4132024-02-02T03:12:36.502Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4142024-02-02T03:12:36.502Z+ success 'ZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists'
4152024-02-02T03:12:36.502Z+ set +x
4162024-02-02T03:12:36.503ZZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists
4172024-02-02T03:12:36.503Z++ zpool list -o name
4182024-02-02T03:12:36.503Z++ grep oxp_31bd71cd-4736-4a12-a387-9b74b050396f
4192024-02-02T03:12:36.503Z+ [[ -z '' ]]
4202024-02-02T03:12:36.503Z+ zpool create -f oxp_31bd71cd-4736-4a12-a387-9b74b050396f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4212024-02-02T03:12:36.503Z+ success 'ZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists'
4222024-02-02T03:12:36.503Z+ set +x
4232024-02-02T03:12:36.503ZZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists
4242024-02-02T03:12:36.503ZZpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]
4252024-02-02T03:12:36.503Z+ for ZPOOL in "${ZPOOLS[@]}"
4262024-02-02T03:12:36.503Z+ echo 'Zpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]'
4272024-02-02T03:12:36.503Z+ VDEV_PATH=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4282024-02-02T03:12:36.503Z+ [[ -f /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev ]]
4292024-02-02T03:12:36.504Z+ dd if=/dev/zero of=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev bs=1 count=0 seek=20G
4302024-02-02T03:12:36.504Z0+0 records in
4312024-02-02T03:12:36.504Z0+0 records out
4322024-02-02T03:12:36.504Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4332024-02-02T03:12:36.504Z+ success 'ZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists'
4342024-02-02T03:12:36.504Z+ set +x
4352024-02-02T03:12:36.504ZZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists
4362024-02-02T03:12:36.504Z++ zpool list -o name
4372024-02-02T03:12:36.504Z++ grep oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7
4382024-02-02T03:12:36.504Z+ [[ -z '' ]]
4392024-02-02T03:12:36.504Z+ zpool create -f oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4402024-02-02T03:12:36.504Z+ success 'ZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists'
4412024-02-02T03:12:36.505Z+ set +x
4422024-02-02T03:12:36.505ZZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists
4432024-02-02T03:12:36.505Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
4442024-02-02T03:12:36.505Z+ readarray -t ZPOOLS
4452024-02-02T03:12:36.505Z++ grep '"oxi_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
4462024-02-02T03:12:36.506Z++ sed 's/[ ",]//g'
4472024-02-02T03:12:36.506Z+ for ZPOOL in "${ZPOOLS[@]}"
4482024-02-02T03:12:36.506Z+ echo 'Zpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4492024-02-02T03:12:36.568ZZpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]
4502024-02-02T03:12:36.569Z+ VDEV_PATH=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4512024-02-02T03:12:36.569Z+ [[ -f /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4522024-02-02T03:12:36.569Z+ dd if=/dev/zero of=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4532024-02-02T03:12:36.569Z0+0 records in
4542024-02-02T03:12:36.569Z0+0 records out
4552024-02-02T03:12:36.569Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4562024-02-02T03:12:36.569Z+ success 'ZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4572024-02-02T03:12:36.570Z+ set +x
4582024-02-02T03:12:36.570ZZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4592024-02-02T03:12:36.570Z++ zpool list -o name
4602024-02-02T03:12:36.570Z++ grep oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b
4612024-02-02T03:12:36.570Z+ [[ -z '' ]]
4622024-02-02T03:12:36.570Z+ zpool create -f oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4632024-02-02T03:12:36.570Z+ success 'ZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4642024-02-02T03:12:36.570Z+ set +x
4652024-02-02T03:12:36.571ZZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4662024-02-02T03:12:36.571Z+ for ZPOOL in "${ZPOOLS[@]}"
4672024-02-02T03:12:36.571Z+ echo 'Zpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4682024-02-02T03:12:36.571ZZpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]
4692024-02-02T03:12:36.571Z+ VDEV_PATH=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4702024-02-02T03:12:36.571Z+ [[ -f /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4712024-02-02T03:12:36.571Z+ dd if=/dev/zero of=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4722024-02-02T03:12:36.571Z0+0 records in
4732024-02-02T03:12:36.571Z0+0 records out
4742024-02-02T03:12:36.571Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4752024-02-02T03:12:36.572Z+ success 'ZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4762024-02-02T03:12:36.572Z+ set +x
4772024-02-02T03:12:36.572ZZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4782024-02-02T03:12:36.572Z++ zpool list -o name
4792024-02-02T03:12:36.572Z++ grep oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b
4802024-02-02T03:12:36.572Z+ [[ -z '' ]]
4812024-02-02T03:12:36.572Z+ zpool create -f oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4822024-02-02T03:12:36.573Z+ success 'ZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4832024-02-02T03:12:36.573Z+ set +x
4842024-02-02T03:12:36.573ZZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4852024-02-02T03:12:36.573Z+ [[ zone == \z\o\n\e ]]
4862024-02-02T03:12:36.573Z+ ensure_simulated_links igb0
4872024-02-02T03:12:36.573Z+ local PHYSICAL_LINK=igb0
4882024-02-02T03:12:36.573Z+ INDICES=("0" "1")
4892024-02-02T03:12:36.573Z+ for I in "${INDICES[@]}"
4902024-02-02T03:12:36.574Z++ get_simnet_name_if_exists net0
4912024-02-02T03:12:36.574Z++ dladm show-simnet -p -o LINK net0
4922024-02-02T03:12:36.574Z++ echo ''
4932024-02-02T03:12:36.574Z+ [[ -z '' ]]
4942024-02-02T03:12:36.574Z+ dladm create-simnet -t net0
4952024-02-02T03:12:36.574Z+ dladm create-simnet -t sc0_0
4962024-02-02T03:12:36.574Z+ dladm modify-simnet -t -p net0 sc0_0
4972024-02-02T03:12:36.574Z+ dladm set-linkprop -p mtu=9000 sc0_0
4982024-02-02T03:12:36.574Z+ success 'Simnet net0/sc0_0 exists'
4992024-02-02T03:12:36.574Z+ set +x
5002024-02-02T03:12:36.575ZSimnet net0/sc0_0 exists
5012024-02-02T03:12:36.575Z+ for I in "${INDICES[@]}"
5022024-02-02T03:12:36.575Z++ get_simnet_name_if_exists net1
5032024-02-02T03:12:36.575Z++ dladm show-simnet -p -o LINK net1
5042024-02-02T03:12:36.575Z++ echo ''
5052024-02-02T03:12:36.575Z+ [[ -z '' ]]
5062024-02-02T03:12:36.576Z+ dladm create-simnet -t net1
5072024-02-02T03:12:36.576Z+ dladm create-simnet -t sc1_0
5082024-02-02T03:12:36.576Z+ dladm modify-simnet -t -p net1 sc1_0
5092024-02-02T03:12:36.576Z+ dladm set-linkprop -p mtu=9000 sc1_0
5102024-02-02T03:12:36.576Z+ success 'Simnet net1/sc1_0 exists'
5112024-02-02T03:12:36.577Z+ set +x
5122024-02-02T03:12:36.577ZSimnet net1/sc1_0 exists
5132024-02-02T03:12:36.577Z++ get_vnic_name_if_exists sc0_1
5142024-02-02T03:12:36.577Z++ dladm show-vnic -p -o LINK sc0_1
5152024-02-02T03:12:36.577Z++ echo ''
5162024-02-02T03:12:36.577Z+ [[ -z '' ]]
5172024-02-02T03:12:36.577Z+ dladm create-vnic -t sc0_1 -l igb0 -m a8:e1:de:01:70:1d
5182024-02-02T03:12:36.577Z+ [[ -v PROMISC_FILT_OFF ]]
5192024-02-02T03:12:36.577Z+ success 'Vnic sc0_1 exists'
5202024-02-02T03:12:36.577Z+ set +x
5212024-02-02T03:12:36.577ZVnic sc0_1 exists
5222024-02-02T03:12:36.577Z+ warn_if_no_proxy_arp
5232024-02-02T03:12:36.577Z+ [[ -v PXA_START ]]
5242024-02-02T03:12:36.578Z+ [[ -v PXA_END ]]
5252024-02-02T03:12:36.578Z+ ensure_softnpu_zone
5262024-02-02T03:12:36.578Z+ zoneadm list
5272024-02-02T03:12:36.578Z+ grep -q sidecar_softnpu
5282024-02-02T03:12:36.578Z+ [[ -f out/npuzone/npuzone ]]
5292024-02-02T03:12:36.578Z+ out/npuzone/npuzone create sidecar --omicron-zone --ports sc0_0,tfportrear0_0 --ports sc0_1,tfportqsfp0_0 --sidecar-lite-commit e3ea4b495ba0a71801ded0776ae4bbd31df57e26 --softnpu-commit dbab082dfa89da5db5ca2325c257089d2f130092
5302024-02-02T03:12:36.578Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/softnpu/image/dbab082dfa89da5db5ca2325c257089d2f130092/softnpu.sha256.txt
5312024-02-02T03:12:46.901Zdownloading softnpu ...done
5322024-02-02T03:12:46.927Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/libsidecar_lite.so.sha256.txt
5332024-02-02T03:12:54.015Zdownloading asic_program.so ...done
5342024-02-02T03:12:54.015Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/scadm.sha256.txt
5352024-02-02T03:13:14.549Zdownloading scadm ...done
5362024-02-02T03:13:14.578Zcreating port tfportrear0_0 for sc0_0
5372024-02-02T03:13:14.578Zcreating port tfportqsfp0_0 for sc0_1
5382024-02-02T03:13:14.578Zsoftnpu zone setup
5392024-02-02T03:13:14.578Zconfigure zone
5402024-02-02T03:13:14.604Zinstall zone
5412024-02-02T03:13:15.625Zboot zone
5422024-02-02T03:13:18.806Z[sidecar_softnpu] svcs milestone/network
5432024-02-02T03:13:18.836ZSTATE STIME FMRI
5442024-02-02T03:13:18.836Zonline 3:13:17 svc:/milestone/network:default
5452024-02-02T03:13:18.836Z[sidecar_softnpu] /softnpu/npu start
5462024-02-02T03:13:18.836Z
5472024-02-02T03:13:18.836Z+ /opt/oxide/work/tools/scrimlet/softnpu-init.sh
5482024-02-02T03:13:18.836Z+ GATEWAY_IP=10.151.1.100
5492024-02-02T03:13:18.836Z+ echo 'Using 10.151.1.100 as gateway ip'
5502024-02-02T03:13:18.837ZUsing 10.151.1.100 as gateway ip
5512024-02-02T03:13:18.837Z+ [[ ! -v GATEWAY_MAC ]]
5522024-02-02T03:13:18.837Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
5532024-02-02T03:13:18.837Z++ grep -c -E '^([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}$' /dev/fd/63
5542024-02-02T03:13:18.837Z+++ echo 18:c0:4d:0d:9f:b2
5552024-02-02T03:13:18.837Z+ COUNT=1
5562024-02-02T03:13:18.837Z+ [[ 1 -ne 1 ]]
5572024-02-02T03:13:18.837Z+ echo 'Using 18:c0:4d:0d:9f:b2 as gateway mac'
5582024-02-02T03:13:18.837ZUsing 18:c0:4d:0d:9f:b2 as gateway mac
5592024-02-02T03:13:18.837Z+ z_scadm add-arp-entry 10.151.1.100 18:c0:4d:0d:9f:b2
5602024-02-02T03:13:18.837Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-arp-entry 10.151.1.100 18:c0:4d:0d:9f:b2
5612024-02-02T03:13:18.837Z+ PXA_MAC=a8:e1:de:01:70:1d
5622024-02-02T03:13:18.837Z+ [[ -v PXA_START ]]
5632024-02-02T03:13:18.837Z+ z_scadm add-proxy-arp 10.151.1.164 10.151.1.227 a8:e1:de:01:70:1d
5642024-02-02T03:13:18.837Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-proxy-arp 10.151.1.164 10.151.1.227 a8:e1:de:01:70:1d
5652024-02-02T03:13:18.970Z+ z_scadm dump-state
5662024-02-02T03:13:18.970Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
5672024-02-02T03:13:19.021Zlocal v6:
5682024-02-02T03:13:19.021Zlocal v4:
5692024-02-02T03:13:19.021Zrouter v6:
5702024-02-02T03:13:19.021Zrouter v4_idx:
5712024-02-02T03:13:19.021Zrouter v4_routes:
5722024-02-02T03:13:19.022Zresolver v4:
5732024-02-02T03:13:19.022Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
5742024-02-02T03:13:19.022Zresolver v6:
5752024-02-02T03:13:19.022Znat v4:
5762024-02-02T03:13:19.022Znat_v6:
5772024-02-02T03:13:19.022Zport_mac:
5782024-02-02T03:13:19.022Zproxy arp:
5792024-02-02T03:13:19.022Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
5802024-02-02T03:13:19.022Z+ success 'softnpu zone exists'
5812024-02-02T03:13:19.022Z+ set +x
5822024-02-02T03:13:19.022Zsoftnpu zone exists
5832024-02-02T03:13:19.022Z
5842024-02-02T03:13:19.022Zreal 43.853576441
5852024-02-02T03:13:19.022Zuser 0.689447674
5862024-02-02T03:13:19.022Zsys 1.513095112
5872024-02-02T03:13:19.022Ztrap 0.000091271
5882024-02-02T03:13:19.022Ztflt 0.000186106
5892024-02-02T03:13:19.022Zdflt 0.001087738
5902024-02-02T03:13:19.022Zkflt 0.000000000
5912024-02-02T03:13:19.022Zlock 34:30.713061683
5922024-02-02T03:13:19.022Zslp 2:11.309538431
5932024-02-02T03:13:19.022Zlat 0.065675419
5942024-02-02T03:13:19.022Zstop 0.368018335
5952024-02-02T03:13:19.022Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml
5962024-02-02T03:13:19.023Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
5972024-02-02T03:13:19.023Z+ SILO_NAME=recovery
5982024-02-02T03:13:19.023Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
5992024-02-02T03:13:19.023Z+ EXTERNAL_DNS_DOMAIN=oxide.test
6002024-02-02T03:13:19.023Z+ sed -i~ '
6012024-02-02T03:13:19.023Z /^external_dns_ips/c\
6022024-02-02T03:13:19.023Zexternal_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
6032024-02-02T03:13:19.023Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
6042024-02-02T03:13:19.023Z /^first/c\
6052024-02-02T03:13:19.023Zfirst = "10.151.1.164"
6062024-02-02T03:13:19.023Z /^last/c\
6072024-02-02T03:13:19.023Zlast = "10.151.1.173"
6082024-02-02T03:13:19.023Z }
6092024-02-02T03:13:19.023Z /^\[rack_network_config/,/^$/ {
6102024-02-02T03:13:19.023Z /^infra_ip_first/c\
6112024-02-02T03:13:19.023Zinfra_ip_first = "10.151.1.174"
6122024-02-02T03:13:19.023Z /^infra_ip_last/c\
6132024-02-02T03:13:19.023Zinfra_ip_last = "10.151.1.174"
6142024-02-02T03:13:19.023Z }
6152024-02-02T03:13:19.023Z /^\[\[rack_network_config.ports/,/^$/ {
6162024-02-02T03:13:19.023Z /^routes/c\
6172024-02-02T03:13:19.023Zroutes = \[{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}\]
6182024-02-02T03:13:19.023Z /^addresses/c\
6192024-02-02T03:13:19.023Zaddresses = \["10.151.1.174/24"\]
6202024-02-02T03:13:19.023Z }
6212024-02-02T03:13:19.024Z' pkg/config-rss.toml
6222024-02-02T03:13:19.024Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
6232024-02-02T03:13:19.024Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
6242024-02-02T03:13:19.024Z+++ pkg/config-rss.toml Fri Feb 2 03:13:19 2024
6252024-02-02T03:13:19.024Z@@ -41,7 +41,7 @@
6262024-02-02T03:13:19.024Z # the DNS domain delegated to the rack by the customer. Each of these addresses
6272024-02-02T03:13:19.024Z # must be contained in one of the "internal services" IP Pool ranges listed
6282024-02-02T03:13:19.024Z # below.
6292024-02-02T03:13:19.024Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
6302024-02-02T03:13:19.024Z+external_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
6312024-02-02T03:13:19.024Z
6322024-02-02T03:13:19.024Z # Initial TLS certificates for the external API
6332024-02-02T03:13:19.024Z #
6342024-02-02T03:13:19.024Z@@ -77,8 +77,8 @@
6352024-02-02T03:13:19.024Z #
6362024-02-02T03:13:19.024Z # For more on this and what to put here, see docs/how-to-run.adoc.
6372024-02-02T03:13:19.024Z [[internal_services_ip_pool_ranges]]
6382024-02-02T03:13:19.024Z-first = "192.168.1.20"
6392024-02-02T03:13:19.024Z-last = "192.168.1.29"
6402024-02-02T03:13:19.024Z+first = "10.151.1.164"
6412024-02-02T03:13:19.024Z+last = "10.151.1.173"
6422024-02-02T03:13:19.024Z
6432024-02-02T03:13:19.024Z # TODO - this configuration is subject to change going forward. Ultimately these
6442024-02-02T03:13:19.024Z # parameters should be provided to the control plane via wicket, but we need to
6452024-02-02T03:13:19.025Z@@ -92,8 +92,8 @@
6462024-02-02T03:13:19.025Z # A range of IP addresses used by Boundary Services on the external network. In
6472024-02-02T03:13:19.025Z # a real system, these would be addresses of the uplink ports on the Sidecar.
6482024-02-02T03:13:19.025Z # With softnpu, only one address is used.
6492024-02-02T03:13:19.025Z-infra_ip_first = "192.168.1.30"
6502024-02-02T03:13:19.025Z-infra_ip_last = "192.168.1.30"
6512024-02-02T03:13:19.025Z+infra_ip_first = "10.151.1.174"
6522024-02-02T03:13:19.025Z+infra_ip_last = "10.151.1.174"
6532024-02-02T03:13:19.025Z
6542024-02-02T03:13:19.025Z # Configurations for BGP routers to run on the scrimlets.
6552024-02-02T03:13:19.025Z bgp = []
6562024-02-02T03:13:19.025Z@@ -101,9 +101,9 @@
6572024-02-02T03:13:19.025Z # You can configure multiple uplinks by repeating the following stanza
6582024-02-02T03:13:19.025Z [[rack_network_config.ports]]
6592024-02-02T03:13:19.025Z # Routes associated with this port.
6602024-02-02T03:13:19.025Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
6612024-02-02T03:13:19.025Z+routes = [{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}]
6622024-02-02T03:13:19.025Z # Addresses associated with this port.
6632024-02-02T03:13:19.025Z-addresses = ["192.168.1.30/24"]
6642024-02-02T03:13:19.025Z+addresses = ["10.151.1.174/24"]
6652024-02-02T03:13:19.025Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
6662024-02-02T03:13:19.025Z port = "qsfp0"
6672024-02-02T03:13:19.059Z # The speed of this port.
6682024-02-02T03:13:19.059Z+ true
6692024-02-02T03:13:19.059Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml
6702024-02-02T03:13:19.059Za pkg/config-rss.toml 7K
6712024-02-02T03:13:19.059Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~
6722024-02-02T03:13:19.059Z+ TLS_NAME=recovery.sys.oxide.test
6732024-02-02T03:13:19.059Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
6742024-02-02T03:13:19.245Z.+....+..+..........+..+...+....+........................+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+.+..+......+.......+.....+...+......+...+..........+...+...+.....+......+.+.....+....+.....+...+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+..+......+.......+...........+.........+.+......+...............+...+...+..+............+...+.+.....+.+.....+......+...+.+.....+......+.......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6752024-02-02T03:13:19.946Z.........+.........+......+..........+.........+...+.....+.........+....+..+...+......+.+...+......+...+............+......+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+.........+....+..+..........+.........+...+..+...+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.........+...+..........+......+...+..+.........+.+......+..+...+....+.....+.............+........+.........+...............+......+...+.......+...............+..+.+......+.....+.........+.+...+..+......+.......+......+...........+.....................+.......+.....+...+..................+.......+...+.....+............+.......+...+.....+...+......+...+...+.........+.........+.+.........+...........................+...+......+...........+.......................................+.........+.+........+..........+...........+......+...+.......+......+........+....+..+...............+.+..+.......+...+.....+.......+......+...............+........+.+........+..................+.........+...................+...............+......+...+...+...+..+...+.+.....................+........+......+......+..........+...+.........+......+...........+...+......+.......+...+......+........+.......+......+.....+.......+..............................+......+.............................+......+.........+.+............+...+........................+......+.........+........+...............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6762024-02-02T03:13:19.971Z-----
6772024-02-02T03:13:19.971Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6782024-02-02T03:13:19.971Za pkg/initial-tls-cert.pem 2K
6792024-02-02T03:13:19.971Za pkg/initial-tls-key.pem 4K
6802024-02-02T03:13:19.971Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6812024-02-02T03:13:19.971Z+ rmdir pkg
6822024-02-02T03:13:19.971Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
6832024-02-02T03:13:19.971Z+ pfexec mkdir -p /usr/oxide
6842024-02-02T03:13:19.971Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0
6852024-02-02T03:13:20.700Z+ pfexec chmod +x /usr/oxide/oxide
6862024-02-02T03:13:20.740Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
6872024-02-02T03:13:38.939Z+ OMICRON_NO_UNINSTALL=1
6882024-02-02T03:13:38.939Z+ ptime -m pfexec ./target/release/omicron-package -t test install
6892024-02-02T03:13:38.966ZLogging to: /opt/oxide/work/out/LOG
6902024-02-02T03:13:41.168Z
6912024-02-02T03:13:41.210Zreal 0.512291606
6922024-02-02T03:13:41.211Zuser 0.108329222
6932024-02-02T03:13:41.211Zsys 2.263608523
6942024-02-02T03:13:41.211Ztrap 0.000005009
6952024-02-02T03:13:41.211Ztflt 0.000021439
6962024-02-02T03:13:41.211Zdflt 0.001609311
6972024-02-02T03:13:41.211Zkflt 0.000008020
6982024-02-02T03:13:41.211Zlock 46.270212713
6992024-02-02T03:13:41.212Zslp 0.733630221
7002024-02-02T03:13:41.212Zlat 0.003140247
7012024-02-02T03:13:41.212Zstop 0.215691085
7022024-02-02T03:13:41.212Z+ retry=0
7032024-02-02T03:13:41.212Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7042024-02-02T03:13:41.212Z+ [[ 0 -gt 30 ]]
7052024-02-02T03:13:41.212Z+ sleep 1
7062024-02-02T03:13:41.293Z+ retry=1
7072024-02-02T03:13:41.293Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7082024-02-02T03:13:41.293Z+ [[ 1 -gt 30 ]]
7092024-02-02T03:13:41.293Z+ sleep 1
7102024-02-02T03:13:41.484Z+ retry=2
7112024-02-02T03:13:41.484Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7122024-02-02T03:13:41.528Z+ [[ 2 -gt 30 ]]
7132024-02-02T03:13:41.528Z+ sleep 1
7142024-02-02T03:13:42.497Z+ retry=3
7152024-02-02T03:13:42.497Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7162024-02-02T03:13:42.522Z+ [[ 3 -gt 30 ]]
7172024-02-02T03:13:42.522Z+ sleep 1
7182024-02-02T03:13:43.512Z+ retry=4
7192024-02-02T03:13:43.512Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7202024-02-02T03:14:08.699Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
7212024-02-02T03:14:08.726Zlocal v6:
7222024-02-02T03:14:08.726Zfd37:3afd:1e1:586e::1
7232024-02-02T03:14:08.726Zfe80::4f3:81ff:fee7:77b7
7242024-02-02T03:14:08.726Zlocal v4:
7252024-02-02T03:14:08.726Z10.151.1.174
7262024-02-02T03:14:08.726Zrouter v6:
7272024-02-02T03:14:08.726Zfd00:1122:3344:3::/64 -> fe80::d4d7:7aff:fe66:24c1 (1)
7282024-02-02T03:14:08.726Zfd00:1122:3344:101::/64 -> fe80::d4d7:7aff:fe66:24c1 (1)
7292024-02-02T03:14:08.727Zfd00:1122:3344:2::/64 -> fe80::d4d7:7aff:fe66:24c1 (1)
7302024-02-02T03:14:08.727Zfdb0:18c0:4d0d:9fb2::/64 -> fe80::d4d7:7aff:fe66:24c1 (1)
7312024-02-02T03:14:08.727Zfd00:1122:3344:1::/64 -> fe80::d4d7:7aff:fe66:24c1 (1)
7322024-02-02T03:14:08.727Zrouter v4_idx:
7332024-02-02T03:14:08.727Z0.0.0.0/0 -> 0
7342024-02-02T03:14:08.727Zrouter v4_routes:
7352024-02-02T03:14:08.727Z0 -> 10.151.1.100 (2)
7362024-02-02T03:14:08.727Zresolver v4:
7372024-02-02T03:14:08.727Z10.151.1.174 -> 32:c7:e1:d3:2f:f8
7382024-02-02T03:14:08.727Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
7392024-02-02T03:14:08.727Zresolver v6:
7402024-02-02T03:14:08.727Zfe80::4f3:81ff:fee7:77b7 -> 06:f3:81:e7:77:b7
7412024-02-02T03:14:08.727Zfe80::d4d7:7aff:fe66:24c1 -> d6:d7:7a:66:24:c1
7422024-02-02T03:14:08.727Znat v4:
7432024-02-02T03:14:08.727Z10.151.1.169 0/16383 -> fd00:1122:3344:101::1 100/a8:40:25:ff:9e:21
7442024-02-02T03:14:08.727Znat_v6:
7452024-02-02T03:14:08.727Zport_mac:
7462024-02-02T03:14:08.727Z1000: a8:40:25:26:bc:8c
7472024-02-02T03:14:08.727Z1: a8:40:25:26:bc:8d
7482024-02-02T03:14:08.727Zproxy arp:
7492024-02-02T03:14:08.727Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
7502024-02-02T03:14:08.727Z+ retry=0
7512024-02-02T03:14:08.752Z+++ zoneadm list -n
7522024-02-02T03:14:08.753Z+++ grep oxz_ntp
7532024-02-02T03:14:08.753Z++ pfexec svcs -z oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4 -Hostate oxide/ntp
7542024-02-02T03:14:08.753Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7552024-02-02T03:14:08.753Z++ true
7562024-02-02T03:14:08.753Z+ [[ '' != online ]]
7572024-02-02T03:14:08.753Z+ [[ 0 -gt 60 ]]
7582024-02-02T03:14:08.753Z+ sleep 1
7592024-02-02T03:14:09.753Z+ retry=1
7602024-02-02T03:14:09.780Z+++ zoneadm list -n
7612024-02-02T03:14:09.780Z+++ grep oxz_ntp
7622024-02-02T03:14:09.780Z++ pfexec svcs -z oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4 -Hostate oxide/ntp
7632024-02-02T03:14:09.780Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7642024-02-02T03:14:09.780Z++ true
7652024-02-02T03:14:09.805Z+ [[ '' != online ]]
7662024-02-02T03:14:09.805Z+ [[ 1 -gt 60 ]]
7672024-02-02T03:14:09.805Z+ sleep 1
7682024-02-02T03:14:10.785Z+ retry=2
7692024-02-02T03:14:10.814Z+++ zoneadm list -n
7702024-02-02T03:14:10.814Z+++ grep oxz_ntp
7712024-02-02T03:14:10.814Z++ pfexec svcs -z oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4 -Hostate oxide/ntp
7722024-02-02T03:14:10.814Z+ [[ disabled != online ]]
7732024-02-02T03:14:10.814Z+ [[ 2 -gt 60 ]]
7742024-02-02T03:14:10.814Z+ sleep 1
7752024-02-02T03:14:11.807Z+ retry=3
7762024-02-02T03:14:11.835Z+++ zoneadm list -n
7772024-02-02T03:14:11.835Z+++ grep oxz_ntp
7782024-02-02T03:14:11.836Z++ pfexec svcs -z oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4 -Hostate oxide/ntp
7792024-02-02T03:14:11.836Z+ [[ disabled != online ]]
7802024-02-02T03:14:11.836Z+ [[ 3 -gt 60 ]]
7812024-02-02T03:14:11.836Z+ sleep 1
7822024-02-02T03:14:12.828Z+ retry=4
7832024-02-02T03:14:12.855Z+++ zoneadm list -n
7842024-02-02T03:14:12.856Z+++ grep oxz_ntp
7852024-02-02T03:14:12.856Z++ pfexec svcs -z oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4 -Hostate oxide/ntp
7862024-02-02T03:14:12.856Z+ [[ disabled != online ]]
7872024-02-02T03:14:12.857Z+ [[ 4 -gt 60 ]]
7882024-02-02T03:14:12.857Z+ sleep 1
7892024-02-02T03:14:13.853Z+ retry=5
7902024-02-02T03:14:13.890Z+++ zoneadm list -n
7912024-02-02T03:14:13.890Z+++ grep oxz_ntp
7922024-02-02T03:14:13.890Z++ pfexec svcs -z oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4 -Hostate oxide/ntp
7932024-02-02T03:14:13.890Z+ [[ disabled != online ]]
7942024-02-02T03:14:13.890Z+ [[ 5 -gt 60 ]]
7952024-02-02T03:14:13.890Z+ sleep 1
7962024-02-02T03:14:14.878Z+ retry=6
7972024-02-02T03:14:14.906Z+++ zoneadm list -n
7982024-02-02T03:14:14.906Z+++ grep oxz_ntp
7992024-02-02T03:14:14.906Z++ pfexec svcs -z oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4 -Hostate oxide/ntp
8002024-02-02T03:14:14.906Z+ [[ disabled != online ]]
8012024-02-02T03:14:14.906Z+ [[ 6 -gt 60 ]]
8022024-02-02T03:14:14.906Z+ sleep 1
8032024-02-02T03:14:15.897Z+ retry=7
8042024-02-02T03:14:15.927Z+++ zoneadm list -n
8052024-02-02T03:14:15.927Z+++ grep oxz_ntp
8062024-02-02T03:14:15.927Z++ pfexec svcs -z oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4 -Hostate oxide/ntp
8072024-02-02T03:14:15.927Z+ [[ disabled != online ]]
8082024-02-02T03:14:15.927Z+ [[ 7 -gt 60 ]]
8092024-02-02T03:14:15.927Z+ sleep 1
 
8102024-02-02T03:14:16.813ZFeb 2 03:14:17 buskin ip: ip: joining multicasts failed (4) on vopte0 - will use link layer broadcasts for multicast
 
8112024-02-02T03:14:16.916Z+ retry=8
8122024-02-02T03:14:16.941Z+++ zoneadm list -n
8132024-02-02T03:14:16.941Z+++ grep oxz_ntp
8142024-02-02T03:14:16.941Z++ pfexec svcs -z oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4 -Hostate oxide/ntp
8152024-02-02T03:14:16.941Z+ [[ disabled != online ]]
8162024-02-02T03:14:16.941Z+ [[ 8 -gt 60 ]]
8172024-02-02T03:14:16.941Z+ sleep 1
8182024-02-02T03:14:17.944Z+ retry=9
8192024-02-02T03:14:17.969Z+++ zoneadm list -n
8202024-02-02T03:14:17.969Z+++ grep oxz_ntp
8212024-02-02T03:14:17.969Z++ pfexec svcs -z oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4 -Hostate oxide/ntp
8222024-02-02T03:14:17.969Z+ [[ online != online ]]
8232024-02-02T03:14:17.969Z+ echo 'Waited for chrony: 9s'
8242024-02-02T03:14:17.969ZWaited for chrony: 9s
8252024-02-02T03:14:17.969Z+ export RUST_BACKTRACE=1
8262024-02-02T03:14:17.969Z+ RUST_BACKTRACE=1
8272024-02-02T03:14:17.969Z+ export E2E_TLS_CERT IPPOOL_START IPPOOL_END
8282024-02-02T03:14:17.969Z++ ./tests/bootstrap
8292024-02-02T03:14:17.993ZUsing external DNS server at 10.151.1.164:53
8302024-02-02T03:14:17.994Z2024-02-02 03:14:18.563687909 UTC: attempting to log into API
8312024-02-02T03:14:33.029Z2024-02-02 03:14:33.368064143 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8322024-02-02T03:14:34.034Z2024-02-02 03:14:34.369876992 UTC: attempting to log into API
8332024-02-02T03:14:49.066Z2024-02-02 03:14:49.403219275 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8342024-02-02T03:14:50.069Z2024-02-02 03:14:50.404996625 UTC: attempting to log into API
8352024-02-02T03:15:05.101Z2024-02-02 03:15:05.437350630 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8362024-02-02T03:15:06.100Z2024-02-02 03:15:06.438182821 UTC: attempting to log into API
8372024-02-02T03:15:21.133Z2024-02-02 03:15:21.470903499 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8382024-02-02T03:15:22.539Z2024-02-02 03:15:22.472629748 UTC: attempting to log into API
8392024-02-02T03:15:37.169Z2024-02-02 03:15:36.918254141 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8402024-02-02T03:15:38.170Z2024-02-02 03:15:37.919020981 UTC: attempting to log into API
8412024-02-02T03:15:53.619Z2024-02-02 03:15:52.951761440 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8422024-02-02T03:15:54.204Z2024-02-02 03:15:53.952571361 UTC: attempting to log into API
8432024-02-02T03:16:09.237Z2024-02-02 03:16:08.985872484 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8442024-02-02T03:16:10.244Z2024-02-02 03:16:09.986596894 UTC: attempting to log into API
8452024-02-02T03:16:25.938Z2024-02-02 03:16:25.019331352 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8462024-02-02T03:16:26.274Z2024-02-02 03:16:26.020161814 UTC: attempting to log into API
8472024-02-02T03:16:41.306Z2024-02-02 03:16:41.085740581 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8482024-02-02T03:16:42.306Z2024-02-02 03:16:42.090084883 UTC: attempting to log into API
8492024-02-02T03:16:57.753Z2024-02-02 03:16:57.163429571 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8502024-02-02T03:16:58.341Z2024-02-02 03:16:58.167322414 UTC: attempting to log into API
8512024-02-02T03:17:13.376Z2024-02-02 03:17:13.241224131 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8522024-02-02T03:17:14.381Z2024-02-02 03:17:14.244722399 UTC: attempting to log into API
8532024-02-02T03:17:29.411Z2024-02-02 03:17:29.318317088 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8542024-02-02T03:17:30.418Z2024-02-02 03:17:30.321888997 UTC: attempting to log into API
8552024-02-02T03:17:45.446Z2024-02-02 03:17:45.396754450 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8562024-02-02T03:17:46.444Z2024-02-02 03:17:46.401012444 UTC: attempting to log into API
8572024-02-02T03:18:01.493Z2024-02-02 03:18:01.467018144 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8582024-02-02T03:18:02.481Z2024-02-02 03:18:02.468942544 UTC: attempting to log into API
8592024-02-02T03:18:18.444Z2024-02-02 03:18:17.503179163 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8602024-02-02T03:18:18.606Z2024-02-02 03:18:18.506322453 UTC: attempting to log into API
8612024-02-02T03:18:33.595Z2024-02-02 03:18:33.558041287 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8622024-02-02T03:18:34.574Z2024-02-02 03:18:34.559071390 UTC: attempting to log into API
8632024-02-02T03:18:50.078Z2024-02-02 03:18:49.592566072 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8642024-02-02T03:18:51.245Z2024-02-02 03:18:50.593228602 UTC: attempting to log into API
8652024-02-02T03:19:18.810Z2024-02-02 03:19:05.627200148 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8662024-02-02T03:19:18.810Z2024-02-02 03:19:06.629234299 UTC: attempting to log into API
8672024-02-02T03:19:21.680Z2024-02-02 03:19:21.664560897 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8682024-02-02T03:19:22.689Z2024-02-02 03:19:22.665959002 UTC: attempting to log into API
8692024-02-02T03:19:37.731Z2024-02-02 03:19:37.716361151 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8702024-02-02T03:19:38.731Z2024-02-02 03:19:38.716750339 UTC: attempting to log into API
 
8712024-02-02T03:19:40.181ZFeb 2 03:19:39 buskin ip: ip: joining multicasts failed (4) on vopte3 - will use link layer broadcasts for multicast
8722024-02-02T03:19:46.318ZFeb 2 03:19:46 buskin ip: ip: joining multicasts failed (4) on vopte1 - will use link layer broadcasts for multicast
8732024-02-02T03:19:52.447ZFeb 2 03:19:51 buskin ip: ip: joining multicasts failed (4) on vopte4 - will use link layer broadcasts for multicast
 
8742024-02-02T03:19:53.769Z2024-02-02 03:19:53.752450043 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8752024-02-02T03:19:54.768Z2024-02-02 03:19:54.753134632 UTC: attempting to log into API
8762024-02-02T03:20:09.807Z2024-02-02 03:20:09.786164881 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8772024-02-02T03:20:10.806Z2024-02-02 03:20:10.787202944 UTC: attempting to log into API
 
8782024-02-02T03:20:14.779ZFeb 2 03:19:57 buskin ip: ip: joining multicasts failed (4) on vopte2 - will use link layer broadcasts for multicast
8792024-02-02T03:20:14.820ZFeb 2 03:20:04 buskin ip: ip: joining multicasts failed (4) on vopte5 - will use link layer broadcasts for multicast
 
8802024-02-02T03:20:21.776Z2024-02-02 03:20:21.598000747 UTC: login succeeded
8812024-02-02T03:20:21.919Zwaiting for nexus to come up...
8822024-02-02T03:20:21.919Zcreating IP pool... 10.151.1.175 - 10.151.1.227
8832024-02-02T03:20:21.947Zsetting up quotas...
8842024-02-02T03:20:21.972Zensuring datasets are ready...
8852024-02-02T03:20:29.013Zlet's roll.
8862024-02-02T03:20:29.416Z+ eval 'OXIDE_HOST=https://recovery.sys.oxide.test
8872024-02-02T03:20:29.417ZOXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
8882024-02-02T03:20:29.417ZOXIDE_TOKEN=oxide-token-741421d2af144595b3e463d6312383d8afd30f6e'
8892024-02-02T03:20:29.417Z++ OXIDE_HOST=https://recovery.sys.oxide.test
8902024-02-02T03:20:29.417Z++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
8912024-02-02T03:20:29.417Z++ OXIDE_TOKEN=oxide-token-741421d2af144595b3e463d6312383d8afd30f6e
8922024-02-02T03:20:29.417Z+ export OXIDE_HOST OXIDE_TOKEN
8932024-02-02T03:20:29.417Z+ retry=0
8942024-02-02T03:20:29.417Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8952024-02-02T03:20:29.417Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images'
8962024-02-02T03:20:29.417Z{"status":"ok"}success
8972024-02-02T03:20:29.417ZProject {
8982024-02-02T03:20:29.417Z description: "some images",
8992024-02-02T03:20:29.417Z id: e50403dc-d229-465e-a9bd-c8e2315a3a99,
9002024-02-02T03:20:29.417Z name: Name(
9012024-02-02T03:20:29.417Z "images",
9022024-02-02T03:20:29.417Z ),
9032024-02-02T03:20:29.417Z time_created: 2024-02-02T03:20:24.807380Z,
9042024-02-02T03:20:29.417Z time_modified: 2024-02-02T03:20:24.807380Z,
9052024-02-02T03:20:29.417Z}
9062024-02-02T03:20:29.417Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11
9072024-02-02T03:26:21.907Zdone!
9082024-02-02T03:26:22.005Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11
9092024-02-02T03:26:22.104Zsuccess
9102024-02-02T03:26:22.104ZImage {
9112024-02-02T03:26:22.104Z block_size: ByteCount(
9122024-02-02T03:26:22.104Z 512,
9132024-02-02T03:26:22.104Z ),
9142024-02-02T03:26:22.138Z description: "debian 11 original base image",
9152024-02-02T03:26:22.139Z digest: None,
9162024-02-02T03:26:22.139Z id: 2cbeae47-63cb-456a-b759-0b5555b1ae94,
9172024-02-02T03:26:22.139Z name: Name(
9182024-02-02T03:26:22.139Z "debian11",
9192024-02-02T03:26:22.139Z ),
9202024-02-02T03:26:22.139Z os: "debian",
9212024-02-02T03:26:22.139Z project_id: None,
9222024-02-02T03:26:22.139Z size: ByteCount(
9232024-02-02T03:26:22.139Z 2147483648,
9242024-02-02T03:26:22.139Z ),
9252024-02-02T03:26:22.139Z time_created: 2024-02-02T03:26:21.871292Z,
9262024-02-02T03:26:22.139Z time_modified: 2024-02-02T03:26:22.079154Z,
9272024-02-02T03:26:22.139Z url: None,
9282024-02-02T03:26:22.139Z version: "11",
9292024-02-02T03:26:22.139Z}
9302024-02-02T03:26:22.139Z+ rm ./tests/bootstrap
9312024-02-02T03:26:22.176Z+ for test_bin in tests/*
9322024-02-02T03:26:22.176Z+ ./tests/bootstrap-4fa0aa7f5e8d8c20
9332024-02-02T03:26:22.216Z
9342024-02-02T03:26:22.216Zrunning 0 tests
9352024-02-02T03:26:22.245Z
9362024-02-02T03:26:22.245Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
9372024-02-02T03:26:22.245Z
9382024-02-02T03:26:22.245Z+ for test_bin in tests/*
9392024-02-02T03:26:22.245Z+ ./tests/end_to_end_tests-378ef3b1593728ce
9402024-02-02T03:26:22.320Z
9412024-02-02T03:26:22.320Zrunning 2 tests
9422024-02-02T03:26:22.556Ztest no_spoof::no_spoof_here ... ok
9432024-02-02T03:26:57.116Ztest instance_launch::instance_launch ... ok
9442024-02-02T03:26:57.142Z
9452024-02-02T03:26:57.142Ztest result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 34.80s
9462024-02-02T03:26:57.143Z
9472024-02-02T03:26:57.143Z+ _exit_trap
9482024-02-02T03:26:57.143Z+ local status=0
9492024-02-02T03:26:57.143Z+ [[ 0 -eq 0 ]]
9502024-02-02T03:26:57.144Z+ exit 0
9512024-02-02T03:26:57.144Zprocess exited: duration 913767 ms, exit code 0
 
9522024-02-02T03:26:57.281Zfound 85 output files
9532024-02-02T03:26:57.281Zuploading: /var/svc/log/oxide-sled-agent:default.log (751265 bytes)
9542024-02-02T03:26:59.765Zuploaded: /var/svc/log/oxide-sled-agent:default.log
9552024-02-02T03:26:59.794Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log (2994010 bytes)
9562024-02-02T03:27:03.473Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log" changed size mid upload: 2994010 -> 3020381
9572024-02-02T03:27:04.547Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log
9582024-02-02T03:27:04.547Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log.0 (315900 bytes)
9592024-02-02T03:27:05.965Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log.0
9602024-02-02T03:27:05.965Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log (102153 bytes)
9612024-02-02T03:27:06.163Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log" changed size mid upload: 102153 -> 103404
9622024-02-02T03:27:07.241Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log
9632024-02-02T03:27:07.242Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log.0 (12732 bytes)
9642024-02-02T03:27:08.370Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log.0
9652024-02-02T03:27:08.370Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log (122210 bytes)
9662024-02-02T03:27:08.556Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log" changed size mid upload: 122210 -> 125041
9672024-02-02T03:27:09.642Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log
9682024-02-02T03:27:09.643Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log.0 (16565 bytes)
9692024-02-02T03:27:10.784Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log.0
9702024-02-02T03:27:10.784Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log (1497235 bytes)
9712024-02-02T03:27:11.912Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log" changed size mid upload: 1497235 -> 1528392
9722024-02-02T03:27:12.989Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log
9732024-02-02T03:27:12.989Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log.0 (149506 bytes)
9742024-02-02T03:27:14.384Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log.0
9752024-02-02T03:27:14.385Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log (935222 bytes)
9762024-02-02T03:27:15.117Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log" changed size mid upload: 935222 -> 954612
9772024-02-02T03:27:16.233Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log
9782024-02-02T03:27:16.233Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log.0 (80527 bytes)
9792024-02-02T03:27:17.461Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log.0
9802024-02-02T03:27:17.461Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log (0 bytes)
9812024-02-02T03:27:18.599Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log
9822024-02-02T03:27:18.599Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log.0 (1388 bytes)
9832024-02-02T03:27:19.702Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log.0
9842024-02-02T03:27:19.702Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log (346524 bytes)
9852024-02-02T03:27:20.521Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log" changed size mid upload: 346524 -> 357106
9862024-02-02T03:27:21.605Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log
9872024-02-02T03:27:21.605Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log.0 (30101 bytes)
9882024-02-02T03:27:23.274Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log.0
9892024-02-02T03:27:23.274Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log (0 bytes)
9902024-02-02T03:27:24.349Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log
9912024-02-02T03:27:24.349Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log.0 (617 bytes)
9922024-02-02T03:27:25.454Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log.0
9932024-02-02T03:27:25.455Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log (19125 bytes)
9942024-02-02T03:27:25.978Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log" changed size mid upload: 19125 -> 19922
9952024-02-02T03:27:27.054Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log
9962024-02-02T03:27:27.054Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log.0 (6987 bytes)
9972024-02-02T03:27:28.176Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log.0
9982024-02-02T03:27:28.176Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_348ff6f1-1bf1-4c0e-ba53-2fa860e9379b/root/var/svc/log/oxide-clickhouse:default.log (788220 bytes)
9992024-02-02T03:27:29.329Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_348ff6f1-1bf1-4c0e-ba53-2fa860e9379b/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 788220 -> 852402
10002024-02-02T03:27:30.401Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_348ff6f1-1bf1-4c0e-ba53-2fa860e9379b/root/var/svc/log/oxide-clickhouse:default.log
10012024-02-02T03:27:30.402Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_348ff6f1-1bf1-4c0e-ba53-2fa860e9379b/root/var/svc/log/oxide-zone-network-setup:default.log (1708 bytes)
10022024-02-02T03:27:31.499Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_348ff6f1-1bf1-4c0e-ba53-2fa860e9379b/root/var/svc/log/oxide-zone-network-setup:default.log
10032024-02-02T03:27:31.499Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_55bf3ccb-3ee9-4a5c-b9c8-ff6e61b6cae6/root/var/svc/log/oxide-cockroachdb:default.log (4408 bytes)
10042024-02-02T03:27:32.615Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_55bf3ccb-3ee9-4a5c-b9c8-ff6e61b6cae6/root/var/svc/log/oxide-cockroachdb:default.log
10052024-02-02T03:27:32.615Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_55bf3ccb-3ee9-4a5c-b9c8-ff6e61b6cae6/root/var/svc/log/oxide-zone-network-setup:default.log (1710 bytes)
10062024-02-02T03:27:33.724Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_55bf3ccb-3ee9-4a5c-b9c8-ff6e61b6cae6/root/var/svc/log/oxide-zone-network-setup:default.log
10072024-02-02T03:27:33.724Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_14a41c99-5d3b-43d0-bc7f-44355e6a8df4/root/var/svc/log/oxide-crucible-agent:default.log (3435 bytes)
10082024-02-02T03:27:34.841Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_14a41c99-5d3b-43d0-bc7f-44355e6a8df4/root/var/svc/log/oxide-crucible-agent:default.log
10092024-02-02T03:27:34.841Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_14a41c99-5d3b-43d0-bc7f-44355e6a8df4/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10102024-02-02T03:27:35.944Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_14a41c99-5d3b-43d0-bc7f-44355e6a8df4/root/var/svc/log/oxide-crucible-downstairs:default.log
10112024-02-02T03:27:35.944Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_30ac3b78-37a6-47ce-91a9-2963356cf58a/root/var/svc/log/oxide-external_dns:default.log (26945 bytes)
10122024-02-02T03:27:36.063Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_30ac3b78-37a6-47ce-91a9-2963356cf58a/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 26945 -> 30722
10132024-02-02T03:27:37.147Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_30ac3b78-37a6-47ce-91a9-2963356cf58a/root/var/svc/log/oxide-external_dns:default.log
10142024-02-02T03:27:37.147Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_4f5fe18a-4d8d-4ac4-8886-31f22a291127/root/var/svc/log/oxide-internal_dns:default.log (103162 bytes)
10152024-02-02T03:27:37.386Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_4f5fe18a-4d8d-4ac4-8886-31f22a291127/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 103162 -> 106918
10162024-02-02T03:27:38.477Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_4f5fe18a-4d8d-4ac4-8886-31f22a291127/root/var/svc/log/oxide-internal_dns:default.log
10172024-02-02T03:27:38.477Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_4f5fe18a-4d8d-4ac4-8886-31f22a291127/root/var/svc/log/oxide-internal_dns:default.log.0 (7536 bytes)
10182024-02-02T03:27:39.604Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_4f5fe18a-4d8d-4ac4-8886-31f22a291127/root/var/svc/log/oxide-internal_dns:default.log.0
10192024-02-02T03:27:39.604Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_53fc7ec7-6b35-4853-ad4a-3310d65bc67f/root/var/svc/log/oxide-oximeter:default.log (247510 bytes)
10202024-02-02T03:27:40.040Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_53fc7ec7-6b35-4853-ad4a-3310d65bc67f/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 247510 -> 275387
10212024-02-02T03:27:41.123Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_53fc7ec7-6b35-4853-ad4a-3310d65bc67f/root/var/svc/log/oxide-oximeter:default.log
10222024-02-02T03:27:41.123Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_53fc7ec7-6b35-4853-ad4a-3310d65bc67f/root/var/svc/log/oxide-zone-network-setup:default.log (1697 bytes)
10232024-02-02T03:27:42.635Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_53fc7ec7-6b35-4853-ad4a-3310d65bc67f/root/var/svc/log/oxide-zone-network-setup:default.log
10242024-02-02T03:27:42.635Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_5fa5b8e5-75bb-44b3-bdce-b6977a5c7e58/root/var/svc/log/oxide-cockroachdb:default.log (4408 bytes)
10252024-02-02T03:27:43.741Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_5fa5b8e5-75bb-44b3-bdce-b6977a5c7e58/root/var/svc/log/oxide-cockroachdb:default.log
10262024-02-02T03:27:43.741Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_5fa5b8e5-75bb-44b3-bdce-b6977a5c7e58/root/var/svc/log/oxide-zone-network-setup:default.log (1711 bytes)
10272024-02-02T03:27:44.859Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_5fa5b8e5-75bb-44b3-bdce-b6977a5c7e58/root/var/svc/log/oxide-zone-network-setup:default.log
10282024-02-02T03:27:44.859Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_5c1ddb1b-ccc1-48a1-90b0-5d6c264bfa2a/root/var/svc/log/oxide-crucible-agent:default.log (28676 bytes)
10292024-02-02T03:27:46.023Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_5c1ddb1b-ccc1-48a1-90b0-5d6c264bfa2a/root/var/svc/log/oxide-crucible-agent:default.log
10302024-02-02T03:27:46.023Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_5c1ddb1b-ccc1-48a1-90b0-5d6c264bfa2a/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10312024-02-02T03:27:47.138Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_5c1ddb1b-ccc1-48a1-90b0-5d6c264bfa2a/root/var/svc/log/oxide-crucible-downstairs:default.log
10322024-02-02T03:27:47.138Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_5c1ddb1b-ccc1-48a1-90b0-5d6c264bfa2a/root/var/svc/log/oxide-crucible-downstairs:downstairs-21e4aec7-7ba3-41ec-9549-6b6ca2f0b3dd.log (2943 bytes)
10332024-02-02T03:27:48.234Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_5c1ddb1b-ccc1-48a1-90b0-5d6c264bfa2a/root/var/svc/log/oxide-crucible-downstairs:downstairs-21e4aec7-7ba3-41ec-9549-6b6ca2f0b3dd.log
10342024-02-02T03:27:48.234Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_5c1ddb1b-ccc1-48a1-90b0-5d6c264bfa2a/root/var/svc/log/oxide-crucible-downstairs:downstairs-616c8f13-4b1a-4fce-932f-30f898d60b1a.log (5623 bytes)
10352024-02-02T03:27:49.337Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_5c1ddb1b-ccc1-48a1-90b0-5d6c264bfa2a/root/var/svc/log/oxide-crucible-downstairs:downstairs-616c8f13-4b1a-4fce-932f-30f898d60b1a.log
10362024-02-02T03:27:49.337Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_5c1ddb1b-ccc1-48a1-90b0-5d6c264bfa2a/root/var/svc/log/oxide-crucible-downstairs:snapshot-616c8f13-4b1a-4fce-932f-30f898d60b1a-67e3f87d-f7ac-4c8e-a7f9-d0b6f02f8cba.log (5180 bytes)
10372024-02-02T03:27:50.447Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_5c1ddb1b-ccc1-48a1-90b0-5d6c264bfa2a/root/var/svc/log/oxide-crucible-downstairs:snapshot-616c8f13-4b1a-4fce-932f-30f898d60b1a-67e3f87d-f7ac-4c8e-a7f9-d0b6f02f8cba.log
10382024-02-02T03:27:50.448Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_502c734e-4a17-49a0-a04b-e3804beb4df6/root/var/svc/log/oxide-external_dns:default.log (26946 bytes)
10392024-02-02T03:27:50.544Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_502c734e-4a17-49a0-a04b-e3804beb4df6/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 26946 -> 30721
10402024-02-02T03:27:51.619Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_502c734e-4a17-49a0-a04b-e3804beb4df6/root/var/svc/log/oxide-external_dns:default.log
10412024-02-02T03:27:51.619Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_0298f24c-9439-4ebe-8e21-97865df71c75/root/var/svc/log/oxide-internal_dns:default.log (103168 bytes)
10422024-02-02T03:27:51.832Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_0298f24c-9439-4ebe-8e21-97865df71c75/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 103168 -> 106925
10432024-02-02T03:27:52.930Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_0298f24c-9439-4ebe-8e21-97865df71c75/root/var/svc/log/oxide-internal_dns:default.log
10442024-02-02T03:27:52.930Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_0298f24c-9439-4ebe-8e21-97865df71c75/root/var/svc/log/oxide-internal_dns:default.log.0 (9161 bytes)
10452024-02-02T03:27:54.101Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_0298f24c-9439-4ebe-8e21-97865df71c75/root/var/svc/log/oxide-internal_dns:default.log.0
10462024-02-02T03:27:54.101Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_78fbf059-aec9-4f39-b88f-72adf16cf12b/root/var/svc/log/oxide-cockroachdb:default.log (4408 bytes)
10472024-02-02T03:27:55.216Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_78fbf059-aec9-4f39-b88f-72adf16cf12b/root/var/svc/log/oxide-cockroachdb:default.log
10482024-02-02T03:27:55.217Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_78fbf059-aec9-4f39-b88f-72adf16cf12b/root/var/svc/log/oxide-zone-network-setup:default.log (1711 bytes)
10492024-02-02T03:27:56.322Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_78fbf059-aec9-4f39-b88f-72adf16cf12b/root/var/svc/log/oxide-zone-network-setup:default.log
10502024-02-02T03:27:56.351Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_5f05ad59-a7f0-4780-a071-ed2775cdf0d9/root/var/svc/log/oxide-crucible-agent:default.log (24361 bytes)
10512024-02-02T03:27:57.475Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_5f05ad59-a7f0-4780-a071-ed2775cdf0d9/root/var/svc/log/oxide-crucible-agent:default.log
10522024-02-02T03:27:57.475Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_5f05ad59-a7f0-4780-a071-ed2775cdf0d9/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10532024-02-02T03:27:58.579Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_5f05ad59-a7f0-4780-a071-ed2775cdf0d9/root/var/svc/log/oxide-crucible-downstairs:default.log
10542024-02-02T03:27:58.579Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_5f05ad59-a7f0-4780-a071-ed2775cdf0d9/root/var/svc/log/oxide-crucible-downstairs:downstairs-c5a77696-8e4c-4173-9c96-651848a0454b.log (5530 bytes)
10552024-02-02T03:27:59.679Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_5f05ad59-a7f0-4780-a071-ed2775cdf0d9/root/var/svc/log/oxide-crucible-downstairs:downstairs-c5a77696-8e4c-4173-9c96-651848a0454b.log
10562024-02-02T03:27:59.679Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_5f05ad59-a7f0-4780-a071-ed2775cdf0d9/root/var/svc/log/oxide-crucible-downstairs:downstairs-cb3359ee-835b-4d7d-a8b0-d004931577a8.log (3049 bytes)
10572024-02-02T03:28:00.774Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_5f05ad59-a7f0-4780-a071-ed2775cdf0d9/root/var/svc/log/oxide-crucible-downstairs:downstairs-cb3359ee-835b-4d7d-a8b0-d004931577a8.log
10582024-02-02T03:28:00.774Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_pantry_1bcd2619-aa8a-49c7-b8bf-d52fab0e71d9/root/var/svc/log/oxide-crucible-pantry:default.log (2338 bytes)
10592024-02-02T03:28:01.871Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_pantry_1bcd2619-aa8a-49c7-b8bf-d52fab0e71d9/root/var/svc/log/oxide-crucible-pantry:default.log
10602024-02-02T03:28:01.871Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_f176cc3a-d69c-446d-927e-39ae8db5a39c/root/var/svc/log/oxide-internal_dns:default.log (103158 bytes)
10612024-02-02T03:28:02.489Zupload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_f176cc3a-d69c-446d-927e-39ae8db5a39c/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 103158 -> 106914
10622024-02-02T03:28:03.234Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_f176cc3a-d69c-446d-927e-39ae8db5a39c/root/var/svc/log/oxide-internal_dns:default.log
10632024-02-02T03:28:03.234Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_f176cc3a-d69c-446d-927e-39ae8db5a39c/root/var/svc/log/oxide-internal_dns:default.log.0 (5998 bytes)
10642024-02-02T03:28:04.343Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_f176cc3a-d69c-446d-927e-39ae8db5a39c/root/var/svc/log/oxide-internal_dns:default.log.0
10652024-02-02T03:28:04.343Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_632ffcfb-8557-4efc-9009-17a48d1c4f52/root/var/svc/log/oxide-cockroachdb:default.log (4408 bytes)
10662024-02-02T03:28:05.456Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_632ffcfb-8557-4efc-9009-17a48d1c4f52/root/var/svc/log/oxide-cockroachdb:default.log
10672024-02-02T03:28:05.456Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_632ffcfb-8557-4efc-9009-17a48d1c4f52/root/var/svc/log/oxide-zone-network-setup:default.log (1711 bytes)
10682024-02-02T03:28:08.281Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_632ffcfb-8557-4efc-9009-17a48d1c4f52/root/var/svc/log/oxide-zone-network-setup:default.log
10692024-02-02T03:28:08.281Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bede069f-b11e-4218-a83f-355a288a17ce/root/var/svc/log/oxide-crucible-agent:default.log (14584 bytes)
10702024-02-02T03:28:09.442Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bede069f-b11e-4218-a83f-355a288a17ce/root/var/svc/log/oxide-crucible-agent:default.log
10712024-02-02T03:28:09.442Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bede069f-b11e-4218-a83f-355a288a17ce/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10722024-02-02T03:28:21.431Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bede069f-b11e-4218-a83f-355a288a17ce/root/var/svc/log/oxide-crucible-downstairs:default.log
10732024-02-02T03:28:21.431Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bede069f-b11e-4218-a83f-355a288a17ce/root/var/svc/log/oxide-crucible-downstairs:downstairs-0295ed86-a904-4f5b-a389-80a8dcc64c59.log (5537 bytes)
10742024-02-02T03:28:22.559Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_bede069f-b11e-4218-a83f-355a288a17ce/root/var/svc/log/oxide-crucible-downstairs:downstairs-0295ed86-a904-4f5b-a389-80a8dcc64c59.log
10752024-02-02T03:28:22.559Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_pantry_01f58fff-6bee-4c25-ab5a-a2924ae7ff1e/root/var/svc/log/oxide-crucible-pantry:default.log (1226092 bytes)
10762024-02-02T03:28:24.125Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_pantry_01f58fff-6bee-4c25-ab5a-a2924ae7ff1e/root/var/svc/log/oxide-crucible-pantry:default.log
10772024-02-02T03:28:24.125Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_27215e88-e4f1-4448-bf03-92854a6b545a/root/var/svc/log/oxide-cockroachdb:default.log (4380 bytes)
10782024-02-02T03:28:25.679Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_27215e88-e4f1-4448-bf03-92854a6b545a/root/var/svc/log/oxide-cockroachdb:default.log
10792024-02-02T03:28:25.679Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_27215e88-e4f1-4448-bf03-92854a6b545a/root/var/svc/log/oxide-zone-network-setup:default.log (1711 bytes)
10802024-02-02T03:28:26.333Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_27215e88-e4f1-4448-bf03-92854a6b545a/root/var/svc/log/oxide-zone-network-setup:default.log
10812024-02-02T03:28:26.334Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_dfd9d160-3ddd-48c2-900a-680b3a5739ea/root/var/svc/log/oxide-crucible-agent:default.log (14116 bytes)
10822024-02-02T03:28:27.455Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_dfd9d160-3ddd-48c2-900a-680b3a5739ea/root/var/svc/log/oxide-crucible-agent:default.log
10832024-02-02T03:28:27.456Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_dfd9d160-3ddd-48c2-900a-680b3a5739ea/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10842024-02-02T03:28:28.547Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_dfd9d160-3ddd-48c2-900a-680b3a5739ea/root/var/svc/log/oxide-crucible-downstairs:default.log
10852024-02-02T03:28:28.547Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_dfd9d160-3ddd-48c2-900a-680b3a5739ea/root/var/svc/log/oxide-crucible-downstairs:downstairs-e6cb7fc0-1dcd-4e36-a166-9219314c9258.log (3047 bytes)
10862024-02-02T03:28:29.640Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_dfd9d160-3ddd-48c2-900a-680b3a5739ea/root/var/svc/log/oxide-crucible-downstairs:downstairs-e6cb7fc0-1dcd-4e36-a166-9219314c9258.log
10872024-02-02T03:28:29.640Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/svc/log/oxide-ntp:default.log (272 bytes)
10882024-02-02T03:28:30.736Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/svc/log/oxide-ntp:default.log
10892024-02-02T03:28:30.736Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/svc/log/oxide-ntp:default.log.0 (915 bytes)
10902024-02-02T03:28:31.844Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/svc/log/oxide-ntp:default.log.0
10912024-02-02T03:28:31.844Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_c0e888bd-9e93-47d5-9184-281ba5aa14d0/root/var/svc/log/oxide-crucible-agent:default.log (9792 bytes)
10922024-02-02T03:28:33.376Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_c0e888bd-9e93-47d5-9184-281ba5aa14d0/root/var/svc/log/oxide-crucible-agent:default.log
10932024-02-02T03:28:33.376Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_c0e888bd-9e93-47d5-9184-281ba5aa14d0/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10942024-02-02T03:28:34.485Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_c0e888bd-9e93-47d5-9184-281ba5aa14d0/root/var/svc/log/oxide-crucible-downstairs:default.log
10952024-02-02T03:28:34.485Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_c0e888bd-9e93-47d5-9184-281ba5aa14d0/root/var/svc/log/oxide-crucible-downstairs:downstairs-e46743a0-49f9-45ed-a53b-f7a40793aae2.log (2944 bytes)
10962024-02-02T03:28:35.587Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_c0e888bd-9e93-47d5-9184-281ba5aa14d0/root/var/svc/log/oxide-crucible-downstairs:downstairs-e46743a0-49f9-45ed-a53b-f7a40793aae2.log
10972024-02-02T03:28:35.587Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d5d5da3c-933b-4a79-9a85-844780875988/root/var/svc/log/oxide-crucible-agent:default.log (9792 bytes)
10982024-02-02T03:28:36.707Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d5d5da3c-933b-4a79-9a85-844780875988/root/var/svc/log/oxide-crucible-agent:default.log
10992024-02-02T03:28:36.707Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d5d5da3c-933b-4a79-9a85-844780875988/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11002024-02-02T03:28:38.250Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d5d5da3c-933b-4a79-9a85-844780875988/root/var/svc/log/oxide-crucible-downstairs:default.log
11012024-02-02T03:28:38.250Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d5d5da3c-933b-4a79-9a85-844780875988/root/var/svc/log/oxide-crucible-downstairs:downstairs-a426a032-e7c5-4cb7-89c6-263c16c5fef6.log (2943 bytes)
11022024-02-02T03:28:39.364Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_d5d5da3c-933b-4a79-9a85-844780875988/root/var/svc/log/oxide-crucible-downstairs:downstairs-a426a032-e7c5-4cb7-89c6-263c16c5fef6.log
11032024-02-02T03:28:39.364Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_pantry_21e36a94-38d5-45fc-ab4b-209e754ca0fc/root/var/svc/log/oxide-crucible-pantry:default.log (2333 bytes)
11042024-02-02T03:28:40.472Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_pantry_21e36a94-38d5-45fc-ab4b-209e754ca0fc/root/var/svc/log/oxide-crucible-pantry:default.log
11052024-02-02T03:28:40.472Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_nexus_64d4576c-1f4f-4c7e-93c5-389d0ae27e57/root/var/svc/log/oxide-nexus:default.log (1186564 bytes)
11062024-02-02T03:28:42.458Zupload warning: file "/pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_nexus_64d4576c-1f4f-4c7e-93c5-389d0ae27e57/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 1186564 -> 1406720
11072024-02-02T03:28:43.540Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_nexus_64d4576c-1f4f-4c7e-93c5-389d0ae27e57/root/var/svc/log/oxide-nexus:default.log
11082024-02-02T03:28:43.540Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8e267ec8-0a68-4db1-9401-3d93e73979ad/root/var/svc/log/oxide-crucible-agent:default.log (20151 bytes)
11092024-02-02T03:28:44.701Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8e267ec8-0a68-4db1-9401-3d93e73979ad/root/var/svc/log/oxide-crucible-agent:default.log
11102024-02-02T03:28:44.701Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8e267ec8-0a68-4db1-9401-3d93e73979ad/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11112024-02-02T03:28:45.803Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8e267ec8-0a68-4db1-9401-3d93e73979ad/root/var/svc/log/oxide-crucible-downstairs:default.log
11122024-02-02T03:28:45.803Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8e267ec8-0a68-4db1-9401-3d93e73979ad/root/var/svc/log/oxide-crucible-downstairs:downstairs-5bb82bf2-4a77-4c19-b33c-fe26618a48be.log (5623 bytes)
11132024-02-02T03:28:46.927Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8e267ec8-0a68-4db1-9401-3d93e73979ad/root/var/svc/log/oxide-crucible-downstairs:downstairs-5bb82bf2-4a77-4c19-b33c-fe26618a48be.log
11142024-02-02T03:28:46.927Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8e267ec8-0a68-4db1-9401-3d93e73979ad/root/var/svc/log/oxide-crucible-downstairs:snapshot-5bb82bf2-4a77-4c19-b33c-fe26618a48be-67e3f87d-f7ac-4c8e-a7f9-d0b6f02f8cba.log (5179 bytes)
11152024-02-02T03:28:48.021Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_8e267ec8-0a68-4db1-9401-3d93e73979ad/root/var/svc/log/oxide-crucible-downstairs:snapshot-5bb82bf2-4a77-4c19-b33c-fe26618a48be-67e3f87d-f7ac-4c8e-a7f9-d0b6f02f8cba.log
11162024-02-02T03:28:48.152Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_3996313f-af0d-4861-9365-e5a7349b7fb7/root/var/svc/log/oxide-nexus:default.log (35983403 bytes)
11172024-02-02T03:29:08.156Zupload warning: file "/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_3996313f-af0d-4861-9365-e5a7349b7fb7/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 35983403 -> 36298022
11182024-02-02T03:29:10.886Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_3996313f-af0d-4861-9365-e5a7349b7fb7/root/var/svc/log/oxide-nexus:default.log
11192024-02-02T03:29:10.886Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_af9a77bd-b2fa-4d26-9dd0-54df7fe3e247/root/var/svc/log/oxide-nexus:default.log (1206647 bytes)
11202024-02-02T03:29:12.502Zupload warning: file "/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_af9a77bd-b2fa-4d26-9dd0-54df7fe3e247/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 1206647 -> 1513471
11212024-02-02T03:29:13.606Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_af9a77bd-b2fa-4d26-9dd0-54df7fe3e247/root/var/svc/log/oxide-nexus:default.log
11222024-02-02T03:29:13.606Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_b4cf0e43-886a-44eb-8562-ff1c8783c4ed/root/var/svc/log/oxide-crucible-agent:default.log (38511 bytes)
11232024-02-02T03:29:15.102Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_b4cf0e43-886a-44eb-8562-ff1c8783c4ed/root/var/svc/log/oxide-crucible-agent:default.log
11242024-02-02T03:29:15.229Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_b4cf0e43-886a-44eb-8562-ff1c8783c4ed/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11252024-02-02T03:29:16.283Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_b4cf0e43-886a-44eb-8562-ff1c8783c4ed/root/var/svc/log/oxide-crucible-downstairs:default.log
11262024-02-02T03:29:16.283Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_b4cf0e43-886a-44eb-8562-ff1c8783c4ed/root/var/svc/log/oxide-crucible-downstairs:downstairs-6034cee5-0613-4a69-ad22-5514069da8f7.log (3046 bytes)
11272024-02-02T03:29:17.494Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_b4cf0e43-886a-44eb-8562-ff1c8783c4ed/root/var/svc/log/oxide-crucible-downstairs:downstairs-6034cee5-0613-4a69-ad22-5514069da8f7.log
11282024-02-02T03:29:17.494Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_b4cf0e43-886a-44eb-8562-ff1c8783c4ed/root/var/svc/log/oxide-crucible-downstairs:downstairs-61c73bc0-f061-4dd0-8cd1-8789bfc6b071.log (5537 bytes)
11292024-02-02T03:29:19.340Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_b4cf0e43-886a-44eb-8562-ff1c8783c4ed/root/var/svc/log/oxide-crucible-downstairs:downstairs-61c73bc0-f061-4dd0-8cd1-8789bfc6b071.log
11302024-02-02T03:29:19.340Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_b4cf0e43-886a-44eb-8562-ff1c8783c4ed/root/var/svc/log/oxide-crucible-downstairs:downstairs-747c1f60-0b56-4d70-ac84-92e9b99e6820.log (5623 bytes)
11312024-02-02T03:29:20.476Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_b4cf0e43-886a-44eb-8562-ff1c8783c4ed/root/var/svc/log/oxide-crucible-downstairs:downstairs-747c1f60-0b56-4d70-ac84-92e9b99e6820.log
11322024-02-02T03:29:20.476Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_b4cf0e43-886a-44eb-8562-ff1c8783c4ed/root/var/svc/log/oxide-crucible-downstairs:snapshot-747c1f60-0b56-4d70-ac84-92e9b99e6820-67e3f87d-f7ac-4c8e-a7f9-d0b6f02f8cba.log (5176 bytes)
11332024-02-02T03:29:39.111Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_b4cf0e43-886a-44eb-8562-ff1c8783c4ed/root/var/svc/log/oxide-crucible-downstairs:snapshot-747c1f60-0b56-4d70-ac84-92e9b99e6820-67e3f87d-f7ac-4c8e-a7f9-d0b6f02f8cba.log
11342024-02-02T03:29:39.111Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/log/chrony/measurements.log (27537 bytes)
11352024-02-02T03:29:39.633Zupload warning: file "/pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/log/chrony/measurements.log" changed size mid upload: 27537 -> 34250
11362024-02-02T03:29:40.727Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/log/chrony/measurements.log
11372024-02-02T03:29:40.727Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/log/chrony/statistics.log (17784 bytes)
11382024-02-02T03:29:40.820Zupload warning: file "/pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/log/chrony/statistics.log" changed size mid upload: 17784 -> 23166
11392024-02-02T03:29:41.893Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/log/chrony/statistics.log
11402024-02-02T03:29:41.893Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/log/chrony/tracking.log (2250 bytes)
11412024-02-02T03:29:41.944Zupload warning: file "/pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/log/chrony/tracking.log" changed size mid upload: 2250 -> 2646
11422024-02-02T03:29:43.025Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_ntp_43646849-4b66-4fd4-8e91-88c4322bccd4/root/var/log/chrony/tracking.log