01HPNF5CXQVE1N86NGJKYF8961: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log*",
        "%/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.28",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01HPNF62QQACGTQ7JTHF5DN3AC

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12024-02-15T04:47:55.255Zjob dependencies complete; ready to run (waiting for 40 m 3 s)
22024-02-15T05:46:35.056Zjob assigned to worker 01HPNMSP961YRP7VABSJKAAD88 (queued for 58 m 39 s)
32024-02-15T05:46:35.181Zhttp://172.20.0.5:9969/ipxe/buskin... ok
42024-02-15T05:46:35.232Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/kernel/amd64/unix... ok
52024-02-15T05:46:35.282Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive... ok
62024-02-15T05:46:35.337Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive.hash... ok
72024-02-15T05:46:35.389Zhttp://172.20.0.5:9969/postboot/buskin... ok
82024-02-15T05:46:35.443ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
92024-02-15T05:46:35.500Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d1
102024-02-15T05:46:35.543ZOxide Helios Version helios-2.0.22451 64-bit
112024-02-15T05:46:35.594ZConfiguring devices.
122024-02-15T05:46:35.647Z##################################################
132024-02-15T05:46:35.693Z#### ############# ##
142024-02-15T05:46:35.740Z### ### ############ ##
152024-02-15T05:46:35.783Z## ### # ## ### ## Oxide Computer Company ##
162024-02-15T05:46:35.845Z## ## ## ### # ### ##
172024-02-15T05:46:35.899Z## # ### #### #### This Station Under ##
182024-02-15T05:46:35.945Z### ### #### # ### Computer Control ##
192024-02-15T05:46:35.982Z#### #### ### ## ##
202024-02-15T05:46:36.021Z##################################################
212024-02-15T05:46:36.060ZHostname: buskin
222024-02-15T05:46:36.102ZLoading smf(7) service descriptions:
232024-02-15T05:46:36.713Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap-4ef928d4a311d736.gz
242024-02-15T05:46:40.361Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap-4ef928d4a311d736.gz
252024-02-15T05:46:40.361Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
262024-02-15T05:47:02.906Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
272024-02-15T05:47:02.906Zdownloading input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-d04a49cb4bbc61f6.gz
282024-02-15T05:47:26.353Zdownloaded input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-d04a49cb4bbc61f6.gz
292024-02-15T05:47:26.354Zdownloading input: /input/ci-tools/work/caboose-util.gz
302024-02-15T05:47:27.376Zdownloaded input: /input/ci-tools/work/caboose-util.gz
312024-02-15T05:47:27.403Zdownloading input: /input/ci-tools/work/tufaceous.gz
322024-02-15T05:47:30.444Zdownloaded input: /input/ci-tools/work/tufaceous.gz
332024-02-15T05:47:30.468Zdownloading input: /input/package/work/version.txt
342024-02-15T05:47:30.492Zdownloaded input: /input/package/work/version.txt
352024-02-15T05:47:30.492Zdownloading input: /input/package/work/package.tar.gz
362024-02-15T05:47:49.953Zdownloaded input: /input/package/work/package.tar.gz
372024-02-15T05:47:49.953Zdownloading input: /input/package/work/global-zone-packages.tar.gz
382024-02-15T05:48:20.924Zdownloaded input: /input/package/work/global-zone-packages.tar.gz
392024-02-15T05:48:20.924Zdownloading input: /input/package/work/trampoline-global-zone-packages.tar.gz
402024-02-15T05:48:29.017Zdownloaded input: /input/package/work/trampoline-global-zone-packages.tar.gz
412024-02-15T05:48:29.041Zdownloading input: /input/package/work/zones/clickhouse.tar.gz
422024-02-15T05:49:11.765Zdownloaded input: /input/package/work/zones/clickhouse.tar.gz
432024-02-15T05:49:11.791Zdownloading input: /input/package/work/zones/clickhouse_keeper.tar.gz
442024-02-15T05:49:50.621Zdownloaded input: /input/package/work/zones/clickhouse_keeper.tar.gz
452024-02-15T05:49:50.621Zdownloading input: /input/package/work/zones/cockroachdb.tar.gz
462024-02-15T05:50:23.294Zdownloaded input: /input/package/work/zones/cockroachdb.tar.gz
472024-02-15T05:50:23.319Zdownloading input: /input/package/work/zones/crucible-pantry.tar.gz
482024-02-15T05:50:26.251Zdownloaded input: /input/package/work/zones/crucible-pantry.tar.gz
492024-02-15T05:50:26.251Zdownloading input: /input/package/work/zones/crucible.tar.gz
502024-02-15T05:50:31.730Zdownloaded input: /input/package/work/zones/crucible.tar.gz
512024-02-15T05:50:31.730Zdownloading input: /input/package/work/zones/external-dns.tar.gz
522024-02-15T05:50:40.315Zdownloaded input: /input/package/work/zones/external-dns.tar.gz
532024-02-15T05:50:40.315Zdownloading input: /input/package/work/zones/internal-dns.tar.gz
542024-02-15T05:50:45.785Zdownloaded input: /input/package/work/zones/internal-dns.tar.gz
552024-02-15T05:50:45.785Zdownloading input: /input/package/work/zones/ntp.tar.gz
562024-02-15T05:50:45.821Zdownloaded input: /input/package/work/zones/ntp.tar.gz
572024-02-15T05:50:45.845Zdownloading input: /input/package/work/zones/omicron-gateway-asic.tar.gz
582024-02-15T05:50:50.223Zdownloaded input: /input/package/work/zones/omicron-gateway-asic.tar.gz
592024-02-15T05:50:50.223Zdownloading input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
602024-02-15T05:50:54.367Zdownloaded input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
612024-02-15T05:50:54.367Zdownloading input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
622024-02-15T05:51:10.477Zdownloaded input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
632024-02-15T05:51:10.502Zdownloading input: /input/package/work/zones/omicron-nexus.tar.gz
642024-02-15T05:51:26.234Zdownloaded input: /input/package/work/zones/omicron-nexus.tar.gz
652024-02-15T05:51:26.235Zdownloading input: /input/package/work/zones/overlay.tar.gz
662024-02-15T05:51:26.379Zdownloaded input: /input/package/work/zones/overlay.tar.gz
672024-02-15T05:51:26.379Zdownloading input: /input/package/work/zones/oximeter.tar.gz
682024-02-15T05:51:45.956Zdownloaded input: /input/package/work/zones/oximeter.tar.gz
692024-02-15T05:51:45.956Zdownloading input: /input/package/work/zones/propolis-server.tar.gz
702024-02-15T05:52:04.567Zdownloaded input: /input/package/work/zones/propolis-server.tar.gz
712024-02-15T05:52:04.567Zdownloading input: /input/package/work/zones/switch-asic.tar.gz
722024-02-15T05:53:08.043Zdownloaded input: /input/package/work/zones/switch-asic.tar.gz
732024-02-15T05:53:08.043Zdownloading input: /input/package/work/zones/switch-softnpu.tar.gz
742024-02-15T05:53:50.657Zdownloaded input: /input/package/work/zones/switch-softnpu.tar.gz
 
752024-02-15T05:53:50.682Zstarting task 0: "setup"
762024-02-15T05:53:50.682Z++ uname -s
772024-02-15T05:53:50.682Z+ kern=SunOS
782024-02-15T05:53:50.682Z+ case "$kern" in
792024-02-15T05:53:50.683Z+ groupadd -g 12345 build
802024-02-15T05:53:50.683Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
812024-02-15T05:53:52.673Z+ zfs create -o mountpoint=/work rpool/work
822024-02-15T05:53:52.704Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
832024-02-15T05:53:52.728Z+ home_fs=zfs
842024-02-15T05:53:52.728Z+ [[ zfs == autofs ]]
852024-02-15T05:53:52.728Z+ mkdir -p /home/build
862024-02-15T05:53:52.728Z+ chown build:build /home/build /work
872024-02-15T05:53:54.712Z+ chmod 0700 /home/build /work
882024-02-15T05:53:54.747Zprocess exited: duration 4058 ms, exit code 0
 
892024-02-15T05:53:54.808Zstarting task 1: "authentication"
902024-02-15T05:53:54.862Zprocess exited: duration 45 ms, exit code 0
 
912024-02-15T05:53:54.919Zstarting task 2: "build"
922024-02-15T05:53:54.945Z+ trap _exit_trap EXIT
932024-02-15T05:53:54.945Z+ set +x
942024-02-15T05:54:24.851Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
952024-02-15T05:54:25.100Zsecpolicy_fs_linkdir: 0x55 = 0x55
962024-02-15T05:54:25.100Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
972024-02-15T05:54:25.101Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
982024-02-15T05:54:25.101Zsecpolicy_fs_linkdir+3: 0 = 0xe5
992024-02-15T05:54:25.145Zsecpolicy_fs_linkdir+4: 0 = 0xb8
1002024-02-15T05:54:25.145Zsecpolicy_fs_linkdir+5: 0 = 0x1
1012024-02-15T05:54:25.146Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
1022024-02-15T05:54:25.146Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
1032024-02-15T05:54:25.146Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
1042024-02-15T05:54:25.146Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
1052024-02-15T05:54:25.146Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
1062024-02-15T05:54:25.146Z+ [[ -d /opt/oxide ]]
1072024-02-15T05:54:25.146Z+ pfexec mv /opt/oxide /opt/oxide-underneath
1082024-02-15T05:54:25.146Z+ pfexec mkdir /opt/oxide
1092024-02-15T05:54:25.146Z+ pfexec mount -F tmpfs -O swap /opt/oxide
1102024-02-15T05:54:25.146Z+ [[ -d /opt/oxide-underneath ]]
1112024-02-15T05:54:25.146Z+ cd /opt/oxide-underneath
1122024-02-15T05:54:25.146Z+ pfexec tar ceEp@/f - .
1132024-02-15T05:54:25.146Z+ cd /opt/oxide
1142024-02-15T05:54:25.146Z+ pfexec tar xveEp@/f -
1152024-02-15T05:54:25.177Zx ., 0 bytes, 0 tape blocks
1162024-02-15T05:54:25.177Zx ./opte, 0 bytes, 0 tape blocks
1172024-02-15T05:54:25.177Zx ./opte/bin, 0 bytes, 0 tape blocks
1182024-02-15T05:54:25.177Zx ./opte/bin/opteadm, 51799224 bytes, 101171 tape blocks
1192024-02-15T05:54:25.177Z+ pfexec rm -rf /opt/oxide-underneath
1202024-02-15T05:54:25.177Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
1212024-02-15T05:54:25.208Z+ pfexec mkdir /opt/oxide/work
1222024-02-15T05:54:25.208Z+ pfexec chown build:build /opt/oxide/work
1232024-02-15T05:54:25.208Z+ cd /opt/oxide/work
1242024-02-15T05:54:25.208Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
1252024-02-15T05:54:26.920ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
1262024-02-15T05:54:26.920Zx out/mg-ddm-gz.tar, 69308416 bytes, 135368 tape blocks
1272024-02-15T05:54:26.920Zx out/omicron-sled-agent.tar, 122800640 bytes, 239845 tape blocks
1282024-02-15T05:54:26.920Zx out/oxlog.tar, 8382464 bytes, 16372 tape blocks
1292024-02-15T05:54:26.921Zx out/target/test, 75 bytes, 1 tape blocks
1302024-02-15T05:54:26.921Zx out/npuzone/npuzone, 37184104 bytes, 72626 tape blocks
1312024-02-15T05:54:26.921Zx package-manifest.toml, 23732 bytes, 47 tape blocks
1322024-02-15T05:54:26.921Zx smf/sled-agent/non-gimlet/config.toml, 3448 bytes, 7 tape blocks
1332024-02-15T05:54:26.921Zx target/release/omicron-package, 45722256 bytes, 89302 tape blocks
1342024-02-15T05:54:26.921Zx tools/create_virtual_hardware.sh, 3184 bytes, 7 tape blocks
1352024-02-15T05:54:26.921Zx tools/virtual_hardware.sh, 3416 bytes, 7 tape blocks
1362024-02-15T05:54:26.922Zx tools/scrimlet/create-softnpu-zone.sh, 627 bytes, 2 tape blocks
1372024-02-15T05:54:26.922Zx tools/scrimlet/destroy-softnpu-zone.sh, 203 bytes, 1 tape blocks
1382024-02-15T05:54:26.922Zx tools/scrimlet/softnpu-init.sh, 2157 bytes, 5 tape blocks
1392024-02-15T05:54:26.922Zx tools/scrimlet/softnpu-zone.txt, 330 bytes, 1 tape blocks
1402024-02-15T05:54:26.922Zx tools/scrimlet/softnpu.toml, 171 bytes, 1 tape blocks
1412024-02-15T05:54:26.951Z
1422024-02-15T05:54:26.951Zreal 1.714530818
1432024-02-15T05:54:26.951Zuser 1.541682186
1442024-02-15T05:54:26.952Zsys 0.308155452
1452024-02-15T05:54:26.953Ztrap 0.000070301
1462024-02-15T05:54:26.953Ztflt 0.000000000
1472024-02-15T05:54:26.954Zdflt 0.000000000
1482024-02-15T05:54:26.958Zkflt 0.000000000
1492024-02-15T05:54:26.958Zlock 0.000000000
1502024-02-15T05:54:26.958Zslp 1.538728787
1512024-02-15T05:54:26.958Zlat 0.038161694
1522024-02-15T05:54:26.959Zstop 0.000132623
1532024-02-15T05:54:26.959Z+ cp /input/package/work/zones/clickhouse_keeper.tar.gz /input/package/work/zones/clickhouse.tar.gz /input/package/work/zones/cockroachdb.tar.gz /input/package/work/zones/crucible-pantry.tar.gz /input/package/work/zones/crucible.tar.gz /input/package/work/zones/external-dns.tar.gz /input/package/work/zones/internal-dns.tar.gz /input/package/work/zones/ntp.tar.gz /input/package/work/zones/omicron-gateway-asic.tar.gz /input/package/work/zones/omicron-gateway-softnpu.tar.gz /input/package/work/zones/omicron-nexus-single-sled.tar.gz /input/package/work/zones/omicron-nexus.tar.gz /input/package/work/zones/overlay.tar.gz /input/package/work/zones/oximeter.tar.gz /input/package/work/zones/propolis-server.tar.gz /input/package/work/zones/switch-asic.tar.gz /input/package/work/zones/switch-softnpu.tar.gz out/
1542024-02-15T05:54:28.102Z+ mv out/omicron-nexus-single-sled.tar.gz out/omicron-nexus.tar.gz
1552024-02-15T05:54:28.127Z+ mkdir tests
1562024-02-15T05:54:28.127Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1572024-02-15T05:54:28.127Z+ ptime -m gunzip
1582024-02-15T05:54:28.127Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-4ef928d4a311d736
1592024-02-15T05:54:28.358Z
1602024-02-15T05:54:28.386Zreal 0.233361805
1612024-02-15T05:54:28.386Zuser 0.213562172
1622024-02-15T05:54:28.386Zsys 0.019554144
1632024-02-15T05:54:28.386Ztrap 0.000021010
1642024-02-15T05:54:28.386Ztflt 0.000000000
1652024-02-15T05:54:28.386Zdflt 0.000000000
1662024-02-15T05:54:28.386Zkflt 0.000000000
1672024-02-15T05:54:28.386Zlock 0.000000000
1682024-02-15T05:54:28.386Zslp 0.000018550
1692024-02-15T05:54:28.387Zlat 0.000022340
1702024-02-15T05:54:28.387Zstop 0.000123904
1712024-02-15T05:54:28.387Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-4ef928d4a311d736
1722024-02-15T05:54:28.387Z+ chmod a+x tests/bootstrap-4ef928d4a311d736
1732024-02-15T05:54:28.387Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1742024-02-15T05:54:28.387Z+ ptime -m gunzip
1752024-02-15T05:54:28.387Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1762024-02-15T05:54:30.869Z
1772024-02-15T05:54:30.909Zreal 2.498124087
1782024-02-15T05:54:30.910Zuser 2.342636864
1792024-02-15T05:54:30.910Zsys 0.155088771
1802024-02-15T05:54:30.910Ztrap 0.000166075
1812024-02-15T05:54:30.910Ztflt 0.000000000
1822024-02-15T05:54:30.910Zdflt 0.000000000
1832024-02-15T05:54:30.910Zkflt 0.000000000
1842024-02-15T05:54:30.911Zlock 0.000000000
1852024-02-15T05:54:30.911Zslp 0.000022940
1862024-02-15T05:54:30.911Zlat 0.000026220
1872024-02-15T05:54:30.912Zstop 0.000116783
1882024-02-15T05:54:30.912Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1892024-02-15T05:54:30.912Z+ chmod a+x tests/bootstrap
1902024-02-15T05:54:30.912Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1912024-02-15T05:54:30.912Z+ ptime -m gunzip
1922024-02-15T05:54:30.912Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-d04a49cb4bbc61f6
1932024-02-15T05:54:33.440Z
1942024-02-15T05:54:33.475Zreal 2.562243905
1952024-02-15T05:54:33.475Zuser 2.401518712
1962024-02-15T05:54:33.475Zsys 0.160269679
1972024-02-15T05:54:33.475Ztrap 0.000199795
1982024-02-15T05:54:33.475Ztflt 0.000000000
1992024-02-15T05:54:33.475Zdflt 0.000000000
2002024-02-15T05:54:33.475Zkflt 0.000000000
2012024-02-15T05:54:33.475Zlock 0.000000000
2022024-02-15T05:54:33.475Zslp 0.000019770
2032024-02-15T05:54:33.475Zlat 0.000023640
2042024-02-15T05:54:33.475Zstop 0.000147394
2052024-02-15T05:54:33.475Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-d04a49cb4bbc61f6
2062024-02-15T05:54:33.475Z+ chmod a+x tests/end_to_end_tests-d04a49cb4bbc61f6
2072024-02-15T05:54:33.475Z+ bmat address ls
2082024-02-15T05:54:33.475ZNAME CIDR FIRST LAST COUNT
2092024-02-15T05:54:33.475Zextra 10.151.1.0/24 10.151.1.164 10.151.1.227 64
2102024-02-15T05:54:33.475Z++ bmat address ls -f extra -Ho first,last,count
2112024-02-15T05:54:33.475Z+ set -- 10.151.1.164 10.151.1.227 64
2122024-02-15T05:54:33.475Z+ EXTRA_IP_START=10.151.1.164
2132024-02-15T05:54:33.475Z+ EXTRA_IP_END=10.151.1.227
2142024-02-15T05:54:33.475Z+ EXTRA_IP_COUNT=64
2152024-02-15T05:54:33.475Z+ (( EXTRA_IP_COUNT >= 32 ))
2162024-02-15T05:54:33.476Z+ EXTRA_IP_BASE=10.151.1
2172024-02-15T05:54:33.476Z+ EXTRA_IP_FOCTET=164
2182024-02-15T05:54:33.476Z+ EXTRA_IP_LOCTET=227
2192024-02-15T05:54:33.476Z+ SERVICE_IP_POOL_START=10.151.1.164
2202024-02-15T05:54:33.476Z+ SERVICE_IP_POOL_END=10.151.1.173
2212024-02-15T05:54:33.476Z+ DNS_IP1=10.151.1.164
2222024-02-15T05:54:33.476Z+ DNS_IP2=10.151.1.165
2232024-02-15T05:54:33.476Z+ UPLINK_IP=10.151.1.174
2242024-02-15T05:54:33.476Z+ IPPOOL_START=10.151.1.175
2252024-02-15T05:54:33.476Z+ IPPOOL_END=10.151.1.227
2262024-02-15T05:54:33.476Z++ ipadm show-addr -po type,addr
2272024-02-15T05:54:33.476Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
2282024-02-15T05:54:33.503Z+ GATEWAY_IP=10.151.1.100
2292024-02-15T05:54:33.503Z+ [[ -n 10.151.1.100 ]]
2302024-02-15T05:54:33.503Z+ ping -s 10.151.1.100 56 1
2312024-02-15T05:54:33.503ZPING 10.151.1.100: 56 data bytes
2322024-02-15T05:54:33.503Z64 bytes from 10.151.1.100: icmp_seq=0. time=0.011 ms
2332024-02-15T05:54:33.503Z
2342024-02-15T05:54:33.503Z----10.151.1.100 PING Statistics----
2352024-02-15T05:54:33.503Z1 packets transmitted, 1 packets received, 0% packet loss
2362024-02-15T05:54:33.503Zround-trip (ms) min/avg/max/stddev = 0.011/0.011/0.011/-nan
2372024-02-15T05:54:33.503Z++ arp -an
2382024-02-15T05:54:33.504Z++ awk -vgw=10.151.1.100 '$2 == gw {print $NF}'
2392024-02-15T05:54:33.504Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
2402024-02-15T05:54:33.504Z+ routeadm -e ipv4-forwarding -u
2412024-02-15T05:54:33.603Z+ PXA_START=10.151.1.164
2422024-02-15T05:54:33.604Z+ PXA_END=10.151.1.227
2432024-02-15T05:54:33.636Z+ export GATEWAY_IP GATEWAY_MAC PXA_START PXA_END
2442024-02-15T05:54:33.636Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
2452024-02-15T05:54:33.803Z+ ZPOOL_VDEV_DIR=/scratch
2462024-02-15T05:54:33.803Z+ ptime -m pfexec ./tools/create_virtual_hardware.sh
2472024-02-15T05:54:33.831Z+++ dirname ./tools/create_virtual_hardware.sh
2482024-02-15T05:54:33.831Z++ cd ./tools
2492024-02-15T05:54:33.831Z++ pwd
2502024-02-15T05:54:33.831Z+ SOURCE_DIR=/opt/oxide/work/tools
2512024-02-15T05:54:33.831Z+ OMICRON_TOP=/opt/oxide/work/tools/..
2522024-02-15T05:54:33.831Z+ SOFTNPU_MODE=zone
2532024-02-15T05:54:33.831Z+ . /opt/oxide/work/tools/virtual_hardware.sh
2542024-02-15T05:54:33.831Z++ MARKER=/etc/opt/oxide/NO_INSTALL
2552024-02-15T05:54:33.831Z++ [[ -f /etc/opt/oxide/NO_INSTALL ]]
2562024-02-15T05:54:33.831Z++ dladm show-phys -p -o LINK
2572024-02-15T05:54:33.831Z++ head -1
2582024-02-15T05:54:33.831Z+ PHYSICAL_LINK=igb0
2592024-02-15T05:54:33.831Z+ echo 'Using igb0 as physical link'
2602024-02-15T05:54:33.831ZUsing igb0 as physical link
2612024-02-15T05:54:33.831Z+ ensure_run_as_root
2622024-02-15T05:54:33.831Z++ id -u
2632024-02-15T05:54:33.831Z+ [[ 0 -ne 0 ]]
2642024-02-15T05:54:33.831Z+ ensure_zpools
2652024-02-15T05:54:33.831Z+ ZPOOL_TYPES=('oxp_' 'oxi_')
2662024-02-15T05:54:33.831Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
2672024-02-15T05:54:33.832Z+ readarray -t ZPOOLS
2682024-02-15T05:54:33.861Z++ grep '"oxp_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
2692024-02-15T05:54:33.861Z++ sed 's/[ ",]//g'
2702024-02-15T05:54:33.861Z+ for ZPOOL in "${ZPOOLS[@]}"
2712024-02-15T05:54:33.861Z+ echo 'Zpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]'
2722024-02-15T05:54:33.861ZZpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]
2732024-02-15T05:54:33.861Z+ VDEV_PATH=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2742024-02-15T05:54:33.861Z+ [[ -f /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
2752024-02-15T05:54:33.861Z+ dd if=/dev/zero of=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
2762024-02-15T05:54:33.861Z0+0 records in
2772024-02-15T05:54:33.861Z0+0 records out
2782024-02-15T05:54:33.861Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
2792024-02-15T05:54:33.861Z+ success 'ZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
2802024-02-15T05:54:33.861Z+ set +x
2812024-02-15T05:54:33.861ZZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
2822024-02-15T05:54:33.861Z++ zpool list -o name
2832024-02-15T05:54:33.861Z++ grep oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
2842024-02-15T05:54:33.861Z+ [[ -z '' ]]
2852024-02-15T05:54:33.862Z+ zpool create -f oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2862024-02-15T05:54:33.891Z+ success 'ZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
2872024-02-15T05:54:33.891Z+ set +x
2882024-02-15T05:54:33.891ZZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists
2892024-02-15T05:54:33.891Z+ for ZPOOL in "${ZPOOLS[@]}"
2902024-02-15T05:54:33.891Z+ echo 'Zpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
2912024-02-15T05:54:33.891ZZpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]
2922024-02-15T05:54:33.891Z+ VDEV_PATH=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
2932024-02-15T05:54:33.891Z+ [[ -f /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
2942024-02-15T05:54:33.891Z+ dd if=/dev/zero of=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
2952024-02-15T05:54:33.891Z0+0 records in
2962024-02-15T05:54:33.891Z0+0 records out
2972024-02-15T05:54:33.891Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
2982024-02-15T05:54:33.891Z+ success 'ZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
2992024-02-15T05:54:33.891Z+ set +x
3002024-02-15T05:54:33.891ZZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3012024-02-15T05:54:33.891Z++ zpool list -o name
3022024-02-15T05:54:33.891Z++ grep oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
3032024-02-15T05:54:33.919Z+ [[ -z '' ]]
3042024-02-15T05:54:33.919Z+ zpool create -f oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3052024-02-15T05:54:33.948Z+ success 'ZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3062024-02-15T05:54:33.949Z+ set +x
3072024-02-15T05:54:33.949ZZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3082024-02-15T05:54:33.950Z+ for ZPOOL in "${ZPOOLS[@]}"
3092024-02-15T05:54:33.950Z+ echo 'Zpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3102024-02-15T05:54:33.950ZZpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]
3112024-02-15T05:54:33.950Z+ VDEV_PATH=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3122024-02-15T05:54:33.950Z+ [[ -f /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3132024-02-15T05:54:33.951Z+ dd if=/dev/zero of=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3142024-02-15T05:54:33.951Z0+0 records in
3152024-02-15T05:54:33.951Z0+0 records out
3162024-02-15T05:54:33.951Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3172024-02-15T05:54:33.951Z+ success 'ZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3182024-02-15T05:54:33.951Z+ set +x
3192024-02-15T05:54:33.951ZZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3202024-02-15T05:54:33.951Z++ zpool list -o name
3212024-02-15T05:54:33.951Z++ grep oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
3222024-02-15T05:54:33.951Z+ [[ -z '' ]]
3232024-02-15T05:54:33.951Z+ zpool create -f oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3242024-02-15T05:54:33.991Z+ success 'ZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3252024-02-15T05:54:33.991Z+ set +x
3262024-02-15T05:54:33.991ZZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3272024-02-15T05:54:33.991Z+ for ZPOOL in "${ZPOOLS[@]}"
3282024-02-15T05:54:33.992ZZpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]
3292024-02-15T05:54:33.992Z+ echo 'Zpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3302024-02-15T05:54:33.992Z+ VDEV_PATH=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3312024-02-15T05:54:33.992Z+ [[ -f /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3322024-02-15T05:54:33.992Z+ dd if=/dev/zero of=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3332024-02-15T05:54:33.992Z0+0 records in
3342024-02-15T05:54:33.992Z0+0 records out
3352024-02-15T05:54:33.993Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3362024-02-15T05:54:33.993Z+ success 'ZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3372024-02-15T05:54:33.993Z+ set +x
3382024-02-15T05:54:33.993ZZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3392024-02-15T05:54:33.993Z++ zpool list -o name
3402024-02-15T05:54:33.994Z++ grep oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03
3412024-02-15T05:54:33.994Z+ [[ -z '' ]]
3422024-02-15T05:54:33.994Z+ zpool create -f oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3432024-02-15T05:54:34.020Z+ success 'ZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3442024-02-15T05:54:34.020Z+ set +x
3452024-02-15T05:54:34.020ZZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3462024-02-15T05:54:34.020Z+ for ZPOOL in "${ZPOOLS[@]}"
3472024-02-15T05:54:34.021Z+ echo 'Zpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3482024-02-15T05:54:34.021ZZpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]
3492024-02-15T05:54:34.021Z+ VDEV_PATH=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3502024-02-15T05:54:34.021Z+ [[ -f /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3512024-02-15T05:54:34.021Z+ dd if=/dev/zero of=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3522024-02-15T05:54:34.021Z0+0 records in
3532024-02-15T05:54:34.021Z0+0 records out
3542024-02-15T05:54:34.021Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3552024-02-15T05:54:34.021Z+ success 'ZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3562024-02-15T05:54:34.021Z+ set +x
3572024-02-15T05:54:34.021ZZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3582024-02-15T05:54:34.021Z++ zpool list -o name
3592024-02-15T05:54:34.021Z++ grep oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03
3602024-02-15T05:54:34.021Z+ [[ -z '' ]]
3612024-02-15T05:54:34.021Z+ zpool create -f oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3622024-02-15T05:54:34.056Z+ success 'ZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3632024-02-15T05:54:34.057Z+ set +x
3642024-02-15T05:54:34.057ZZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3652024-02-15T05:54:34.057Z+ for ZPOOL in "${ZPOOLS[@]}"
3662024-02-15T05:54:34.057Z+ echo 'Zpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]'
3672024-02-15T05:54:34.057ZZpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]
3682024-02-15T05:54:34.057Z+ VDEV_PATH=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3692024-02-15T05:54:34.057Z+ [[ -f /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev ]]
3702024-02-15T05:54:34.057Z+ dd if=/dev/zero of=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev bs=1 count=0 seek=20G
3712024-02-15T05:54:34.057Z0+0 records in
3722024-02-15T05:54:34.057Z0+0 records out
3732024-02-15T05:54:34.058Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3742024-02-15T05:54:34.058Z+ success 'ZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists'
3752024-02-15T05:54:34.058Z+ set +x
3762024-02-15T05:54:34.058ZZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists
3772024-02-15T05:54:34.086Z++ zpool list -o name
3782024-02-15T05:54:34.086Z++ grep oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930
3792024-02-15T05:54:34.086Z+ [[ -z '' ]]
3802024-02-15T05:54:34.087Z+ zpool create -f oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3812024-02-15T05:54:34.087Z+ success 'ZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists'
3822024-02-15T05:54:34.087Z+ set +x
3832024-02-15T05:54:34.087ZZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists
3842024-02-15T05:54:34.087Z+ for ZPOOL in "${ZPOOLS[@]}"
3852024-02-15T05:54:34.087Z+ echo 'Zpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]'
3862024-02-15T05:54:34.087ZZpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]
3872024-02-15T05:54:34.087Z+ VDEV_PATH=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
3882024-02-15T05:54:34.087Z+ [[ -f /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev ]]
3892024-02-15T05:54:34.087Z+ dd if=/dev/zero of=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev bs=1 count=0 seek=20G
3902024-02-15T05:54:34.112Z0+0 records in
3912024-02-15T05:54:34.112Z0+0 records out
3922024-02-15T05:54:34.112Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3932024-02-15T05:54:34.112Z+ success 'ZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists'
3942024-02-15T05:54:34.112Z+ set +x
3952024-02-15T05:54:34.112ZZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists
3962024-02-15T05:54:34.112Z++ zpool list -o name
3972024-02-15T05:54:34.112Z++ grep oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60
3982024-02-15T05:54:34.112Z+ [[ -z '' ]]
3992024-02-15T05:54:34.112Z+ zpool create -f oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
4002024-02-15T05:54:34.160Z+ success 'ZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists'
4012024-02-15T05:54:34.160Z+ set +x
4022024-02-15T05:54:34.160ZZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists
4032024-02-15T05:54:34.160Z+ for ZPOOL in "${ZPOOLS[@]}"
4042024-02-15T05:54:34.160Z+ echo 'Zpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]'
4052024-02-15T05:54:34.160ZZpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]
4062024-02-15T05:54:34.160Z+ VDEV_PATH=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4072024-02-15T05:54:34.160Z+ [[ -f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev ]]
4082024-02-15T05:54:34.161Z+ dd if=/dev/zero of=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev bs=1 count=0 seek=20G
4092024-02-15T05:54:34.161Z0+0 records in
4102024-02-15T05:54:34.161Z0+0 records out
4112024-02-15T05:54:34.161Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4122024-02-15T05:54:34.161Z+ success 'ZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists'
4132024-02-15T05:54:34.161Z+ set +x
4142024-02-15T05:54:34.161ZZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists
4152024-02-15T05:54:34.161Z++ zpool list -o name
4162024-02-15T05:54:34.161Z++ grep oxp_31bd71cd-4736-4a12-a387-9b74b050396f
4172024-02-15T05:54:34.161Z+ [[ -z '' ]]
4182024-02-15T05:54:34.162Z+ zpool create -f oxp_31bd71cd-4736-4a12-a387-9b74b050396f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4192024-02-15T05:54:34.208Z+ success 'ZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists'
4202024-02-15T05:54:34.208Z+ set +x
4212024-02-15T05:54:34.208ZZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists
4222024-02-15T05:54:34.208Z+ for ZPOOL in "${ZPOOLS[@]}"
4232024-02-15T05:54:34.208Z+ echo 'Zpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]'
4242024-02-15T05:54:34.208ZZpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]
4252024-02-15T05:54:34.208Z+ VDEV_PATH=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4262024-02-15T05:54:34.208Z+ [[ -f /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev ]]
4272024-02-15T05:54:34.208Z+ dd if=/dev/zero of=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev bs=1 count=0 seek=20G
4282024-02-15T05:54:34.208Z0+0 records in
4292024-02-15T05:54:34.208Z0+0 records out
4302024-02-15T05:54:34.208Z0 bytes (0 iB) transferred in 0.000005 secs (0 iB/sec)
4312024-02-15T05:54:34.208Z+ success 'ZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists'
4322024-02-15T05:54:34.208Z+ set +x
4332024-02-15T05:54:34.208ZZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists
4342024-02-15T05:54:34.209Z++ zpool list -o name
4352024-02-15T05:54:34.209Z++ grep oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7
4362024-02-15T05:54:34.209Z+ [[ -z '' ]]
4372024-02-15T05:54:34.209Z+ zpool create -f oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4382024-02-15T05:54:34.237Z+ success 'ZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists'
4392024-02-15T05:54:34.238Z+ set +x
4402024-02-15T05:54:34.238ZZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists
4412024-02-15T05:54:34.238Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
4422024-02-15T05:54:34.238Z+ readarray -t ZPOOLS
4432024-02-15T05:54:34.238Z++ grep '"oxi_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
4442024-02-15T05:54:34.238Z++ sed 's/[ ",]//g'
4452024-02-15T05:54:34.238Z+ for ZPOOL in "${ZPOOLS[@]}"
4462024-02-15T05:54:34.239Z+ echo 'Zpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4472024-02-15T05:54:34.239ZZpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]
4482024-02-15T05:54:34.239Z+ VDEV_PATH=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4492024-02-15T05:54:34.239Z+ [[ -f /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4502024-02-15T05:54:34.239Z+ dd if=/dev/zero of=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4512024-02-15T05:54:34.239Z0+0 records in
4522024-02-15T05:54:34.239Z0+0 records out
4532024-02-15T05:54:34.239Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4542024-02-15T05:54:34.239Z+ success 'ZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4552024-02-15T05:54:34.240Z+ set +x
4562024-02-15T05:54:34.240ZZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4572024-02-15T05:54:34.240Z++ zpool list -o name
4582024-02-15T05:54:34.240Z++ grep oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b
4592024-02-15T05:54:34.240Z+ [[ -z '' ]]
4602024-02-15T05:54:34.240Z+ zpool create -f oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4612024-02-15T05:54:34.267Z+ success 'ZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4622024-02-15T05:54:34.267Z+ set +x
4632024-02-15T05:54:34.267ZZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4642024-02-15T05:54:34.268Z+ for ZPOOL in "${ZPOOLS[@]}"
4652024-02-15T05:54:34.268Z+ echo 'Zpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4662024-02-15T05:54:34.268ZZpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]
4672024-02-15T05:54:34.268Z+ VDEV_PATH=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4682024-02-15T05:54:34.268Z+ [[ -f /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4692024-02-15T05:54:34.268Z+ dd if=/dev/zero of=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4702024-02-15T05:54:34.268Z0+0 records in
4712024-02-15T05:54:34.268Z0+0 records out
4722024-02-15T05:54:34.268Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4732024-02-15T05:54:34.268Z+ success 'ZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4742024-02-15T05:54:34.268Z+ set +x
4752024-02-15T05:54:34.268ZZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4762024-02-15T05:54:34.294Z++ zpool list -o name
4772024-02-15T05:54:34.294Z++ grep oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b
4782024-02-15T05:54:34.294Z+ [[ -z '' ]]
4792024-02-15T05:54:34.294Z+ zpool create -f oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4802024-02-15T05:54:34.322Z+ success 'ZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4812024-02-15T05:54:34.323Z+ set +x
4822024-02-15T05:54:34.323ZZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4832024-02-15T05:54:34.323Z+ [[ zone == \z\o\n\e ]]
4842024-02-15T05:54:34.323Z+ ensure_simulated_links igb0
4852024-02-15T05:54:34.323Z+ local PHYSICAL_LINK=igb0
4862024-02-15T05:54:34.323Z+ INDICES=("0" "1")
4872024-02-15T05:54:34.323Z+ for I in "${INDICES[@]}"
4882024-02-15T05:54:34.324Z++ get_simnet_name_if_exists net0
4892024-02-15T05:54:34.324Z++ dladm show-simnet -p -o LINK net0
4902024-02-15T05:54:34.324Z++ echo ''
4912024-02-15T05:54:34.324Z+ [[ -z '' ]]
4922024-02-15T05:54:34.324Z+ dladm create-simnet -t net0
4932024-02-15T05:54:34.324Z+ dladm create-simnet -t sc0_0
4942024-02-15T05:54:34.364Z+ dladm modify-simnet -t -p net0 sc0_0
4952024-02-15T05:54:34.364Z+ dladm set-linkprop -p mtu=9000 sc0_0
4962024-02-15T05:54:34.364Z+ success 'Simnet net0/sc0_0 exists'
4972024-02-15T05:54:34.364Z+ set +x
4982024-02-15T05:54:34.364ZSimnet net0/sc0_0 exists
4992024-02-15T05:54:34.364Z+ for I in "${INDICES[@]}"
5002024-02-15T05:54:34.364Z++ get_simnet_name_if_exists net1
5012024-02-15T05:54:34.364Z++ dladm show-simnet -p -o LINK net1
5022024-02-15T05:54:34.364Z++ echo ''
5032024-02-15T05:54:34.364Z+ [[ -z '' ]]
5042024-02-15T05:54:34.364Z+ dladm create-simnet -t net1
5052024-02-15T05:54:34.364Z+ dladm create-simnet -t sc1_0
5062024-02-15T05:54:34.364Z+ dladm modify-simnet -t -p net1 sc1_0
5072024-02-15T05:54:34.364Z+ dladm set-linkprop -p mtu=9000 sc1_0
5082024-02-15T05:54:34.364Z+ success 'Simnet net1/sc1_0 exists'
5092024-02-15T05:54:34.364Z+ set +x
5102024-02-15T05:54:34.365ZSimnet net1/sc1_0 exists
5112024-02-15T05:54:34.365Z++ get_vnic_name_if_exists sc0_1
5122024-02-15T05:54:34.365Z++ dladm show-vnic -p -o LINK sc0_1
5132024-02-15T05:54:34.365Z++ echo ''
5142024-02-15T05:54:34.365Z+ [[ -z '' ]]
5152024-02-15T05:54:34.365Z+ dladm create-vnic -t sc0_1 -l igb0 -m a8:e1:de:01:70:1d
5162024-02-15T05:54:34.365Z+ [[ -v PROMISC_FILT_OFF ]]
5172024-02-15T05:54:34.366Z+ success 'Vnic sc0_1 exists'
5182024-02-15T05:54:34.366Z+ set +x
5192024-02-15T05:54:34.366ZVnic sc0_1 exists
5202024-02-15T05:54:34.366Z+ warn_if_no_proxy_arp
5212024-02-15T05:54:34.366Z+ [[ -v PXA_START ]]
5222024-02-15T05:54:34.366Z+ [[ -v PXA_END ]]
5232024-02-15T05:54:34.366Z+ ensure_softnpu_zone
5242024-02-15T05:54:34.396Z+ zoneadm list
5252024-02-15T05:54:34.396Z+ grep -q sidecar_softnpu
5262024-02-15T05:54:34.396Z+ [[ -f out/npuzone/npuzone ]]
5272024-02-15T05:54:34.396Z+ out/npuzone/npuzone create sidecar --omicron-zone --ports sc0_0,tfportrear0_0 --ports sc0_1,tfportqsfp0_0 --sidecar-lite-commit e3ea4b495ba0a71801ded0776ae4bbd31df57e26 --softnpu-commit dbab082dfa89da5db5ca2325c257089d2f130092
5282024-02-15T05:54:34.396Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/softnpu/image/dbab082dfa89da5db5ca2325c257089d2f130092/softnpu.sha256.txt
5292024-02-15T05:54:45.688Zdownloading softnpu ...done
5302024-02-15T05:54:45.715Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/libsidecar_lite.so.sha256.txt
5312024-02-15T05:54:51.225Zdownloading asic_program.so ...done
5322024-02-15T05:54:51.254Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/scadm.sha256.txt
5332024-02-15T05:55:00.549Zdownloading scadm ...done
5342024-02-15T05:55:00.587Zcreating port tfportrear0_0 for sc0_0
5352024-02-15T05:55:00.587Zcreating port tfportqsfp0_0 for sc0_1
5362024-02-15T05:55:00.587Zsoftnpu zone setup
5372024-02-15T05:55:00.587Zconfigure zone
5382024-02-15T05:55:00.614Zinstall zone
5392024-02-15T05:55:01.617Zboot zone
5402024-02-15T05:55:04.717Z[sidecar_softnpu] svcs milestone/network
5412024-02-15T05:55:04.757ZSTATE STIME FMRI
5422024-02-15T05:55:04.757Zonline 5:55:03 svc:/milestone/network:default
5432024-02-15T05:55:04.757Z[sidecar_softnpu] /softnpu/npu start
5442024-02-15T05:55:04.786Z
5452024-02-15T05:55:04.786Z+ /opt/oxide/work/tools/scrimlet/softnpu-init.sh
5462024-02-15T05:55:04.786Z+ GATEWAY_IP=10.151.1.100
5472024-02-15T05:55:04.786Z+ echo 'Using 10.151.1.100 as gateway ip'
5482024-02-15T05:55:04.786ZUsing 10.151.1.100 as gateway ip
5492024-02-15T05:55:04.786Z+ [[ ! -v GATEWAY_MAC ]]
5502024-02-15T05:55:04.786Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
5512024-02-15T05:55:04.786Z++ grep -c -E '^([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}$' /dev/fd/63
5522024-02-15T05:55:04.786Z+++ echo 18:c0:4d:0d:9f:b2
5532024-02-15T05:55:04.786Z+ COUNT=1
5542024-02-15T05:55:04.786Z+ [[ 1 -ne 1 ]]
5552024-02-15T05:55:04.786Z+ echo 'Using 18:c0:4d:0d:9f:b2 as gateway mac'
5562024-02-15T05:55:04.786ZUsing 18:c0:4d:0d:9f:b2 as gateway mac
5572024-02-15T05:55:04.786Z+ z_scadm add-arp-entry 10.151.1.100 18:c0:4d:0d:9f:b2
5582024-02-15T05:55:04.786Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-arp-entry 10.151.1.100 18:c0:4d:0d:9f:b2
5592024-02-15T05:55:04.851Z+ PXA_MAC=a8:e1:de:01:70:1d
5602024-02-15T05:55:04.851Z+ [[ -v PXA_START ]]
5612024-02-15T05:55:04.851Z+ z_scadm add-proxy-arp 10.151.1.164 10.151.1.227 a8:e1:de:01:70:1d
5622024-02-15T05:55:04.851Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-proxy-arp 10.151.1.164 10.151.1.227 a8:e1:de:01:70:1d
5632024-02-15T05:55:04.851Z+ z_scadm dump-state
5642024-02-15T05:55:04.851Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
5652024-02-15T05:55:04.876Zlocal v6:
5662024-02-15T05:55:04.877Zlocal v4:
5672024-02-15T05:55:04.877Zrouter v6:
5682024-02-15T05:55:04.877Zrouter v4_idx:
5692024-02-15T05:55:04.877Zrouter v4_routes:
5702024-02-15T05:55:04.877Zresolver v4:
5712024-02-15T05:55:04.877Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
5722024-02-15T05:55:04.877Zresolver v6:
5732024-02-15T05:55:04.877Znat v4:
5742024-02-15T05:55:04.877Znat_v6:
5752024-02-15T05:55:04.877Zport_mac:
5762024-02-15T05:55:04.877Zproxy arp:
5772024-02-15T05:55:04.877Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
5782024-02-15T05:55:04.877Z+ success 'softnpu zone exists'
5792024-02-15T05:55:04.877Z+ set +x
5802024-02-15T05:55:04.877Zsoftnpu zone exists
5812024-02-15T05:55:04.877Z
5822024-02-15T05:55:04.877Zreal 31.066060141
5832024-02-15T05:55:04.877Zuser 0.708639654
5842024-02-15T05:55:04.877Zsys 1.500296162
5852024-02-15T05:55:04.877Ztrap 0.000062115
5862024-02-15T05:55:04.877Ztflt 0.000128918
5872024-02-15T05:55:04.877Zdflt 0.000879974
5882024-02-15T05:55:04.877Zkflt 0.000000000
5892024-02-15T05:55:04.878Zlock 24:17.572543308
5902024-02-15T05:55:04.878Zslp 1:32.927607010
5912024-02-15T05:55:04.878Zlat 0.054044963
5922024-02-15T05:55:04.878Zstop 0.359453958
5932024-02-15T05:55:04.878Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml
5942024-02-15T05:55:05.320Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
5952024-02-15T05:55:05.321Z+ SILO_NAME=recovery
5962024-02-15T05:55:05.364Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
5972024-02-15T05:55:05.364Z+ EXTERNAL_DNS_DOMAIN=oxide.test
5982024-02-15T05:55:05.364Z+ sed -i~ '
5992024-02-15T05:55:05.365Z /^external_dns_ips/c\
6002024-02-15T05:55:05.365Zexternal_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
6012024-02-15T05:55:05.365Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
6022024-02-15T05:55:05.365Z /^first/c\
6032024-02-15T05:55:05.365Zfirst = "10.151.1.164"
6042024-02-15T05:55:05.365Z /^last/c\
6052024-02-15T05:55:05.366Zlast = "10.151.1.173"
6062024-02-15T05:55:05.366Z }
6072024-02-15T05:55:05.366Z /^\[rack_network_config/,/^$/ {
6082024-02-15T05:55:05.366Z /^infra_ip_first/c\
6092024-02-15T05:55:05.366Zinfra_ip_first = "10.151.1.174"
6102024-02-15T05:55:05.366Z /^infra_ip_last/c\
6112024-02-15T05:55:05.366Zinfra_ip_last = "10.151.1.174"
6122024-02-15T05:55:05.366Z }
6132024-02-15T05:55:05.366Z /^\[\[rack_network_config.ports/,/^$/ {
6142024-02-15T05:55:05.366Z /^routes/c\
6152024-02-15T05:55:05.366Zroutes = \[{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}\]
6162024-02-15T05:55:05.366Z /^addresses/c\
6172024-02-15T05:55:05.367Zaddresses = \["10.151.1.174/24"\]
6182024-02-15T05:55:05.367Z }
6192024-02-15T05:55:05.367Z' pkg/config-rss.toml
6202024-02-15T05:55:05.367Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
6212024-02-15T05:55:05.367Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
6222024-02-15T05:55:05.367Z+++ pkg/config-rss.toml Thu Feb 15 05:55:05 2024
6232024-02-15T05:55:05.367Z@@ -33,7 +33,7 @@
6242024-02-15T05:55:05.367Z # the DNS domain delegated to the rack by the customer. Each of these addresses
6252024-02-15T05:55:05.367Z # must be contained in one of the "internal services" IP Pool ranges listed
6262024-02-15T05:55:05.367Z # below.
6272024-02-15T05:55:05.368Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
6282024-02-15T05:55:05.368Z+external_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
6292024-02-15T05:55:05.368Z
6302024-02-15T05:55:05.368Z # Initial TLS certificates for the external API
6312024-02-15T05:55:05.368Z #
6322024-02-15T05:55:05.368Z@@ -69,8 +69,8 @@
6332024-02-15T05:55:05.368Z #
6342024-02-15T05:55:05.369Z # For more on this and what to put here, see docs/how-to-run.adoc.
6352024-02-15T05:55:05.369Z [[internal_services_ip_pool_ranges]]
6362024-02-15T05:55:05.369Z-first = "192.168.1.20"
6372024-02-15T05:55:05.369Z-last = "192.168.1.29"
6382024-02-15T05:55:05.369Z+first = "10.151.1.164"
6392024-02-15T05:55:05.369Z+last = "10.151.1.173"
6402024-02-15T05:55:05.369Z
6412024-02-15T05:55:05.369Z # TODO - this configuration is subject to change going forward. Ultimately these
6422024-02-15T05:55:05.369Z # parameters should be provided to the control plane via wicket, but we need to
6432024-02-15T05:55:05.369Z@@ -100,9 +100,9 @@
6442024-02-15T05:55:05.369Z # You can configure multiple uplinks by repeating the following stanza
6452024-02-15T05:55:05.369Z [[rack_network_config.ports]]
6462024-02-15T05:55:05.369Z # Routes associated with this port.
6472024-02-15T05:55:05.369Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
6482024-02-15T05:55:05.369Z+routes = [{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}]
6492024-02-15T05:55:05.370Z # Addresses associated with this port.
6502024-02-15T05:55:05.370Z-addresses = ["192.168.1.30/24"]
6512024-02-15T05:55:05.370Z+addresses = ["10.151.1.174/24"]
6522024-02-15T05:55:05.370Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
6532024-02-15T05:55:05.370Z port = "qsfp0"
6542024-02-15T05:55:05.370Z # The speed of this port.
6552024-02-15T05:55:05.370Z+ true
6562024-02-15T05:55:05.370Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml
6572024-02-15T05:55:05.370Za pkg/config-rss.toml 7K
6582024-02-15T05:55:05.370Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~
6592024-02-15T05:55:05.370Z+ TLS_NAME=recovery.sys.oxide.test
6602024-02-15T05:55:05.370Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
6612024-02-15T05:55:06.348Z..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.......+....+..+......+...+....+...............+......+............+........+.+...+..+....+............+..+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.....+...+...+.......+.....+...+....+...+...+..+............+.......+..............+...............+......+...+.+..........................................+...+.........+..........................................+........+......+......+..........+...+..+.+.....+.......+.....+...+......................+......+.....................+.....+.+.........+.....+................+.................+..................+...+.+...+...............+...+...+.....+......+...+...+.......+..+.+.............................+.+...+...........+...................+..+.......+..+................+..+...+.......+........+....+...+..+.........+.......+.........+..............+.......+...+.....+......+....+.....+.........+...+....+...+.....+..................+.......+..+..........+.....+......+....+......+...+..+...+..........+...............+......+........+............+...............+.+..+...+....+..........................+.........+...+.......+........+.+............+.....+...............+...............+.......+..+....+.....+.+.........+.........+..+...+......+.+.....+.....................+....+...+...........+.........+.+..+............+..........+.....+.......+......+..+......+.........+...+...+.....................+.........+....+......+......+...+.....+......+.......+............+.....+.+..............+...+.......+..+.+.....+.......+........+..........+...+..+.............+......+..............+....+..+...+.+...........+.+........+.+........+......+.......+...+............+...+.........+.....+......................+.....+.........+.......+.........+...+...+..+...+........................+...+.+..+.......+............+...........+...+.+.....+....+...........+...+.+..+....+.....+...+.......+.....................+.....+....+...+.....+.............+..+...+...+..........+.....+......................+..+......+..........+...........+......+............+...+...+....+...........+....+..+..........+.....+.+..+......+...............+....+...+..............+.............+........+...+....+...+.........+.........+...........................+.....+....+...+......+...............+......+.....+..........+.........+...............+...........+....+..+.........+.+.....+...+...............+............+....+.....+...+.+..+....+......+.....+....+.....+....+.....+.+...........+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6622024-02-15T05:55:06.712Z..+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+..+....+......+.....+.......+.....+.......+.........+.....+......+...+...+.......+......+.........+..+......+.........+...+.+..+......................+...+.........+......+..+.+.........+...+.....+....+.....+.+...+............+...+...........+..........+......+.....+..........+...+...........+....+.........+.....+......+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+..+..........+.....+...+....+...........+...+.+..............+................+...+..+...+...+.......+...+...+..+.......+...............+..+.........+.+...........+...+..........+....................+.........+...+.......+...+.................................+..+.............+......+.....+.........+.........+.+..............+......+.+.....+.........+..........+.............................+..........+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6632024-02-15T05:55:06.746Z-----
6642024-02-15T05:55:06.746Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6652024-02-15T05:55:06.746Za pkg/initial-tls-cert.pem 2K
6662024-02-15T05:55:06.746Za pkg/initial-tls-key.pem 4K
6672024-02-15T05:55:06.746Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6682024-02-15T05:55:06.746Z+ rmdir pkg
6692024-02-15T05:55:06.746Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
6702024-02-15T05:55:06.746Z+ pfexec mkdir -p /usr/oxide
6712024-02-15T05:55:06.746Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0
6722024-02-15T05:55:07.437Z+ pfexec chmod +x /usr/oxide/oxide
6732024-02-15T05:55:07.473Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
6742024-02-15T05:55:25.740Z+ OMICRON_NO_UNINSTALL=1
6752024-02-15T05:55:25.740Z+ ptime -m pfexec ./target/release/omicron-package -t test install
6762024-02-15T05:55:25.773ZLogging to: /opt/oxide/work/out/LOG
6772024-02-15T05:55:26.257Z
6782024-02-15T05:55:26.283Zreal 0.511699178
6792024-02-15T05:55:26.283Zuser 0.108262008
6802024-02-15T05:55:26.283Zsys 2.119058198
6812024-02-15T05:55:26.283Ztrap 0.000002120
6822024-02-15T05:55:26.283Ztflt 0.000135312
6832024-02-15T05:55:26.283Zdflt 0.000974573
6842024-02-15T05:55:26.283Zkflt 0.000000000
6852024-02-15T05:55:26.283Zlock 46.432840080
6862024-02-15T05:55:26.283Zslp 0.723581349
6872024-02-15T05:55:26.283Zlat 0.003176554
6882024-02-15T05:55:26.283Zstop 0.200362393
6892024-02-15T05:55:26.283Z+ retry=0
6902024-02-15T05:55:26.284Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
6912024-02-15T05:55:26.284Z+ [[ 0 -gt 30 ]]
6922024-02-15T05:55:26.284Z+ sleep 1
6932024-02-15T05:55:31.568Z+ retry=1
6942024-02-15T05:55:31.568Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
6952024-02-15T05:55:31.568Z+ [[ 1 -gt 30 ]]
6962024-02-15T05:55:31.568Z+ sleep 1
6972024-02-15T05:55:31.569Z+ retry=2
6982024-02-15T05:55:31.569Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
6992024-02-15T05:55:31.569Z+ [[ 2 -gt 30 ]]
7002024-02-15T05:55:31.569Z+ sleep 1
7012024-02-15T05:55:31.569Z+ retry=3
7022024-02-15T05:55:31.569Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7032024-02-15T05:55:31.569Z+ [[ 3 -gt 30 ]]
7042024-02-15T05:55:31.569Z+ sleep 1
7052024-02-15T05:55:31.569Z+ retry=4
7062024-02-15T05:55:31.569Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7072024-02-15T05:55:55.493Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
7082024-02-15T05:55:55.519Zlocal v6:
7092024-02-15T05:55:55.519Zfda0:cc56:8852:b33b::1
7102024-02-15T05:55:55.519Zfe80::dc5e:19ff:fe1a:dd56
7112024-02-15T05:55:55.519Zlocal v4:
7122024-02-15T05:55:55.519Z10.151.1.174
7132024-02-15T05:55:55.520Zrouter v6:
7142024-02-15T05:55:55.520Zfdb0:18c0:4d0d:9fb2::/64 -> fe80::c833:b6ff:fe26:652 (1)
7152024-02-15T05:55:55.545Zfd00:1122:3344:1::/64 -> fe80::c833:b6ff:fe26:652 (1)
7162024-02-15T05:55:55.545Zfd00:1122:3344:3::/64 -> fe80::c833:b6ff:fe26:652 (1)
7172024-02-15T05:55:55.545Zfd00:1122:3344:2::/64 -> fe80::c833:b6ff:fe26:652 (1)
7182024-02-15T05:55:55.545Zfd00:1122:3344:101::/64 -> fe80::c833:b6ff:fe26:652 (1)
7192024-02-15T05:55:55.545Zrouter v4_idx:
7202024-02-15T05:55:55.545Z0.0.0.0/0 -> 0
7212024-02-15T05:55:55.545Zrouter v4_routes:
7222024-02-15T05:55:55.546Z0 -> 10.151.1.100 (2)
7232024-02-15T05:55:55.546Zresolver v4:
7242024-02-15T05:55:55.546Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
7252024-02-15T05:55:55.546Z10.151.1.174 -> a6:15:6b:f1:55:74
7262024-02-15T05:55:55.546Zresolver v6:
7272024-02-15T05:55:55.546Zfe80::c833:b6ff:fe26:652 -> ca:33:b6:26:06:52
7282024-02-15T05:55:55.546Zfe80::dc5e:19ff:fe1a:dd56 -> de:5e:19:1a:dd:56
7292024-02-15T05:55:55.547Znat v4:
7302024-02-15T05:55:55.547Z10.151.1.169 0/16383 -> fd00:1122:3344:101::1 100/a8:40:25:ff:8a:9f
7312024-02-15T05:55:55.547Znat_v6:
7322024-02-15T05:55:55.547Zport_mac:
7332024-02-15T05:55:55.547Z1000: a8:40:25:70:02:e0
7342024-02-15T05:55:55.547Z2: a8:40:25:70:03:01
7352024-02-15T05:55:55.547Z1: a8:40:25:70:02:e1
7362024-02-15T05:55:55.547Zproxy arp:
7372024-02-15T05:55:55.548Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
7382024-02-15T05:55:55.548Z+ retry=0
7392024-02-15T05:55:55.548Z+++ zoneadm list -n
7402024-02-15T05:55:55.548Z+++ grep oxz_ntp
7412024-02-15T05:55:55.548Z++ pfexec svcs -z oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d -Hostate oxide/ntp
7422024-02-15T05:55:55.573Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7432024-02-15T05:55:55.574Z++ true
7442024-02-15T05:55:55.574Z+ [[ '' != online ]]
7452024-02-15T05:55:55.574Z+ [[ 0 -gt 60 ]]
7462024-02-15T05:55:55.574Z+ sleep 1
7472024-02-15T05:55:56.553Z+ retry=1
7482024-02-15T05:55:56.579Z+++ zoneadm list -n
7492024-02-15T05:55:56.579Z+++ grep oxz_ntp
7502024-02-15T05:55:56.579Z++ pfexec svcs -z oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d -Hostate oxide/ntp
7512024-02-15T05:55:56.579Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7522024-02-15T05:55:56.604Z++ true
7532024-02-15T05:55:56.604Z+ [[ '' != online ]]
7542024-02-15T05:55:56.605Z+ [[ 1 -gt 60 ]]
7552024-02-15T05:55:56.605Z+ sleep 1
7562024-02-15T05:55:57.582Z+ retry=2
7572024-02-15T05:55:57.607Z+++ zoneadm list -n
7582024-02-15T05:55:57.607Z+++ grep oxz_ntp
7592024-02-15T05:55:57.607Z++ pfexec svcs -z oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d -Hostate oxide/ntp
7602024-02-15T05:55:57.633Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7612024-02-15T05:55:57.633Z++ true
7622024-02-15T05:55:57.633Z+ [[ '' != online ]]
7632024-02-15T05:55:57.633Z+ [[ 2 -gt 60 ]]
7642024-02-15T05:55:57.633Z+ sleep 1
7652024-02-15T05:55:58.612Z+ retry=3
7662024-02-15T05:55:58.651Z+++ zoneadm list -n
7672024-02-15T05:55:58.651Z+++ grep oxz_ntp
7682024-02-15T05:55:58.651Z++ pfexec svcs -z oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d -Hostate oxide/ntp
7692024-02-15T05:55:58.651Z+ [[ disabled != online ]]
7702024-02-15T05:55:58.651Z+ [[ 3 -gt 60 ]]
7712024-02-15T05:55:58.651Z+ sleep 1
7722024-02-15T05:55:59.636Z+ retry=4
7732024-02-15T05:55:59.660Z+++ zoneadm list -n
7742024-02-15T05:55:59.660Z+++ grep oxz_ntp
7752024-02-15T05:55:59.661Z++ pfexec svcs -z oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d -Hostate oxide/ntp
7762024-02-15T05:55:59.661Z+ [[ disabled != online ]]
7772024-02-15T05:55:59.661Z+ [[ 4 -gt 60 ]]
7782024-02-15T05:55:59.661Z+ sleep 1
7792024-02-15T05:56:00.658Z+ retry=5
7802024-02-15T05:56:00.684Z+++ zoneadm list -n
7812024-02-15T05:56:00.684Z+++ grep oxz_ntp
7822024-02-15T05:56:00.684Z++ pfexec svcs -z oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d -Hostate oxide/ntp
7832024-02-15T05:56:00.684Z+ [[ disabled != online ]]
7842024-02-15T05:56:00.684Z+ [[ 5 -gt 60 ]]
7852024-02-15T05:56:00.684Z+ sleep 1
7862024-02-15T05:56:01.679Z+ retry=6
7872024-02-15T05:56:01.705Z+++ zoneadm list -n
7882024-02-15T05:56:01.705Z+++ grep oxz_ntp
7892024-02-15T05:56:01.705Z++ pfexec svcs -z oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d -Hostate oxide/ntp
7902024-02-15T05:56:01.705Z+ [[ disabled != online ]]
7912024-02-15T05:56:01.705Z+ [[ 6 -gt 60 ]]
7922024-02-15T05:56:01.706Z+ sleep 1
7932024-02-15T05:56:02.702Z+ retry=7
7942024-02-15T05:56:02.728Z+++ zoneadm list -n
7952024-02-15T05:56:02.729Z+++ grep oxz_ntp
7962024-02-15T05:56:02.729Z++ pfexec svcs -z oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d -Hostate oxide/ntp
7972024-02-15T05:56:02.729Z+ [[ disabled != online ]]
7982024-02-15T05:56:02.729Z+ [[ 7 -gt 60 ]]
7992024-02-15T05:56:02.730Z+ sleep 1
8002024-02-15T05:56:03.725Z+ retry=8
8012024-02-15T05:56:03.752Z+++ zoneadm list -n
8022024-02-15T05:56:03.752Z+++ grep oxz_ntp
8032024-02-15T05:56:03.752Z++ pfexec svcs -z oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d -Hostate oxide/ntp
8042024-02-15T05:56:03.752Z+ [[ disabled != online ]]
8052024-02-15T05:56:03.753Z+ [[ 8 -gt 60 ]]
8062024-02-15T05:56:03.753Z+ sleep 1
8072024-02-15T05:56:04.747Z+ retry=9
8082024-02-15T05:56:04.772Z+++ zoneadm list -n
8092024-02-15T05:56:04.773Z+++ grep oxz_ntp
8102024-02-15T05:56:04.773Z++ pfexec svcs -z oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d -Hostate oxide/ntp
8112024-02-15T05:56:04.773Z+ [[ disabled != online ]]
8122024-02-15T05:56:04.773Z+ [[ 9 -gt 60 ]]
8132024-02-15T05:56:04.773Z+ sleep 1
8142024-02-15T05:56:05.770Z+ retry=10
8152024-02-15T05:56:05.795Z+++ zoneadm list -n
8162024-02-15T05:56:05.795Z+++ grep oxz_ntp
8172024-02-15T05:56:05.795Z++ pfexec svcs -z oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d -Hostate oxide/ntp
8182024-02-15T05:56:05.795Z+ [[ disabled != online ]]
8192024-02-15T05:56:05.796Z+ [[ 10 -gt 60 ]]
8202024-02-15T05:56:05.796Z+ sleep 1
8212024-02-15T05:56:06.792Z+ retry=11
8222024-02-15T05:56:06.817Z+++ zoneadm list -n
8232024-02-15T05:56:06.817Z+++ grep oxz_ntp
8242024-02-15T05:56:06.818Z++ pfexec svcs -z oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d -Hostate oxide/ntp
8252024-02-15T05:56:06.818Z+ [[ online != online ]]
8262024-02-15T05:56:06.818Z+ echo 'Waited for chrony: 11s'
8272024-02-15T05:56:06.818ZWaited for chrony: 11s
8282024-02-15T05:56:06.818Z+ export RUST_BACKTRACE=1
8292024-02-15T05:56:06.818Z+ RUST_BACKTRACE=1
8302024-02-15T05:56:06.818Z+ export E2E_TLS_CERT IPPOOL_START IPPOOL_END
8312024-02-15T05:56:06.818Z++ ./tests/bootstrap
 
8322024-02-15T05:56:06.827Zbuskin console login: Feb 15 05:56:06 buskin ip: ip: joining multicasts failed (4) on vopte0 - will use link layer broadcasts for multicast
 
8332024-02-15T05:56:06.843ZUsing external DNS server at 10.151.1.164:53
8342024-02-15T05:56:06.868Z2024-02-15 05:56:06.968206083 UTC: attempting to log into API
8352024-02-15T05:56:21.882Z2024-02-15 05:56:21.456048692 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8362024-02-15T05:56:22.884Z2024-02-15 05:56:22.457822948 UTC: attempting to log into API
8372024-02-15T05:56:37.916Z2024-02-15 05:56:37.489483117 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8382024-02-15T05:56:38.917Z2024-02-15 05:56:38.490307012 UTC: attempting to log into API
8392024-02-15T05:56:53.947Z2024-02-15 05:56:54.166760441 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8402024-02-15T05:56:54.949Z2024-02-15 05:56:55.165116103 UTC: attempting to log into API
8412024-02-15T05:57:09.981Z2024-02-15 05:57:10.156099151 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8422024-02-15T05:57:10.986Z2024-02-15 05:57:11.154448293 UTC: attempting to log into API
8432024-02-15T05:57:26.014Z2024-02-15 05:57:26.145785955 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8442024-02-15T05:57:27.014Z2024-02-15 05:57:27.144116417 UTC: attempting to log into API
8452024-02-15T05:57:42.047Z2024-02-15 05:57:42.135701986 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8462024-02-15T05:57:43.048Z2024-02-15 05:57:43.134565366 UTC: attempting to log into API
8472024-02-15T05:57:58.080Z2024-02-15 05:57:58.125524483 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8482024-02-15T05:57:59.081Z2024-02-15 05:57:59.123806044 UTC: attempting to log into API
8492024-02-15T05:58:23.776Z2024-02-15 05:58:20.103689651 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8502024-02-15T05:58:23.776Z2024-02-15 05:58:21.126331533 UTC: attempting to log into API
8512024-02-15T05:58:37.670Z2024-02-15 05:58:37.658312297 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8522024-02-15T05:58:38.670Z2024-02-15 05:58:38.659285697 UTC: attempting to log into API
 
8532024-02-15T05:58:50.511ZFeb 15 05:58:49 buskin ip: ip: joining multicasts failed (4) on vopte1 - will use link layer broadcasts for multicast
 
8542024-02-15T05:58:53.703Z2024-02-15 05:58:53.690928685 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8552024-02-15T05:58:54.706Z2024-02-15 05:58:54.692014677 UTC: attempting to log into API
 
8562024-02-15T05:58:56.725ZFeb 15 05:58:56 buskin ip: ip: joining multicasts failed (4) on vopte2 - will use link layer broadcasts for multicast
8572024-02-15T05:59:02.930ZFeb 15 05:59:02 buskin ip: ip: joining multicasts failed (4) on vopte3 - will use link layer broadcasts for multicast
 
8582024-02-15T05:59:09.734Z2024-02-15 05:59:09.723929254 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
 
8592024-02-15T05:59:09.951ZFeb 15 05:59:09 buskin ip: ip: joining multicasts failed (4) on vopte4 - will use link layer broadcasts for multicast
 
8602024-02-15T05:59:10.736Z2024-02-15 05:59:10.725532924 UTC: attempting to log into API
 
8612024-02-15T05:59:17.224ZFeb 15 05:59:16 buskin ip: ip: joining multicasts failed (4) on vopte5 - will use link layer broadcasts for multicast
 
8622024-02-15T05:59:25.766Z2024-02-15 05:59:25.756852662 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8632024-02-15T05:59:26.767Z2024-02-15 05:59:26.757830851 UTC: attempting to log into API
8642024-02-15T05:59:37.575Z2024-02-15 05:59:37.565594710 UTC: login succeeded
8652024-02-15T05:59:37.607Zwaiting for nexus to come up...
8662024-02-15T05:59:37.681Zcreating IP pool... 10.151.1.175 - 10.151.1.227
8672024-02-15T05:59:37.869Zsetting up quotas...
8682024-02-15T05:59:37.915Zensuring datasets are ready...
8692024-02-15T05:59:39.749Zlet's roll.
8702024-02-15T05:59:39.777Z+ eval 'OXIDE_HOST=https://recovery.sys.oxide.test
8712024-02-15T05:59:39.778ZOXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
8722024-02-15T05:59:39.778ZOXIDE_TOKEN=oxide-token-45c200fbadc530f650c1efaa6ccf0bd3a2a6b6ff'
8732024-02-15T05:59:39.778Z++ OXIDE_HOST=https://recovery.sys.oxide.test
8742024-02-15T05:59:39.778Z++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
8752024-02-15T05:59:39.779Z++ OXIDE_TOKEN=oxide-token-45c200fbadc530f650c1efaa6ccf0bd3a2a6b6ff
8762024-02-15T05:59:39.779Z+ export OXIDE_HOST OXIDE_TOKEN
8772024-02-15T05:59:39.779Z+ retry=0
8782024-02-15T05:59:39.779Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8792024-02-15T05:59:39.869Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images'
8802024-02-15T05:59:40.181Z{"status":"ok"}success
8812024-02-15T05:59:40.181ZProject {
8822024-02-15T05:59:40.181Z description: "some images",
8832024-02-15T05:59:40.181Z id: c4d2fa14-0a32-4ac5-8bc8-c44866a197d5,
8842024-02-15T05:59:40.181Z name: Name(
8852024-02-15T05:59:40.181Z "images",
8862024-02-15T05:59:40.181Z ),
8872024-02-15T05:59:40.181Z time_created: 2024-02-15T05:59:39.930753Z,
8882024-02-15T05:59:40.181Z time_modified: 2024-02-15T05:59:39.930753Z,
8892024-02-15T05:59:40.181Z}
8902024-02-15T05:59:40.211Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11
8912024-02-15T06:05:10.244Zdone!
8922024-02-15T06:05:10.347Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11
8932024-02-15T06:05:10.445Zsuccess
8942024-02-15T06:05:10.445ZImage {
8952024-02-15T06:05:10.445Z block_size: ByteCount(
8962024-02-15T06:05:10.470Z 512,
8972024-02-15T06:05:10.471Z ),
8982024-02-15T06:05:10.471Z description: "debian 11 original base image",
8992024-02-15T06:05:10.471Z digest: None,
9002024-02-15T06:05:10.471Z id: bee1b50b-76f9-44ef-9ebd-20941e4e8796,
9012024-02-15T06:05:10.471Z name: Name(
9022024-02-15T06:05:10.471Z "debian11",
9032024-02-15T06:05:10.471Z ),
9042024-02-15T06:05:10.471Z os: "debian",
9052024-02-15T06:05:10.471Z project_id: None,
9062024-02-15T06:05:10.471Z size: ByteCount(
9072024-02-15T06:05:10.471Z 2147483648,
9082024-02-15T06:05:10.471Z ),
9092024-02-15T06:05:10.471Z time_created: 2024-02-15T06:05:10.212571Z,
9102024-02-15T06:05:10.471Z time_modified: 2024-02-15T06:05:10.429423Z,
9112024-02-15T06:05:10.471Z url: None,
9122024-02-15T06:05:10.471Z version: "11",
9132024-02-15T06:05:10.471Z}
9142024-02-15T06:05:10.471Z+ rm ./tests/bootstrap
9152024-02-15T06:05:10.535Z+ for test_bin in tests/*
9162024-02-15T06:05:10.535Z+ ./tests/bootstrap-4ef928d4a311d736
9172024-02-15T06:05:10.618Z
9182024-02-15T06:05:10.618Zrunning 0 tests
9192024-02-15T06:05:10.618Z
9202024-02-15T06:05:10.642Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
9212024-02-15T06:05:10.642Z
9222024-02-15T06:05:10.642Z+ for test_bin in tests/*
9232024-02-15T06:05:10.642Z+ ./tests/end_to_end_tests-d04a49cb4bbc61f6
9242024-02-15T06:05:11.469Z
9252024-02-15T06:05:11.469Zrunning 2 tests
9262024-02-15T06:05:11.758Ztest no_spoof::no_spoof_here ... ok
9272024-02-15T06:05:48.046Ztest instance_launch::instance_launch ... ok
9282024-02-15T06:05:48.069Z
9292024-02-15T06:05:48.069Ztest result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 36.58s
9302024-02-15T06:05:48.069Z
9312024-02-15T06:05:48.070Z+ _exit_trap
9322024-02-15T06:05:48.070Z+ local status=0
9332024-02-15T06:05:48.070Z+ [[ 0 -eq 0 ]]
9342024-02-15T06:05:48.070Z+ exit 0
9352024-02-15T06:05:48.094Zprocess exited: duration 713139 ms, exit code 0
 
9362024-02-15T06:05:48.254Zfound 131 output files
9372024-02-15T06:05:48.255Zuploading: /var/svc/log/oxide-sled-agent:default.log (1925327 bytes)
9382024-02-15T06:05:50.013Zuploaded: /var/svc/log/oxide-sled-agent:default.log
9392024-02-15T06:05:50.014Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log (1468324 bytes)
9402024-02-15T06:05:50.500Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log" changed size mid upload: 1468324 -> 1481391
9412024-02-15T06:05:51.451Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log
9422024-02-15T06:05:51.451Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log.0 (1121286 bytes)
9432024-02-15T06:05:53.695Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log.0
9442024-02-15T06:05:53.695Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log (200 bytes)
9452024-02-15T06:05:54.788Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log
9462024-02-15T06:05:54.788Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log.0 (11838 bytes)
9472024-02-15T06:05:55.927Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log.0
9482024-02-15T06:05:55.928Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log (12539 bytes)
9492024-02-15T06:05:57.097Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log
9502024-02-15T06:05:57.097Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log.0 (11897 bytes)
9512024-02-15T06:05:58.176Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log.0
9522024-02-15T06:05:58.176Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log (874111 bytes)
9532024-02-15T06:05:58.455Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log" changed size mid upload: 874111 -> 895674
9542024-02-15T06:05:59.526Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log
9552024-02-15T06:05:59.526Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log.0 (707611 bytes)
9562024-02-15T06:06:00.847Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log.0
9572024-02-15T06:06:00.847Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log (521366 bytes)
9582024-02-15T06:06:01.102Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log" changed size mid upload: 521366 -> 540048
9592024-02-15T06:06:02.172Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log
9602024-02-15T06:06:02.196Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log.0 (398367 bytes)
9612024-02-15T06:06:03.451Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log.0
9622024-02-15T06:06:03.451Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log (0 bytes)
9632024-02-15T06:06:04.521Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log
9642024-02-15T06:06:04.521Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log.0 (1388 bytes)
9652024-02-15T06:06:05.614Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log.0
9662024-02-15T06:06:05.614Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log (170575 bytes)
9672024-02-15T06:06:05.796Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log" changed size mid upload: 170575 -> 179155
9682024-02-15T06:06:06.878Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log
9692024-02-15T06:06:06.879Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log.0 (119304 bytes)
9702024-02-15T06:06:08.089Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log.0
9712024-02-15T06:06:08.090Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log (0 bytes)
9722024-02-15T06:06:09.168Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log
9732024-02-15T06:06:09.168Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log.0 (617 bytes)
9742024-02-15T06:06:10.260Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log.0
9752024-02-15T06:06:10.260Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log (0 bytes)
9762024-02-15T06:06:11.332Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log
9772024-02-15T06:06:11.332Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log.0 (5392 bytes)
9782024-02-15T06:06:12.429Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log.0
9792024-02-15T06:06:12.429Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_9882b086-6ec2-441a-b8c4-df78d6d0a302/root/var/svc/log/oxide-clickhouse:default.log (558179 bytes)
9802024-02-15T06:06:12.703Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_9882b086-6ec2-441a-b8c4-df78d6d0a302/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 558179 -> 593741
9812024-02-15T06:06:13.773Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_9882b086-6ec2-441a-b8c4-df78d6d0a302/root/var/svc/log/oxide-clickhouse:default.log
9822024-02-15T06:06:14.207Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_9882b086-6ec2-441a-b8c4-df78d6d0a302/root/var/svc/log/oxide-clickhouse:default.log.0 (160774 bytes)
9832024-02-15T06:06:15.000Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_9882b086-6ec2-441a-b8c4-df78d6d0a302/root/var/svc/log/oxide-clickhouse:default.log.0
9842024-02-15T06:06:15.001Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_9882b086-6ec2-441a-b8c4-df78d6d0a302/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
9852024-02-15T06:06:16.074Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_9882b086-6ec2-441a-b8c4-df78d6d0a302/root/var/svc/log/oxide-zone-network-setup:default.log
9862024-02-15T06:06:16.098Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_9882b086-6ec2-441a-b8c4-df78d6d0a302/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1706 bytes)
9872024-02-15T06:06:17.166Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_9882b086-6ec2-441a-b8c4-df78d6d0a302/root/var/svc/log/oxide-zone-network-setup:default.log.0
9882024-02-15T06:06:17.166Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_38d5c382-d544-48a5-9e2e-b01d5cfb5bc9/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
9892024-02-15T06:06:18.259Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_38d5c382-d544-48a5-9e2e-b01d5cfb5bc9/root/var/svc/log/oxide-cockroachdb:default.log
9902024-02-15T06:06:18.259Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_38d5c382-d544-48a5-9e2e-b01d5cfb5bc9/root/var/svc/log/oxide-cockroachdb:default.log.0 (4407 bytes)
9912024-02-15T06:06:19.365Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_38d5c382-d544-48a5-9e2e-b01d5cfb5bc9/root/var/svc/log/oxide-cockroachdb:default.log.0
9922024-02-15T06:06:19.365Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_38d5c382-d544-48a5-9e2e-b01d5cfb5bc9/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
9932024-02-15T06:06:20.447Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_38d5c382-d544-48a5-9e2e-b01d5cfb5bc9/root/var/svc/log/oxide-zone-network-setup:default.log
9942024-02-15T06:06:20.447Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_38d5c382-d544-48a5-9e2e-b01d5cfb5bc9/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1706 bytes)
9952024-02-15T06:06:21.543Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_38d5c382-d544-48a5-9e2e-b01d5cfb5bc9/root/var/svc/log/oxide-zone-network-setup:default.log.0
9962024-02-15T06:06:21.543Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_4b8884ec-5901-4d3f-83a3-b1b1f952dc17/root/var/svc/log/oxide-crucible-agent:default.log (6355 bytes)
9972024-02-15T06:06:22.657Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_4b8884ec-5901-4d3f-83a3-b1b1f952dc17/root/var/svc/log/oxide-crucible-agent:default.log
9982024-02-15T06:06:22.657Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_4b8884ec-5901-4d3f-83a3-b1b1f952dc17/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
9992024-02-15T06:06:23.750Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_4b8884ec-5901-4d3f-83a3-b1b1f952dc17/root/var/svc/log/oxide-crucible-agent:default.log.0
10002024-02-15T06:06:23.750Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_4b8884ec-5901-4d3f-83a3-b1b1f952dc17/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
10012024-02-15T06:06:24.821Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_4b8884ec-5901-4d3f-83a3-b1b1f952dc17/root/var/svc/log/oxide-crucible-downstairs:default.log
10022024-02-15T06:06:24.847Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_4b8884ec-5901-4d3f-83a3-b1b1f952dc17/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
10032024-02-15T06:06:25.913Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_4b8884ec-5901-4d3f-83a3-b1b1f952dc17/root/var/svc/log/oxide-crucible-downstairs:default.log.0
10042024-02-15T06:06:25.914Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_4b8884ec-5901-4d3f-83a3-b1b1f952dc17/root/var/svc/log/oxide-crucible-downstairs:downstairs-bd71750f-5662-4eda-bc42-ee9d2282f3f4.log (3020 bytes)
10052024-02-15T06:06:27.014Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_4b8884ec-5901-4d3f-83a3-b1b1f952dc17/root/var/svc/log/oxide-crucible-downstairs:downstairs-bd71750f-5662-4eda-bc42-ee9d2282f3f4.log
10062024-02-15T06:06:27.014Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_a702a2df-8aec-4a4e-9585-52df0fbccf45/root/var/svc/log/oxide-external_dns:default.log (22659 bytes)
10072024-02-15T06:06:27.106Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_a702a2df-8aec-4a4e-9585-52df0fbccf45/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 22659 -> 26434
10082024-02-15T06:06:28.175Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_a702a2df-8aec-4a4e-9585-52df0fbccf45/root/var/svc/log/oxide-external_dns:default.log
10092024-02-15T06:06:28.176Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_a702a2df-8aec-4a4e-9585-52df0fbccf45/root/var/svc/log/oxide-external_dns:default.log.0 (4281 bytes)
10102024-02-15T06:06:29.271Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_a702a2df-8aec-4a4e-9585-52df0fbccf45/root/var/svc/log/oxide-external_dns:default.log.0
10112024-02-15T06:06:29.271Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_6322d35a-04fa-4d40-ba3c-fb0208e2f2f4/root/var/svc/log/oxide-internal_dns:default.log (22531 bytes)
10122024-02-15T06:06:29.361Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_6322d35a-04fa-4d40-ba3c-fb0208e2f2f4/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 22531 -> 26285
10132024-02-15T06:06:30.429Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_6322d35a-04fa-4d40-ba3c-fb0208e2f2f4/root/var/svc/log/oxide-internal_dns:default.log
10142024-02-15T06:06:30.429Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_6322d35a-04fa-4d40-ba3c-fb0208e2f2f4/root/var/svc/log/oxide-internal_dns:default.log.0 (93203 bytes)
10152024-02-15T06:06:31.649Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_6322d35a-04fa-4d40-ba3c-fb0208e2f2f4/root/var/svc/log/oxide-internal_dns:default.log.0
10162024-02-15T06:06:31.649Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_07eefccc-a16d-41be-9df2-0faa4294ae1c/root/var/svc/log/oxide-oximeter:default.log (198965 bytes)
10172024-02-15T06:06:31.831Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_07eefccc-a16d-41be-9df2-0faa4294ae1c/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 198965 -> 221130
10182024-02-15T06:06:32.925Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_07eefccc-a16d-41be-9df2-0faa4294ae1c/root/var/svc/log/oxide-oximeter:default.log
10192024-02-15T06:06:32.925Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_07eefccc-a16d-41be-9df2-0faa4294ae1c/root/var/svc/log/oxide-oximeter:default.log.0 (32102 bytes)
10202024-02-15T06:06:34.082Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_07eefccc-a16d-41be-9df2-0faa4294ae1c/root/var/svc/log/oxide-oximeter:default.log.0
10212024-02-15T06:06:34.082Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_07eefccc-a16d-41be-9df2-0faa4294ae1c/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
10222024-02-15T06:06:35.160Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_07eefccc-a16d-41be-9df2-0faa4294ae1c/root/var/svc/log/oxide-zone-network-setup:default.log
10232024-02-15T06:06:35.161Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_07eefccc-a16d-41be-9df2-0faa4294ae1c/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1696 bytes)
10242024-02-15T06:06:36.252Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_07eefccc-a16d-41be-9df2-0faa4294ae1c/root/var/svc/log/oxide-zone-network-setup:default.log.0
10252024-02-15T06:06:36.275Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_bf26395e-9671-4bee-b96d-1c6f76e85a02/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
10262024-02-15T06:06:37.321Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_bf26395e-9671-4bee-b96d-1c6f76e85a02/root/var/svc/log/oxide-cockroachdb:default.log
10272024-02-15T06:06:37.321Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_bf26395e-9671-4bee-b96d-1c6f76e85a02/root/var/svc/log/oxide-cockroachdb:default.log.0 (4406 bytes)
10282024-02-15T06:06:38.414Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_bf26395e-9671-4bee-b96d-1c6f76e85a02/root/var/svc/log/oxide-cockroachdb:default.log.0
10292024-02-15T06:06:38.439Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_bf26395e-9671-4bee-b96d-1c6f76e85a02/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
10302024-02-15T06:06:39.483Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_bf26395e-9671-4bee-b96d-1c6f76e85a02/root/var/svc/log/oxide-zone-network-setup:default.log
10312024-02-15T06:06:39.483Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_bf26395e-9671-4bee-b96d-1c6f76e85a02/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1706 bytes)
10322024-02-15T06:06:40.576Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_bf26395e-9671-4bee-b96d-1c6f76e85a02/root/var/svc/log/oxide-zone-network-setup:default.log.0
10332024-02-15T06:06:40.576Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-agent:default.log (18468 bytes)
10342024-02-15T06:06:41.711Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-agent:default.log
10352024-02-15T06:06:41.711Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-agent:default.log.0 (9364 bytes)
10362024-02-15T06:06:42.832Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-agent:default.log.0
10372024-02-15T06:06:42.832Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
10382024-02-15T06:06:43.900Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-downstairs:default.log
10392024-02-15T06:06:43.900Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
10402024-02-15T06:06:44.991Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-downstairs:default.log.0
10412024-02-15T06:06:44.991Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-downstairs:downstairs-44e563ab-8ff8-48e9-9309-6e43dce2eb22.log (5611 bytes)
10422024-02-15T06:06:46.097Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-downstairs:downstairs-44e563ab-8ff8-48e9-9309-6e43dce2eb22.log
10432024-02-15T06:06:46.121Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-downstairs:downstairs-f856737b-60e4-42aa-8e11-5c9d4597edc5.log (1092 bytes)
10442024-02-15T06:06:47.194Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-downstairs:downstairs-f856737b-60e4-42aa-8e11-5c9d4597edc5.log
10452024-02-15T06:06:47.194Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-downstairs:downstairs-f856737b-60e4-42aa-8e11-5c9d4597edc5.log.0 (4604 bytes)
10462024-02-15T06:06:48.309Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-downstairs:downstairs-f856737b-60e4-42aa-8e11-5c9d4597edc5.log.0
10472024-02-15T06:06:48.309Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-downstairs:snapshot-f856737b-60e4-42aa-8e11-5c9d4597edc5-26054eb5-467c-4ad3-988b-6fd144d2b756.log (5302 bytes)
10482024-02-15T06:06:49.821Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9f6066d0-74ff-4236-abb7-8522ff3656f2/root/var/svc/log/oxide-crucible-downstairs:snapshot-f856737b-60e4-42aa-8e11-5c9d4597edc5-26054eb5-467c-4ad3-988b-6fd144d2b756.log
10492024-02-15T06:06:49.847Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_a0f87398-93b9-40a8-90b6-7059441c2401/root/var/svc/log/oxide-crucible-pantry:default.log (0 bytes)
10502024-02-15T06:06:50.891Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_a0f87398-93b9-40a8-90b6-7059441c2401/root/var/svc/log/oxide-crucible-pantry:default.log
10512024-02-15T06:06:50.891Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_a0f87398-93b9-40a8-90b6-7059441c2401/root/var/svc/log/oxide-crucible-pantry:default.log.0 (2333 bytes)
10522024-02-15T06:06:52.064Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_a0f87398-93b9-40a8-90b6-7059441c2401/root/var/svc/log/oxide-crucible-pantry:default.log.0
10532024-02-15T06:06:52.064Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_7e2aceaf-08e1-46f2-95ec-f9032200ba39/root/var/svc/log/oxide-external_dns:default.log (22649 bytes)
10542024-02-15T06:06:52.090Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_7e2aceaf-08e1-46f2-95ec-f9032200ba39/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 22649 -> 26425
10552024-02-15T06:06:53.149Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_7e2aceaf-08e1-46f2-95ec-f9032200ba39/root/var/svc/log/oxide-external_dns:default.log
10562024-02-15T06:06:53.149Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_7e2aceaf-08e1-46f2-95ec-f9032200ba39/root/var/svc/log/oxide-external_dns:default.log.0 (4283 bytes)
10572024-02-15T06:06:54.241Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_7e2aceaf-08e1-46f2-95ec-f9032200ba39/root/var/svc/log/oxide-external_dns:default.log.0
10582024-02-15T06:06:54.264Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_719635e7-d366-4895-9272-9ade9dc77ba8/root/var/svc/log/oxide-internal_dns:default.log (22529 bytes)
10592024-02-15T06:06:54.334Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_719635e7-d366-4895-9272-9ade9dc77ba8/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 22529 -> 26284
10602024-02-15T06:06:55.407Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_719635e7-d366-4895-9272-9ade9dc77ba8/root/var/svc/log/oxide-internal_dns:default.log
10612024-02-15T06:06:55.408Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_719635e7-d366-4895-9272-9ade9dc77ba8/root/var/svc/log/oxide-internal_dns:default.log.0 (95233 bytes)
10622024-02-15T06:06:56.607Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_719635e7-d366-4895-9272-9ade9dc77ba8/root/var/svc/log/oxide-internal_dns:default.log.0
10632024-02-15T06:06:56.656Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_426e6991-0acc-4834-a7bc-68fb4dfbb775/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
10642024-02-15T06:06:57.679Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_426e6991-0acc-4834-a7bc-68fb4dfbb775/root/var/svc/log/oxide-cockroachdb:default.log
10652024-02-15T06:06:57.680Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_426e6991-0acc-4834-a7bc-68fb4dfbb775/root/var/svc/log/oxide-cockroachdb:default.log.0 (4407 bytes)
10662024-02-15T06:06:58.780Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_426e6991-0acc-4834-a7bc-68fb4dfbb775/root/var/svc/log/oxide-cockroachdb:default.log.0
10672024-02-15T06:06:58.780Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_426e6991-0acc-4834-a7bc-68fb4dfbb775/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
10682024-02-15T06:06:59.849Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_426e6991-0acc-4834-a7bc-68fb4dfbb775/root/var/svc/log/oxide-zone-network-setup:default.log
10692024-02-15T06:06:59.849Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_426e6991-0acc-4834-a7bc-68fb4dfbb775/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1706 bytes)
10702024-02-15T06:07:00.949Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_426e6991-0acc-4834-a7bc-68fb4dfbb775/root/var/svc/log/oxide-zone-network-setup:default.log.0
10712024-02-15T06:07:00.974Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-agent:default.log (18474 bytes)
10722024-02-15T06:07:02.093Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-agent:default.log
10732024-02-15T06:07:02.121Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-agent:default.log.0 (9364 bytes)
10742024-02-15T06:07:03.222Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-agent:default.log.0
10752024-02-15T06:07:03.222Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
10762024-02-15T06:07:04.291Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-downstairs:default.log
10772024-02-15T06:07:04.291Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
10782024-02-15T06:07:05.401Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-downstairs:default.log.0
10792024-02-15T06:07:05.401Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-downstairs:downstairs-ce1e4a26-d199-4e3b-8aa1-836674544624.log (5612 bytes)
10802024-02-15T06:07:06.508Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-downstairs:downstairs-ce1e4a26-d199-4e3b-8aa1-836674544624.log
10812024-02-15T06:07:06.508Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-downstairs:downstairs-d81eaef2-ffde-4bdf-bacb-f25f75065639.log (1092 bytes)
10822024-02-15T06:07:07.601Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-downstairs:downstairs-d81eaef2-ffde-4bdf-bacb-f25f75065639.log
10832024-02-15T06:07:07.601Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-downstairs:downstairs-d81eaef2-ffde-4bdf-bacb-f25f75065639.log.0 (4604 bytes)
10842024-02-15T06:07:08.701Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-downstairs:downstairs-d81eaef2-ffde-4bdf-bacb-f25f75065639.log.0
10852024-02-15T06:07:08.701Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-downstairs:snapshot-d81eaef2-ffde-4bdf-bacb-f25f75065639-26054eb5-467c-4ad3-988b-6fd144d2b756.log (5305 bytes)
10862024-02-15T06:07:09.793Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_7df09570-afa6-4fc7-ae24-8a2ea0b32af7/root/var/svc/log/oxide-crucible-downstairs:snapshot-d81eaef2-ffde-4bdf-bacb-f25f75065639-26054eb5-467c-4ad3-988b-6fd144d2b756.log
10872024-02-15T06:07:09.793Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_2ae10e87-0030-442b-9981-0b4c70a9c369/root/var/svc/log/oxide-internal_dns:default.log (24209 bytes)
10882024-02-15T06:07:09.925Zupload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_2ae10e87-0030-442b-9981-0b4c70a9c369/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 24209 -> 27964
10892024-02-15T06:07:11.010Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_2ae10e87-0030-442b-9981-0b4c70a9c369/root/var/svc/log/oxide-internal_dns:default.log
10902024-02-15T06:07:11.010Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_2ae10e87-0030-442b-9981-0b4c70a9c369/root/var/svc/log/oxide-internal_dns:default.log.0 (89989 bytes)
10912024-02-15T06:07:12.348Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_2ae10e87-0030-442b-9981-0b4c70a9c369/root/var/svc/log/oxide-internal_dns:default.log.0
10922024-02-15T06:07:12.348Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_4243c086-1bed-4413-b0c7-7c802e7bc575/root/var/svc/log/oxide-nexus:default.log (854217 bytes)
10932024-02-15T06:07:13.481Zupload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_4243c086-1bed-4413-b0c7-7c802e7bc575/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 854217 -> 980095
10942024-02-15T06:07:14.552Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_4243c086-1bed-4413-b0c7-7c802e7bc575/root/var/svc/log/oxide-nexus:default.log
10952024-02-15T06:07:14.552Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_4243c086-1bed-4413-b0c7-7c802e7bc575/root/var/svc/log/oxide-nexus:default.log.0 (480155 bytes)
10962024-02-15T06:07:16.370Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_4243c086-1bed-4413-b0c7-7c802e7bc575/root/var/svc/log/oxide-nexus:default.log.0
10972024-02-15T06:07:16.370Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_21544c6f-59af-4a59-a720-d40b64e76f5d/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
10982024-02-15T06:07:17.439Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_21544c6f-59af-4a59-a720-d40b64e76f5d/root/var/svc/log/oxide-cockroachdb:default.log
10992024-02-15T06:07:17.440Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_21544c6f-59af-4a59-a720-d40b64e76f5d/root/var/svc/log/oxide-cockroachdb:default.log.0 (4379 bytes)
11002024-02-15T06:07:18.549Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_21544c6f-59af-4a59-a720-d40b64e76f5d/root/var/svc/log/oxide-cockroachdb:default.log.0
11012024-02-15T06:07:18.549Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_21544c6f-59af-4a59-a720-d40b64e76f5d/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
11022024-02-15T06:07:19.618Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_21544c6f-59af-4a59-a720-d40b64e76f5d/root/var/svc/log/oxide-zone-network-setup:default.log
11032024-02-15T06:07:19.642Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_21544c6f-59af-4a59-a720-d40b64e76f5d/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1706 bytes)
11042024-02-15T06:07:20.710Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_21544c6f-59af-4a59-a720-d40b64e76f5d/root/var/svc/log/oxide-zone-network-setup:default.log.0
11052024-02-15T06:07:20.710Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_f7332e25-0a75-4959-b498-faa862ee2c5f/root/var/svc/log/oxide-crucible-agent:default.log (16641 bytes)
11062024-02-15T06:07:21.847Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_f7332e25-0a75-4959-b498-faa862ee2c5f/root/var/svc/log/oxide-crucible-agent:default.log
11072024-02-15T06:07:21.847Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_f7332e25-0a75-4959-b498-faa862ee2c5f/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
11082024-02-15T06:07:22.942Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_f7332e25-0a75-4959-b498-faa862ee2c5f/root/var/svc/log/oxide-crucible-agent:default.log.0
11092024-02-15T06:07:22.942Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_f7332e25-0a75-4959-b498-faa862ee2c5f/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11102024-02-15T06:07:24.017Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_f7332e25-0a75-4959-b498-faa862ee2c5f/root/var/svc/log/oxide-crucible-downstairs:default.log
11112024-02-15T06:07:24.041Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_f7332e25-0a75-4959-b498-faa862ee2c5f/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11122024-02-15T06:07:25.110Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_f7332e25-0a75-4959-b498-faa862ee2c5f/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11132024-02-15T06:07:25.134Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_f7332e25-0a75-4959-b498-faa862ee2c5f/root/var/svc/log/oxide-crucible-downstairs:downstairs-3846ca66-4b50-4511-a8dc-9a7f28ca3817.log (5612 bytes)
11142024-02-15T06:07:26.225Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_f7332e25-0a75-4959-b498-faa862ee2c5f/root/var/svc/log/oxide-crucible-downstairs:downstairs-3846ca66-4b50-4511-a8dc-9a7f28ca3817.log
11152024-02-15T06:07:26.249Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_f7332e25-0a75-4959-b498-faa862ee2c5f/root/var/svc/log/oxide-crucible-downstairs:downstairs-c9305b10-12d9-41f3-aa22-4fa832b935b1.log (3019 bytes)
11162024-02-15T06:07:27.331Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_f7332e25-0a75-4959-b498-faa862ee2c5f/root/var/svc/log/oxide-crucible-downstairs:downstairs-c9305b10-12d9-41f3-aa22-4fa832b935b1.log
11172024-02-15T06:07:27.331Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_ddb8e0b6-c193-419e-b893-def90acf2c6e/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
11182024-02-15T06:07:28.402Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_ddb8e0b6-c193-419e-b893-def90acf2c6e/root/var/svc/log/oxide-cockroachdb:default.log
11192024-02-15T06:07:28.402Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_ddb8e0b6-c193-419e-b893-def90acf2c6e/root/var/svc/log/oxide-cockroachdb:default.log.0 (4407 bytes)
11202024-02-15T06:07:29.498Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_ddb8e0b6-c193-419e-b893-def90acf2c6e/root/var/svc/log/oxide-cockroachdb:default.log.0
11212024-02-15T06:07:29.498Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_ddb8e0b6-c193-419e-b893-def90acf2c6e/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
11222024-02-15T06:07:30.567Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_ddb8e0b6-c193-419e-b893-def90acf2c6e/root/var/svc/log/oxide-zone-network-setup:default.log
11232024-02-15T06:07:30.568Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_ddb8e0b6-c193-419e-b893-def90acf2c6e/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1705 bytes)
11242024-02-15T06:07:31.667Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_ddb8e0b6-c193-419e-b893-def90acf2c6e/root/var/svc/log/oxide-zone-network-setup:default.log.0
11252024-02-15T06:07:31.667Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-agent:default.log (7760 bytes)
11262024-02-15T06:07:32.783Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-agent:default.log
11272024-02-15T06:07:32.783Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-agent:default.log.0 (9361 bytes)
11282024-02-15T06:07:33.899Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-agent:default.log.0
11292024-02-15T06:07:33.899Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11302024-02-15T06:07:34.969Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-downstairs:default.log
11312024-02-15T06:07:34.993Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11322024-02-15T06:07:36.062Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11332024-02-15T06:07:36.062Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-downstairs:downstairs-ac239c33-40fc-48e8-a491-351f0a59a31b.log (1092 bytes)
11342024-02-15T06:07:37.166Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-downstairs:downstairs-ac239c33-40fc-48e8-a491-351f0a59a31b.log
11352024-02-15T06:07:37.190Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-downstairs:downstairs-ac239c33-40fc-48e8-a491-351f0a59a31b.log.0 (4601 bytes)
11362024-02-15T06:07:38.262Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-downstairs:downstairs-ac239c33-40fc-48e8-a491-351f0a59a31b.log.0
11372024-02-15T06:07:38.262Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-downstairs:snapshot-ac239c33-40fc-48e8-a491-351f0a59a31b-26054eb5-467c-4ad3-988b-6fd144d2b756.log (5301 bytes)
11382024-02-15T06:07:39.367Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_d6729570-1e52-4d84-af84-f4f23623dfb6/root/var/svc/log/oxide-crucible-downstairs:snapshot-ac239c33-40fc-48e8-a491-351f0a59a31b-26054eb5-467c-4ad3-988b-6fd144d2b756.log
11392024-02-15T06:07:39.368Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_2cb47b44-e1c8-4597-82f0-7ee9d5d86893/root/var/svc/log/oxide-crucible-pantry:default.log (0 bytes)
11402024-02-15T06:07:40.438Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_2cb47b44-e1c8-4597-82f0-7ee9d5d86893/root/var/svc/log/oxide-crucible-pantry:default.log
11412024-02-15T06:07:40.438Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_2cb47b44-e1c8-4597-82f0-7ee9d5d86893/root/var/svc/log/oxide-crucible-pantry:default.log.0 (2338 bytes)
11422024-02-15T06:07:41.536Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_2cb47b44-e1c8-4597-82f0-7ee9d5d86893/root/var/svc/log/oxide-crucible-pantry:default.log.0
11432024-02-15T06:07:41.537Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-agent:default.log (5929 bytes)
11442024-02-15T06:07:42.638Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-agent:default.log
11452024-02-15T06:07:42.638Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-agent:default.log.0 (13685 bytes)
11462024-02-15T06:07:44.186Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-agent:default.log.0
11472024-02-15T06:07:44.187Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11482024-02-15T06:07:45.256Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-downstairs:default.log
11492024-02-15T06:07:45.256Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11502024-02-15T06:07:46.350Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11512024-02-15T06:07:46.350Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-downstairs:downstairs-6e381e23-c258-4be7-861a-e94d0a5907f6.log (3018 bytes)
11522024-02-15T06:07:47.446Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-downstairs:downstairs-6e381e23-c258-4be7-861a-e94d0a5907f6.log
11532024-02-15T06:07:47.447Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-downstairs:downstairs-8619bfd6-f23d-452c-99ab-fdfd881e0438.log (0 bytes)
11542024-02-15T06:07:48.516Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-downstairs:downstairs-8619bfd6-f23d-452c-99ab-fdfd881e0438.log
11552024-02-15T06:07:48.544Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-downstairs:downstairs-8619bfd6-f23d-452c-99ab-fdfd881e0438.log.0 (3125 bytes)
11562024-02-15T06:07:49.611Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_1d816ece-0442-4979-9d32-7665ef85afde/root/var/svc/log/oxide-crucible-downstairs:downstairs-8619bfd6-f23d-452c-99ab-fdfd881e0438.log.0
11572024-02-15T06:07:49.611Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_98f9db1c-1595-4f22-a928-321363864abe/root/var/svc/log/oxide-crucible-agent:default.log (0 bytes)
11582024-02-15T06:07:50.681Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_98f9db1c-1595-4f22-a928-321363864abe/root/var/svc/log/oxide-crucible-agent:default.log
11592024-02-15T06:07:50.681Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_98f9db1c-1595-4f22-a928-321363864abe/root/var/svc/log/oxide-crucible-agent:default.log.0 (13685 bytes)
11602024-02-15T06:07:51.797Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_98f9db1c-1595-4f22-a928-321363864abe/root/var/svc/log/oxide-crucible-agent:default.log.0
11612024-02-15T06:07:51.798Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_98f9db1c-1595-4f22-a928-321363864abe/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11622024-02-15T06:07:52.867Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_98f9db1c-1595-4f22-a928-321363864abe/root/var/svc/log/oxide-crucible-downstairs:default.log
11632024-02-15T06:07:52.867Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_98f9db1c-1595-4f22-a928-321363864abe/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11642024-02-15T06:07:53.961Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_98f9db1c-1595-4f22-a928-321363864abe/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11652024-02-15T06:07:53.962Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_98f9db1c-1595-4f22-a928-321363864abe/root/var/svc/log/oxide-crucible-downstairs:downstairs-4b3e3539-136e-4fcd-8953-2c44d261017e.log (0 bytes)
11662024-02-15T06:07:55.032Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_98f9db1c-1595-4f22-a928-321363864abe/root/var/svc/log/oxide-crucible-downstairs:downstairs-4b3e3539-136e-4fcd-8953-2c44d261017e.log
11672024-02-15T06:07:55.032Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_98f9db1c-1595-4f22-a928-321363864abe/root/var/svc/log/oxide-crucible-downstairs:downstairs-4b3e3539-136e-4fcd-8953-2c44d261017e.log.0 (3124 bytes)
11682024-02-15T06:07:56.125Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_98f9db1c-1595-4f22-a928-321363864abe/root/var/svc/log/oxide-crucible-downstairs:downstairs-4b3e3539-136e-4fcd-8953-2c44d261017e.log.0
11692024-02-15T06:07:56.125Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_nexus_9363e23e-d709-4bd9-bca4-7fbe3ad55235/root/var/svc/log/oxide-nexus:default.log (832098 bytes)
11702024-02-15T06:07:56.841Zupload warning: file "/pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_nexus_9363e23e-d709-4bd9-bca4-7fbe3ad55235/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 832098 -> 1061014
11712024-02-15T06:07:57.911Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_nexus_9363e23e-d709-4bd9-bca4-7fbe3ad55235/root/var/svc/log/oxide-nexus:default.log
11722024-02-15T06:07:57.911Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_nexus_9363e23e-d709-4bd9-bca4-7fbe3ad55235/root/var/svc/log/oxide-nexus:default.log.0 (356813 bytes)
11732024-02-15T06:07:59.189Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_nexus_9363e23e-d709-4bd9-bca4-7fbe3ad55235/root/var/svc/log/oxide-nexus:default.log.0
11742024-02-15T06:07:59.190Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9677c70d-c2ae-4cdf-b7c0-3ded4a8fcd23/root/var/svc/log/oxide-crucible-agent:default.log (0 bytes)
11752024-02-15T06:08:00.260Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9677c70d-c2ae-4cdf-b7c0-3ded4a8fcd23/root/var/svc/log/oxide-crucible-agent:default.log
11762024-02-15T06:08:00.260Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9677c70d-c2ae-4cdf-b7c0-3ded4a8fcd23/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
11772024-02-15T06:08:01.358Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9677c70d-c2ae-4cdf-b7c0-3ded4a8fcd23/root/var/svc/log/oxide-crucible-agent:default.log.0
11782024-02-15T06:08:01.358Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9677c70d-c2ae-4cdf-b7c0-3ded4a8fcd23/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11792024-02-15T06:08:02.431Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9677c70d-c2ae-4cdf-b7c0-3ded4a8fcd23/root/var/svc/log/oxide-crucible-downstairs:default.log
11802024-02-15T06:08:02.431Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9677c70d-c2ae-4cdf-b7c0-3ded4a8fcd23/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11812024-02-15T06:08:03.540Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9677c70d-c2ae-4cdf-b7c0-3ded4a8fcd23/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11822024-02-15T06:08:03.540Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_0e3a6bd4-4538-4ccd-a063-9c7342f19203/root/var/svc/log/oxide-crucible-pantry:default.log (1133361 bytes)
11832024-02-15T06:08:05.307Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_0e3a6bd4-4538-4ccd-a063-9c7342f19203/root/var/svc/log/oxide-crucible-pantry:default.log
11842024-02-15T06:08:05.307Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_0e3a6bd4-4538-4ccd-a063-9c7342f19203/root/var/svc/log/oxide-crucible-pantry:default.log.0 (85086 bytes)
11852024-02-15T06:08:06.540Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_0e3a6bd4-4538-4ccd-a063-9c7342f19203/root/var/svc/log/oxide-crucible-pantry:default.log.0
11862024-02-15T06:08:06.566Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9bfdfe18-7777-415f-abb1-32f1b43e3b55/root/var/svc/log/oxide-crucible-agent:default.log (0 bytes)
11872024-02-15T06:08:07.618Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9bfdfe18-7777-415f-abb1-32f1b43e3b55/root/var/svc/log/oxide-crucible-agent:default.log
11882024-02-15T06:08:07.618Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9bfdfe18-7777-415f-abb1-32f1b43e3b55/root/var/svc/log/oxide-crucible-agent:default.log.0 (13683 bytes)
11892024-02-15T06:08:08.772Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9bfdfe18-7777-415f-abb1-32f1b43e3b55/root/var/svc/log/oxide-crucible-agent:default.log.0
11902024-02-15T06:08:08.800Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9bfdfe18-7777-415f-abb1-32f1b43e3b55/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11912024-02-15T06:08:09.861Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9bfdfe18-7777-415f-abb1-32f1b43e3b55/root/var/svc/log/oxide-crucible-downstairs:default.log
11922024-02-15T06:08:09.861Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9bfdfe18-7777-415f-abb1-32f1b43e3b55/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11932024-02-15T06:08:10.962Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9bfdfe18-7777-415f-abb1-32f1b43e3b55/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11942024-02-15T06:08:10.962Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9bfdfe18-7777-415f-abb1-32f1b43e3b55/root/var/svc/log/oxide-crucible-downstairs:downstairs-82480b91-ab39-4953-ba7e-a1e1d6fe31ab.log (0 bytes)
11952024-02-15T06:08:12.033Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9bfdfe18-7777-415f-abb1-32f1b43e3b55/root/var/svc/log/oxide-crucible-downstairs:downstairs-82480b91-ab39-4953-ba7e-a1e1d6fe31ab.log
11962024-02-15T06:08:12.033Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9bfdfe18-7777-415f-abb1-32f1b43e3b55/root/var/svc/log/oxide-crucible-downstairs:downstairs-82480b91-ab39-4953-ba7e-a1e1d6fe31ab.log.0 (3126 bytes)
11972024-02-15T06:08:13.134Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9bfdfe18-7777-415f-abb1-32f1b43e3b55/root/var/svc/log/oxide-crucible-downstairs:downstairs-82480b91-ab39-4953-ba7e-a1e1d6fe31ab.log.0
11982024-02-15T06:08:13.135Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_28701ada-e899-402b-a2fd-f7a3fd7c41df/root/var/svc/log/oxide-nexus:default.log (34216244 bytes)
11992024-02-15T06:08:22.876Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_28701ada-e899-402b-a2fd-f7a3fd7c41df/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 34216244 -> 34706235
12002024-02-15T06:08:23.956Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_28701ada-e899-402b-a2fd-f7a3fd7c41df/root/var/svc/log/oxide-nexus:default.log
12012024-02-15T06:08:23.957Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_28701ada-e899-402b-a2fd-f7a3fd7c41df/root/var/svc/log/oxide-nexus:default.log.0 (4839490 bytes)
12022024-02-15T06:08:26.702Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_28701ada-e899-402b-a2fd-f7a3fd7c41df/root/var/svc/log/oxide-nexus:default.log.0
12032024-02-15T06:08:26.703Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/svc/log/oxide-ntp:default.log (69 bytes)
12042024-02-15T06:08:27.781Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/svc/log/oxide-ntp:default.log
12052024-02-15T06:08:27.781Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/svc/log/oxide-ntp:default.log.0 (1453 bytes)
12062024-02-15T06:08:28.874Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/svc/log/oxide-ntp:default.log.0
12072024-02-15T06:08:28.874Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/log/chrony/measurements.log (23564 bytes)
12082024-02-15T06:08:29.037Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/log/chrony/measurements.log" changed size mid upload: 23564 -> 30140
12092024-02-15T06:08:30.120Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/log/chrony/measurements.log
12102024-02-15T06:08:30.120Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/log/chrony/statistics.log (15912 bytes)
12112024-02-15T06:08:30.243Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/log/chrony/statistics.log" changed size mid upload: 15912 -> 21411
12122024-02-15T06:08:31.316Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/log/chrony/statistics.log
12132024-02-15T06:08:31.316Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/log/chrony/tracking.log (2509 bytes)
12142024-02-15T06:08:31.377Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/log/chrony/tracking.log" changed size mid upload: 2509 -> 2773
12152024-02-15T06:08:32.450Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_a62bfe46-059b-4ea1-b2a5-7f98d258138d/root/var/log/chrony/tracking.log