01HMG3HPDFQY08SSKKMSRPH6WY: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.27",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01HMG3K0RGB6AESQQF8SRGAGSA

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12024-01-19T06:28:04.185Zjob dependencies complete; ready to run (waiting for 50 m 27 s)
22024-01-19T06:45:40.185Zjob assigned to worker 01HMG78HVYBNE08KX2S6WD55S5 (queued for 17 m 36 s)
32024-01-19T06:45:40.364Zhttp://172.20.0.5:9969/ipxe/buskin... ok
42024-01-19T06:45:40.430Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/kernel/amd64/unix... ok
52024-01-19T06:45:40.475Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive... ok
62024-01-19T06:45:40.520Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive.hash... ok
72024-01-19T06:45:40.565Zhttp://172.20.0.5:9969/postboot/buskin... ok
82024-01-19T06:45:40.609ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
92024-01-19T06:45:40.654Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d1
102024-01-19T06:45:40.699ZOxide Helios Version helios-2.0.22333 64-bit
112024-01-19T06:45:40.743ZConfiguring devices.
122024-01-19T06:45:40.788Z##################################################
132024-01-19T06:45:40.833Z#### ############# ##
142024-01-19T06:45:40.880Z### ### ############ ##
152024-01-19T06:45:40.933Z## ### # ## ### ## Oxide Computer Company ##
162024-01-19T06:45:40.978Z## ## ## ### # ### ##
172024-01-19T06:45:41.022Z## # ### #### #### This Station Under ##
182024-01-19T06:45:41.067Z### ### #### # ### Computer Control ##
192024-01-19T06:45:41.111Z#### #### ### ## ##
202024-01-19T06:45:41.155Z##################################################
212024-01-19T06:45:41.220ZHostname: buskin
222024-01-19T06:45:41.267ZLoading smf(7) service descriptions:
232024-01-19T06:45:42.287Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap-d8e74790c9e6e17d.gz
242024-01-19T06:45:44.206Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap-d8e74790c9e6e17d.gz
252024-01-19T06:45:44.206Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
262024-01-19T06:46:14.358Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
272024-01-19T06:46:14.358Zdownloading input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-4b1652e1f243c546.gz
282024-01-19T06:46:45.356Zdownloaded input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-4b1652e1f243c546.gz
292024-01-19T06:46:45.356Zdownloading input: /input/ci-tools/work/caboose-util.gz
302024-01-19T06:46:46.747Zdownloaded input: /input/ci-tools/work/caboose-util.gz
312024-01-19T06:46:46.747Zdownloading input: /input/ci-tools/work/tufaceous.gz
322024-01-19T06:46:51.424Zdownloaded input: /input/ci-tools/work/tufaceous.gz
332024-01-19T06:46:51.463Zdownloading input: /input/package/work/version.txt
342024-01-19T06:46:51.502Zdownloaded input: /input/package/work/version.txt
352024-01-19T06:46:51.502Zdownloading input: /input/package/work/package.tar.gz
362024-01-19T06:47:12.236Zdownloaded input: /input/package/work/package.tar.gz
372024-01-19T06:47:12.236Zdownloading input: /input/package/work/global-zone-packages.tar.gz
382024-01-19T06:47:42.577Zdownloaded input: /input/package/work/global-zone-packages.tar.gz
392024-01-19T06:47:42.577Zdownloading input: /input/package/work/trampoline-global-zone-packages.tar.gz
402024-01-19T06:47:51.898Zdownloaded input: /input/package/work/trampoline-global-zone-packages.tar.gz
412024-01-19T06:47:51.933Zdownloading input: /input/package/work/zones/clickhouse.tar.gz
422024-01-19T06:48:47.835Zdownloaded input: /input/package/work/zones/clickhouse.tar.gz
432024-01-19T06:48:47.835Zdownloading input: /input/package/work/zones/clickhouse_keeper.tar.gz
442024-01-19T06:49:43.577Zdownloaded input: /input/package/work/zones/clickhouse_keeper.tar.gz
452024-01-19T06:49:43.577Zdownloading input: /input/package/work/zones/cockroachdb.tar.gz
462024-01-19T06:50:21.402Zdownloaded input: /input/package/work/zones/cockroachdb.tar.gz
472024-01-19T06:50:21.402Zdownloading input: /input/package/work/zones/crucible-pantry.tar.gz
482024-01-19T06:50:26.116Zdownloaded input: /input/package/work/zones/crucible-pantry.tar.gz
492024-01-19T06:50:26.116Zdownloading input: /input/package/work/zones/crucible.tar.gz
502024-01-19T06:50:34.877Zdownloaded input: /input/package/work/zones/crucible.tar.gz
512024-01-19T06:50:34.877Zdownloading input: /input/package/work/zones/external-dns.tar.gz
522024-01-19T06:50:41.785Zdownloaded input: /input/package/work/zones/external-dns.tar.gz
532024-01-19T06:50:41.786Zdownloading input: /input/package/work/zones/internal-dns.tar.gz
542024-01-19T06:50:48.684Zdownloaded input: /input/package/work/zones/internal-dns.tar.gz
552024-01-19T06:50:48.685Zdownloading input: /input/package/work/zones/ntp.tar.gz
562024-01-19T06:50:48.778Zdownloaded input: /input/package/work/zones/ntp.tar.gz
572024-01-19T06:50:48.818Zdownloading input: /input/package/work/zones/omicron-gateway-asic.tar.gz
582024-01-19T06:50:53.990Zdownloaded input: /input/package/work/zones/omicron-gateway-asic.tar.gz
592024-01-19T06:50:53.991Zdownloading input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
602024-01-19T06:50:59.195Zdownloaded input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
612024-01-19T06:50:59.196Zdownloading input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
622024-01-19T06:51:23.385Zdownloaded input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
632024-01-19T06:51:23.385Zdownloading input: /input/package/work/zones/omicron-nexus.tar.gz
642024-01-19T06:51:46.604Zdownloaded input: /input/package/work/zones/omicron-nexus.tar.gz
652024-01-19T06:51:46.604Zdownloading input: /input/package/work/zones/overlay.tar.gz
662024-01-19T06:51:46.741Zdownloaded input: /input/package/work/zones/overlay.tar.gz
672024-01-19T06:51:46.741Zdownloading input: /input/package/work/zones/oximeter-collector.tar.gz
682024-01-19T06:51:58.149Zdownloaded input: /input/package/work/zones/oximeter-collector.tar.gz
692024-01-19T06:51:58.149Zdownloading input: /input/package/work/zones/propolis-server.tar.gz
702024-01-19T06:52:16.642Zdownloaded input: /input/package/work/zones/propolis-server.tar.gz
712024-01-19T06:52:16.642Zdownloading input: /input/package/work/zones/switch-asic.tar.gz
722024-01-19T06:53:28.764Zdownloaded input: /input/package/work/zones/switch-asic.tar.gz
732024-01-19T06:53:28.764Zdownloading input: /input/package/work/zones/switch-softnpu.tar.gz
742024-01-19T06:54:17.995Zdownloaded input: /input/package/work/zones/switch-softnpu.tar.gz
 
752024-01-19T06:54:18.039Zstarting task 0: "setup"
762024-01-19T06:54:18.039Z++ uname -s
772024-01-19T06:54:18.039Z+ kern=SunOS
782024-01-19T06:54:18.039Z+ case "$kern" in
792024-01-19T06:54:18.039Z+ groupadd -g 12345 build
802024-01-19T06:54:18.039Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
812024-01-19T06:54:20.014Z+ zfs create -o mountpoint=/work rpool/work
822024-01-19T06:54:20.054Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
832024-01-19T06:54:20.054Z+ home_fs=zfs
842024-01-19T06:54:20.054Z+ [[ zfs == autofs ]]
852024-01-19T06:54:20.054Z+ mkdir -p /home/build
862024-01-19T06:54:20.054Z+ chown build:build /home/build /work
872024-01-19T06:54:22.042Z+ chmod 0700 /home/build /work
882024-01-19T06:54:22.081Zprocess exited: duration 4048 ms, exit code 0
 
892024-01-19T06:54:22.185Zstarting task 1: "authentication"
902024-01-19T06:54:22.265Zprocess exited: duration 70 ms, exit code 0
 
912024-01-19T06:54:22.344Zstarting task 2: "build"
922024-01-19T06:54:22.383Z+ trap _exit_trap EXIT
932024-01-19T06:54:22.384Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
942024-01-19T06:54:22.549Zsecpolicy_fs_linkdir: 0x55 = 0x55
952024-01-19T06:54:22.549Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
962024-01-19T06:54:22.549Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
972024-01-19T06:54:22.549Zsecpolicy_fs_linkdir+3: 0 = 0xe5
982024-01-19T06:54:22.549Zsecpolicy_fs_linkdir+4: 0 = 0xb8
992024-01-19T06:54:22.549Zsecpolicy_fs_linkdir+5: 0 = 0x1
1002024-01-19T06:54:22.550Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
1012024-01-19T06:54:22.550Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
1022024-01-19T06:54:22.589Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
1032024-01-19T06:54:22.589Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
1042024-01-19T06:54:22.590Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
1052024-01-19T06:54:22.590Z+ [[ -d /opt/oxide ]]
1062024-01-19T06:54:22.590Z+ pfexec mv /opt/oxide /opt/oxide-underneath
1072024-01-19T06:54:22.590Z+ pfexec mkdir /opt/oxide
1082024-01-19T06:54:22.590Z+ pfexec mount -F tmpfs -O swap /opt/oxide
1092024-01-19T06:54:22.590Z+ [[ -d /opt/oxide-underneath ]]
1102024-01-19T06:54:22.590Z+ cd /opt/oxide-underneath
1112024-01-19T06:54:22.590Z+ pfexec tar ceEp@/f - .
1122024-01-19T06:54:22.590Z+ cd /opt/oxide
1132024-01-19T06:54:22.590Z+ pfexec tar xveEp@/f -
1142024-01-19T06:54:22.678Zx ., 0 bytes, 0 tape blocks
1152024-01-19T06:54:22.678Zx ./opte, 0 bytes, 0 tape blocks
1162024-01-19T06:54:22.678Zx ./opte/bin, 0 bytes, 0 tape blocks
1172024-01-19T06:54:22.678Zx ./opte/bin/opteadm, 52567720 bytes, 102672 tape blocks
1182024-01-19T06:54:22.717Z+ pfexec rm -rf /opt/oxide-underneath
1192024-01-19T06:54:22.718Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
1202024-01-19T06:54:22.718Z+ pfexec mkdir /opt/oxide/work
1212024-01-19T06:54:22.718Z+ pfexec chown build:build /opt/oxide/work
1222024-01-19T06:54:22.718Z+ cd /opt/oxide/work
1232024-01-19T06:54:22.718Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
1242024-01-19T06:54:24.371ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
1252024-01-19T06:54:24.371Zx out/mg-ddm-gz.tar, 62723072 bytes, 122506 tape blocks
1262024-01-19T06:54:24.371Zx out/omicron-sled-agent.tar, 120684032 bytes, 235711 tape blocks
1272024-01-19T06:54:24.371Zx out/target/test, 75 bytes, 1 tape blocks
1282024-01-19T06:54:24.371Zx out/npuzone/npuzone, 37184104 bytes, 72626 tape blocks
1292024-01-19T06:54:24.371Zx package-manifest.toml, 22919 bytes, 45 tape blocks
1302024-01-19T06:54:24.371Zx smf/sled-agent/non-gimlet/config.toml, 3448 bytes, 7 tape blocks
1312024-01-19T06:54:24.371Zx target/release/omicron-package, 44493328 bytes, 86902 tape blocks
1322024-01-19T06:54:24.371Zx tools/create_virtual_hardware.sh, 3176 bytes, 7 tape blocks
1332024-01-19T06:54:24.371Zx tools/virtual_hardware.sh, 3408 bytes, 7 tape blocks
1342024-01-19T06:54:24.371Zx tools/scrimlet/create-softnpu-zone.sh, 619 bytes, 2 tape blocks
1352024-01-19T06:54:24.372Zx tools/scrimlet/destroy-softnpu-zone.sh, 195 bytes, 1 tape blocks
1362024-01-19T06:54:24.372Zx tools/scrimlet/softnpu-init.sh, 2149 bytes, 5 tape blocks
1372024-01-19T06:54:24.372Zx tools/scrimlet/softnpu-zone.txt, 330 bytes, 1 tape blocks
1382024-01-19T06:54:24.372Zx tools/scrimlet/softnpu.toml, 171 bytes, 1 tape blocks
1392024-01-19T06:54:24.415Z
1402024-01-19T06:54:24.415Zreal 1.603917548
1412024-01-19T06:54:24.415Zuser 1.442411439
1422024-01-19T06:54:24.415Zsys 0.286714491
1432024-01-19T06:54:24.415Ztrap 0.000062492
1442024-01-19T06:54:24.415Ztflt 0.000000000
1452024-01-19T06:54:24.415Zdflt 0.000000000
1462024-01-19T06:54:24.415Zkflt 0.000000000
1472024-01-19T06:54:24.415Zlock 0.000000000
1482024-01-19T06:54:24.415Zslp 1.440923959
1492024-01-19T06:54:24.415Zlat 0.035599130
1502024-01-19T06:54:24.415Zstop 0.000128421
1512024-01-19T06:54:24.415Z+ cp /input/package/work/zones/clickhouse_keeper.tar.gz /input/package/work/zones/clickhouse.tar.gz /input/package/work/zones/cockroachdb.tar.gz /input/package/work/zones/crucible-pantry.tar.gz /input/package/work/zones/crucible.tar.gz /input/package/work/zones/external-dns.tar.gz /input/package/work/zones/internal-dns.tar.gz /input/package/work/zones/ntp.tar.gz /input/package/work/zones/omicron-gateway-asic.tar.gz /input/package/work/zones/omicron-gateway-softnpu.tar.gz /input/package/work/zones/omicron-nexus-single-sled.tar.gz /input/package/work/zones/omicron-nexus.tar.gz /input/package/work/zones/overlay.tar.gz /input/package/work/zones/oximeter-collector.tar.gz /input/package/work/zones/propolis-server.tar.gz /input/package/work/zones/switch-asic.tar.gz /input/package/work/zones/switch-softnpu.tar.gz out/
1522024-01-19T06:54:25.451Z+ mv out/omicron-nexus-single-sled.tar.gz out/omicron-nexus.tar.gz
1532024-01-19T06:54:25.550Z+ mkdir tests
1542024-01-19T06:54:25.550Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1552024-01-19T06:54:25.551Z+ ptime -m gunzip
1562024-01-19T06:54:25.551Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-d8e74790c9e6e17d
1572024-01-19T06:54:25.650Z
1582024-01-19T06:54:25.650Zreal 0.139210997
1592024-01-19T06:54:25.650Zuser 0.126302302
1602024-01-19T06:54:25.650Zsys 0.012668422
1612024-01-19T06:54:25.650Ztrap 0.000012299
1622024-01-19T06:54:25.650Ztflt 0.000000000
1632024-01-19T06:54:25.650Zdflt 0.000000000
1642024-01-19T06:54:25.650Zkflt 0.000000000
1652024-01-19T06:54:25.650Zlock 0.000000000
1662024-01-19T06:54:25.650Zslp 0.000022140
1672024-01-19T06:54:25.650Zlat 0.000024239
1682024-01-19T06:54:25.650Zstop 0.000125421
1692024-01-19T06:54:25.650Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-d8e74790c9e6e17d
1702024-01-19T06:54:25.650Z+ chmod a+x tests/bootstrap-d8e74790c9e6e17d
1712024-01-19T06:54:25.650Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1722024-01-19T06:54:25.650Z+ ptime -m gunzip
1732024-01-19T06:54:25.650Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1742024-01-19T06:54:27.978Z
1752024-01-19T06:54:28.018Zreal 2.393843696
1762024-01-19T06:54:28.018Zuser 2.240671144
1772024-01-19T06:54:28.018Zsys 0.152743658
1782024-01-19T06:54:28.018Ztrap 0.000183411
1792024-01-19T06:54:28.018Ztflt 0.000000000
1802024-01-19T06:54:28.018Zdflt 0.000000000
1812024-01-19T06:54:28.019Zkflt 0.000000000
1822024-01-19T06:54:28.019Zlock 0.000000000
1832024-01-19T06:54:28.019Zslp 0.000027220
1842024-01-19T06:54:28.019Zlat 0.000024099
1852024-01-19T06:54:28.019Zstop 0.000132211
1862024-01-19T06:54:28.019Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1872024-01-19T06:54:28.019Z+ chmod a+x tests/bootstrap
1882024-01-19T06:54:28.019Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1892024-01-19T06:54:28.019Z+ ptime -m gunzip
1902024-01-19T06:54:28.019Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-4b1652e1f243c546
1912024-01-19T06:54:30.445Z
1922024-01-19T06:54:30.559Zreal 2.452512232
1932024-01-19T06:54:30.559Zuser 2.303799327
1942024-01-19T06:54:30.559Zsys 0.148284749
1952024-01-19T06:54:30.559Ztrap 0.000185882
1962024-01-19T06:54:30.559Ztflt 0.000000000
1972024-01-19T06:54:30.559Zdflt 0.000000000
1982024-01-19T06:54:30.559Zkflt 0.000000000
1992024-01-19T06:54:30.559Zlock 0.000000000
2002024-01-19T06:54:30.559Zslp 0.000021650
2012024-01-19T06:54:30.559Zlat 0.000021870
2022024-01-19T06:54:30.559Zstop 0.000131831
2032024-01-19T06:54:30.559Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-4b1652e1f243c546
2042024-01-19T06:54:30.559Z+ chmod a+x tests/end_to_end_tests-4b1652e1f243c546
2052024-01-19T06:54:30.559Z+ bmat address ls
2062024-01-19T06:54:30.559ZNAME CIDR FIRST LAST COUNT
2072024-01-19T06:54:30.559Zextra 10.151.1.0/24 10.151.1.164 10.151.1.227 64
2082024-01-19T06:54:30.559Z++ bmat address ls -f extra -Ho first,last,count
2092024-01-19T06:54:30.559Z+ set -- 10.151.1.164 10.151.1.227 64
2102024-01-19T06:54:30.559Z+ EXTRA_IP_START=10.151.1.164
2112024-01-19T06:54:30.560Z+ EXTRA_IP_END=10.151.1.227
2122024-01-19T06:54:30.560Z+ EXTRA_IP_COUNT=64
2132024-01-19T06:54:30.560Z+ (( EXTRA_IP_COUNT >= 32 ))
2142024-01-19T06:54:30.560Z+ EXTRA_IP_BASE=10.151.1
2152024-01-19T06:54:30.560Z+ EXTRA_IP_FOCTET=164
2162024-01-19T06:54:30.560Z+ EXTRA_IP_LOCTET=227
2172024-01-19T06:54:30.560Z+ SERVICE_IP_POOL_START=10.151.1.164
2182024-01-19T06:54:30.560Z+ SERVICE_IP_POOL_END=10.151.1.173
2192024-01-19T06:54:30.560Z+ DNS_IP1=10.151.1.164
2202024-01-19T06:54:30.560Z+ DNS_IP2=10.151.1.165
2212024-01-19T06:54:30.560Z+ UPLINK_IP=10.151.1.174
2222024-01-19T06:54:30.560Z+ IPPOOL_START=10.151.1.175
2232024-01-19T06:54:30.560Z+ IPPOOL_END=10.151.1.227
2242024-01-19T06:54:30.560Z++ ipadm show-addr -po type,addr
2252024-01-19T06:54:30.560Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
2262024-01-19T06:54:30.560Z+ GATEWAY_IP=10.151.1.100
2272024-01-19T06:54:30.560Z+ [[ -n 10.151.1.100 ]]
2282024-01-19T06:54:30.560Z+ ping -s 10.151.1.100 56 1
2292024-01-19T06:54:30.560ZPING 10.151.1.100: 56 data bytes
2302024-01-19T06:54:30.560Z64 bytes from 10.151.1.100: icmp_seq=0. time=0.012 ms
2312024-01-19T06:54:30.560Z
2322024-01-19T06:54:30.561Z----10.151.1.100 PING Statistics----
2332024-01-19T06:54:30.561Z1 packets transmitted, 1 packets received, 0% packet loss
2342024-01-19T06:54:30.600Zround-trip (ms) min/avg/max/stddev = 0.012/0.012/0.012/-nan
2352024-01-19T06:54:30.600Z++ arp -an
2362024-01-19T06:54:30.600Z++ awk -vgw=10.151.1.100 '$2 == gw {print $NF}'
2372024-01-19T06:54:30.601Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
2382024-01-19T06:54:30.601Z+ routeadm -e ipv4-forwarding -u
2392024-01-19T06:54:30.714Z+ PXA_START=10.151.1.164
2402024-01-19T06:54:30.714Z+ PXA_END=10.151.1.227
2412024-01-19T06:54:30.714Z+ export GATEWAY_IP GATEWAY_MAC PXA_START PXA_END
2422024-01-19T06:54:30.714Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
2432024-01-19T06:54:30.754Z+ ZPOOL_VDEV_DIR=/scratch
2442024-01-19T06:54:30.754Z+ ptime -m pfexec ./tools/create_virtual_hardware.sh
2452024-01-19T06:54:30.754Z+++ dirname ./tools/create_virtual_hardware.sh
2462024-01-19T06:54:30.754Z++ cd ./tools
2472024-01-19T06:54:30.754Z++ pwd
2482024-01-19T06:54:30.754Z+ SOURCE_DIR=/opt/oxide/work/tools
2492024-01-19T06:54:30.755Z+ OMICRON_TOP=/opt/oxide/work/tools/..
2502024-01-19T06:54:30.755Z+ SOFTNPU_MODE=zone
2512024-01-19T06:54:30.755Z+ . /opt/oxide/work/tools/virtual_hardware.sh
2522024-01-19T06:54:30.755Z++ MARKER=/etc/opt/oxide/NO_INSTALL
2532024-01-19T06:54:30.755Z++ [[ -f /etc/opt/oxide/NO_INSTALL ]]
2542024-01-19T06:54:30.755Z++ dladm show-phys -p -o LINK
2552024-01-19T06:54:30.755Z++ head -1
2562024-01-19T06:54:30.835Z+ PHYSICAL_LINK=igb0
2572024-01-19T06:54:30.932Z+ echo 'Using igb0 as physical link'
2582024-01-19T06:54:30.932ZUsing igb0 as physical link
2592024-01-19T06:54:30.932Z+ ensure_run_as_root
2602024-01-19T06:54:30.932Z++ id -u
2612024-01-19T06:54:30.932Z+ [[ 0 -ne 0 ]]
2622024-01-19T06:54:30.932Z+ ensure_zpools
2632024-01-19T06:54:30.932Z+ ZPOOL_TYPES=('oxp_' 'oxi_')
2642024-01-19T06:54:30.932Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
2652024-01-19T06:54:30.932Z+ readarray -t ZPOOLS
2662024-01-19T06:54:30.932Z++ grep '"oxp_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
2672024-01-19T06:54:30.932Z++ sed 's/[ ",]//g'
2682024-01-19T06:54:30.932Z+ for ZPOOL in "${ZPOOLS[@]}"
2692024-01-19T06:54:30.932Z+ echo 'Zpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]'
2702024-01-19T06:54:30.932ZZpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]
2712024-01-19T06:54:30.932Z+ VDEV_PATH=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2722024-01-19T06:54:30.932Z+ [[ -f /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
2732024-01-19T06:54:30.932Z+ dd if=/dev/zero of=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
2742024-01-19T06:54:30.932Z0+0 records in
2752024-01-19T06:54:30.933Z0+0 records out
2762024-01-19T06:54:30.933Z0 bytes (0 iB) transferred in 0.000007 secs (0 iB/sec)
2772024-01-19T06:54:30.933Z+ success 'ZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
2782024-01-19T06:54:30.933Z+ set +x
2792024-01-19T06:54:30.933ZZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
2802024-01-19T06:54:30.933Z++ zpool list -o name
2812024-01-19T06:54:30.933Z++ grep oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
2822024-01-19T06:54:30.933Z+ [[ -z '' ]]
2832024-01-19T06:54:30.933Z+ zpool create -f oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2842024-01-19T06:54:31.045Z+ success 'ZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
2852024-01-19T06:54:31.046Z+ set +x
2862024-01-19T06:54:31.046ZZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists
2872024-01-19T06:54:31.046Z+ for ZPOOL in "${ZPOOLS[@]}"
2882024-01-19T06:54:31.046Z+ echo 'Zpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
2892024-01-19T06:54:31.046ZZpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]
2902024-01-19T06:54:31.046Z+ VDEV_PATH=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
2912024-01-19T06:54:31.046Z+ [[ -f /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
2922024-01-19T06:54:31.046Z+ dd if=/dev/zero of=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
2932024-01-19T06:54:31.046Z0+0 records in
2942024-01-19T06:54:31.046Z0+0 records out
2952024-01-19T06:54:31.046Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
2962024-01-19T06:54:31.046Z+ success 'ZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
2972024-01-19T06:54:31.046Z+ set +x
2982024-01-19T06:54:31.046ZZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
2992024-01-19T06:54:31.046Z++ zpool list -o name
3002024-01-19T06:54:31.046Z++ grep oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
3012024-01-19T06:54:31.046Z+ [[ -z '' ]]
3022024-01-19T06:54:31.046Z+ zpool create -f oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3032024-01-19T06:54:31.047Z+ success 'ZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3042024-01-19T06:54:31.047Z+ set +x
3052024-01-19T06:54:31.047ZZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3062024-01-19T06:54:31.047Z+ for ZPOOL in "${ZPOOLS[@]}"
3072024-01-19T06:54:31.047Z+ echo 'Zpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3082024-01-19T06:54:31.047ZZpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]
3092024-01-19T06:54:31.047Z+ VDEV_PATH=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3102024-01-19T06:54:31.047Z+ [[ -f /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3112024-01-19T06:54:31.047Z+ dd if=/dev/zero of=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3122024-01-19T06:54:31.047Z0+0 records in
3132024-01-19T06:54:31.047Z0+0 records out
3142024-01-19T06:54:31.047Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3152024-01-19T06:54:31.047Z+ success 'ZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3162024-01-19T06:54:31.047Z+ set +x
3172024-01-19T06:54:31.047ZZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3182024-01-19T06:54:31.047Z++ zpool list -o name
3192024-01-19T06:54:31.047Z++ grep oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
3202024-01-19T06:54:31.047Z+ [[ -z '' ]]
3212024-01-19T06:54:31.047Z+ zpool create -f oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3222024-01-19T06:54:31.146Z+ success 'ZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3232024-01-19T06:54:31.146Z+ set +x
3242024-01-19T06:54:31.146ZZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3252024-01-19T06:54:31.146Z+ for ZPOOL in "${ZPOOLS[@]}"
3262024-01-19T06:54:31.146Z+ echo 'Zpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3272024-01-19T06:54:31.146ZZpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]
3282024-01-19T06:54:31.146Z+ VDEV_PATH=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3292024-01-19T06:54:31.146Z+ [[ -f /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3302024-01-19T06:54:31.146Z+ dd if=/dev/zero of=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3312024-01-19T06:54:31.146Z0+0 records in
3322024-01-19T06:54:31.146Z0+0 records out
3332024-01-19T06:54:31.146Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3342024-01-19T06:54:31.146Z+ success 'ZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3352024-01-19T06:54:31.146Z+ set +x
3362024-01-19T06:54:31.146ZZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3372024-01-19T06:54:31.146Z++ zpool list -o name
3382024-01-19T06:54:31.146Z++ grep oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03
3392024-01-19T06:54:31.146Z+ [[ -z '' ]]
3402024-01-19T06:54:31.146Z+ zpool create -f oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3412024-01-19T06:54:31.147Z+ success 'ZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3422024-01-19T06:54:31.147Z+ set +x
3432024-01-19T06:54:31.147ZZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3442024-01-19T06:54:31.147Z+ for ZPOOL in "${ZPOOLS[@]}"
3452024-01-19T06:54:31.147Z+ echo 'Zpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3462024-01-19T06:54:31.147ZZpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]
3472024-01-19T06:54:31.147Z+ VDEV_PATH=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3482024-01-19T06:54:31.147Z+ [[ -f /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3492024-01-19T06:54:31.147Z+ dd if=/dev/zero of=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3502024-01-19T06:54:31.147Z0+0 records in
3512024-01-19T06:54:31.147Z0+0 records out
3522024-01-19T06:54:31.147Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3532024-01-19T06:54:31.147Z+ success 'ZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3542024-01-19T06:54:31.147Z+ set +x
3552024-01-19T06:54:31.147ZZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3562024-01-19T06:54:31.147Z++ zpool list -o name
3572024-01-19T06:54:31.147Z++ grep oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03
3582024-01-19T06:54:31.147Z+ [[ -z '' ]]
3592024-01-19T06:54:31.147Z+ zpool create -f oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3602024-01-19T06:54:31.147Z+ success 'ZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3612024-01-19T06:54:31.147Z+ set +x
3622024-01-19T06:54:31.147ZZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3632024-01-19T06:54:31.147Z+ for ZPOOL in "${ZPOOLS[@]}"
3642024-01-19T06:54:31.147Z+ echo 'Zpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]'
3652024-01-19T06:54:31.148ZZpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]
3662024-01-19T06:54:31.148Z+ VDEV_PATH=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3672024-01-19T06:54:31.148Z+ [[ -f /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev ]]
3682024-01-19T06:54:31.148Z+ dd if=/dev/zero of=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev bs=1 count=0 seek=20G
3692024-01-19T06:54:31.148Z0+0 records in
3702024-01-19T06:54:31.148Z0+0 records out
3712024-01-19T06:54:31.148Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3722024-01-19T06:54:31.148Z+ success 'ZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists'
3732024-01-19T06:54:31.148Z+ set +x
3742024-01-19T06:54:31.148ZZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists
3752024-01-19T06:54:31.148Z++ zpool list -o name
3762024-01-19T06:54:31.148Z++ grep oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930
3772024-01-19T06:54:31.148Z+ [[ -z '' ]]
3782024-01-19T06:54:31.148Z+ zpool create -f oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3792024-01-19T06:54:31.188Z+ success 'ZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists'
3802024-01-19T06:54:31.188Z+ set +x
3812024-01-19T06:54:31.189ZZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists
3822024-01-19T06:54:31.189Z+ for ZPOOL in "${ZPOOLS[@]}"
3832024-01-19T06:54:31.189Z+ echo 'Zpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]'
3842024-01-19T06:54:31.189ZZpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]
3852024-01-19T06:54:31.189Z+ VDEV_PATH=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
3862024-01-19T06:54:31.189Z+ [[ -f /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev ]]
3872024-01-19T06:54:31.189Z+ dd if=/dev/zero of=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev bs=1 count=0 seek=20G
3882024-01-19T06:54:31.189Z0+0 records in
3892024-01-19T06:54:31.189Z0+0 records out
3902024-01-19T06:54:31.189Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3912024-01-19T06:54:31.189Z+ success 'ZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists'
3922024-01-19T06:54:31.189Z+ set +x
3932024-01-19T06:54:31.189ZZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists
3942024-01-19T06:54:31.189Z++ zpool list -o name
3952024-01-19T06:54:31.189Z++ grep oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60
3962024-01-19T06:54:31.189Z+ [[ -z '' ]]
3972024-01-19T06:54:31.189Z+ zpool create -f oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
3982024-01-19T06:54:31.189Z+ success 'ZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists'
3992024-01-19T06:54:31.189Z+ set +x
4002024-01-19T06:54:31.189ZZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists
4012024-01-19T06:54:31.189Z+ for ZPOOL in "${ZPOOLS[@]}"
4022024-01-19T06:54:31.189Z+ echo 'Zpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]'
4032024-01-19T06:54:31.190ZZpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]
4042024-01-19T06:54:31.190Z+ VDEV_PATH=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4052024-01-19T06:54:31.190Z+ [[ -f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev ]]
4062024-01-19T06:54:31.190Z+ dd if=/dev/zero of=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev bs=1 count=0 seek=20G
4072024-01-19T06:54:31.190Z0+0 records in
4082024-01-19T06:54:31.190Z0+0 records out
4092024-01-19T06:54:31.190Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4102024-01-19T06:54:31.190Z+ success 'ZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists'
4112024-01-19T06:54:31.190Z+ set +x
4122024-01-19T06:54:31.190ZZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists
4132024-01-19T06:54:31.190Z++ zpool list -o name
4142024-01-19T06:54:31.190Z++ grep oxp_31bd71cd-4736-4a12-a387-9b74b050396f
4152024-01-19T06:54:31.190Z+ [[ -z '' ]]
4162024-01-19T06:54:31.190Z+ zpool create -f oxp_31bd71cd-4736-4a12-a387-9b74b050396f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4172024-01-19T06:54:31.281Z+ success 'ZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists'
4182024-01-19T06:54:31.282Z+ set +x
4192024-01-19T06:54:31.282ZZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists
4202024-01-19T06:54:31.282Z+ for ZPOOL in "${ZPOOLS[@]}"
4212024-01-19T06:54:31.282Z+ echo 'Zpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]'
4222024-01-19T06:54:31.282ZZpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]
4232024-01-19T06:54:31.282Z+ VDEV_PATH=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4242024-01-19T06:54:31.282Z+ [[ -f /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev ]]
4252024-01-19T06:54:31.282Z+ dd if=/dev/zero of=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev bs=1 count=0 seek=20G
4262024-01-19T06:54:31.282Z0+0 records in
4272024-01-19T06:54:31.282Z0+0 records out
4282024-01-19T06:54:31.282Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4292024-01-19T06:54:31.282Z+ success 'ZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists'
4302024-01-19T06:54:31.282Z+ set +x
4312024-01-19T06:54:31.282ZZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists
4322024-01-19T06:54:31.282Z++ zpool list -o name
4332024-01-19T06:54:31.282Z++ grep oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7
4342024-01-19T06:54:31.282Z+ [[ -z '' ]]
4352024-01-19T06:54:31.282Z+ zpool create -f oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4362024-01-19T06:54:31.380Z+ success 'ZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists'
4372024-01-19T06:54:31.381Z+ set +x
4382024-01-19T06:54:31.381ZZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists
4392024-01-19T06:54:31.381Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
4402024-01-19T06:54:31.381Z+ readarray -t ZPOOLS
4412024-01-19T06:54:31.381Z++ grep '"oxi_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
4422024-01-19T06:54:31.381Z++ sed 's/[ ",]//g'
4432024-01-19T06:54:31.381Z+ for ZPOOL in "${ZPOOLS[@]}"
4442024-01-19T06:54:31.381Z+ echo 'Zpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4452024-01-19T06:54:31.381ZZpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]
4462024-01-19T06:54:31.381Z+ VDEV_PATH=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4472024-01-19T06:54:31.382Z+ [[ -f /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4482024-01-19T06:54:31.382Z+ dd if=/dev/zero of=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4492024-01-19T06:54:31.382Z0+0 records in
4502024-01-19T06:54:31.382Z0+0 records out
4512024-01-19T06:54:31.382Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4522024-01-19T06:54:31.382Z+ success 'ZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4532024-01-19T06:54:31.382Z+ set +x
4542024-01-19T06:54:31.382ZZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4552024-01-19T06:54:31.382Z++ zpool list -o name
4562024-01-19T06:54:31.382Z++ grep oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b
4572024-01-19T06:54:31.382Z+ [[ -z '' ]]
4582024-01-19T06:54:31.382Z+ zpool create -f oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4592024-01-19T06:54:31.383Z+ success 'ZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4602024-01-19T06:54:31.383Z+ set +x
4612024-01-19T06:54:31.383ZZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4622024-01-19T06:54:31.383Z+ for ZPOOL in "${ZPOOLS[@]}"
4632024-01-19T06:54:31.383Z+ echo 'Zpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4642024-01-19T06:54:31.383ZZpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]
4652024-01-19T06:54:31.383Z+ VDEV_PATH=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4662024-01-19T06:54:31.383Z+ [[ -f /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4672024-01-19T06:54:31.383Z+ dd if=/dev/zero of=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4682024-01-19T06:54:31.383Z0+0 records in
4692024-01-19T06:54:31.383Z0+0 records out
4702024-01-19T06:54:31.384Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4712024-01-19T06:54:31.384Z+ success 'ZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4722024-01-19T06:54:31.384Z+ set +x
4732024-01-19T06:54:31.384ZZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4742024-01-19T06:54:31.384Z++ zpool list -o name
4752024-01-19T06:54:31.384Z++ grep oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b
4762024-01-19T06:54:31.384Z+ [[ -z '' ]]
4772024-01-19T06:54:31.384Z+ zpool create -f oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4782024-01-19T06:54:31.425Z+ success 'ZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4792024-01-19T06:54:31.425Z+ set +x
4802024-01-19T06:54:31.425ZZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4812024-01-19T06:54:31.425Z+ [[ zone == \z\o\n\e ]]
4822024-01-19T06:54:31.425Z+ ensure_simulated_links igb0
4832024-01-19T06:54:31.425Z+ local PHYSICAL_LINK=igb0
4842024-01-19T06:54:31.425Z+ INDICES=("0" "1")
4852024-01-19T06:54:31.425Z+ for I in "${INDICES[@]}"
4862024-01-19T06:54:31.425Z++ get_simnet_name_if_exists net0
4872024-01-19T06:54:31.425Z++ dladm show-simnet -p -o LINK net0
4882024-01-19T06:54:31.425Z++ echo ''
4892024-01-19T06:54:31.425Z+ [[ -z '' ]]
4902024-01-19T06:54:31.425Z+ dladm create-simnet -t net0
4912024-01-19T06:54:31.425Z+ dladm create-simnet -t sc0_0
4922024-01-19T06:54:31.425Z+ dladm modify-simnet -t -p net0 sc0_0
4932024-01-19T06:54:31.425Z+ dladm set-linkprop -p mtu=9000 sc0_0
4942024-01-19T06:54:31.425Z+ success 'Simnet net0/sc0_0 exists'
4952024-01-19T06:54:31.425Z+ set +x
4962024-01-19T06:54:31.425ZSimnet net0/sc0_0 exists
4972024-01-19T06:54:31.425Z+ for I in "${INDICES[@]}"
4982024-01-19T06:54:31.426Z++ get_simnet_name_if_exists net1
4992024-01-19T06:54:31.426Z++ dladm show-simnet -p -o LINK net1
5002024-01-19T06:54:31.426Z++ echo ''
5012024-01-19T06:54:31.426Z+ [[ -z '' ]]
5022024-01-19T06:54:31.426Z+ dladm create-simnet -t net1
5032024-01-19T06:54:31.426Z+ dladm create-simnet -t sc1_0
5042024-01-19T06:54:31.426Z+ dladm modify-simnet -t -p net1 sc1_0
5052024-01-19T06:54:31.426Z+ dladm set-linkprop -p mtu=9000 sc1_0
5062024-01-19T06:54:31.426Z+ success 'Simnet net1/sc1_0 exists'
5072024-01-19T06:54:31.426Z+ set +x
5082024-01-19T06:54:31.426ZSimnet net1/sc1_0 exists
5092024-01-19T06:54:31.426Z++ get_vnic_name_if_exists sc0_1
5102024-01-19T06:54:31.426Z++ dladm show-vnic -p -o LINK sc0_1
5112024-01-19T06:54:31.426Z++ echo ''
5122024-01-19T06:54:31.426Z+ [[ -z '' ]]
5132024-01-19T06:54:31.426Z+ dladm create-vnic -t sc0_1 -l igb0 -m a8:e1:de:01:70:1d
5142024-01-19T06:54:31.426Z+ [[ -v PROMISC_FILT_OFF ]]
5152024-01-19T06:54:31.426Z+ success 'Vnic sc0_1 exists'
5162024-01-19T06:54:31.426Z+ set +x
5172024-01-19T06:54:31.426ZVnic sc0_1 exists
5182024-01-19T06:54:31.426Z+ warn_if_no_proxy_arp
5192024-01-19T06:54:31.426Z+ [[ -v PXA_START ]]
5202024-01-19T06:54:31.427Z+ [[ -v PXA_END ]]
5212024-01-19T06:54:31.427Z+ ensure_softnpu_zone
5222024-01-19T06:54:31.427Z+ zoneadm list
5232024-01-19T06:54:31.427Z+ grep -q sidecar_softnpu
5242024-01-19T06:54:31.427Z+ [[ -f out/npuzone/npuzone ]]
5252024-01-19T06:54:31.427Z+ out/npuzone/npuzone create sidecar --omicron-zone --ports sc0_0,tfportrear0_0 --ports sc0_1,tfportqsfp0_0 --sidecar-lite-commit e3ea4b495ba0a71801ded0776ae4bbd31df57e26 --softnpu-commit dbab082dfa89da5db5ca2325c257089d2f130092
5262024-01-19T06:54:31.466Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/softnpu/image/dbab082dfa89da5db5ca2325c257089d2f130092/softnpu.sha256.txt
5272024-01-19T06:54:39.465Zdownloading softnpu ...done
5282024-01-19T06:54:39.505Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/libsidecar_lite.so.sha256.txt
5292024-01-19T06:54:46.469Zdownloading asic_program.so ...done
5302024-01-19T06:54:46.508Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/scadm.sha256.txt
5312024-01-19T06:54:54.010Zdownloading scadm ...done
5322024-01-19T06:54:54.049Zcreating port tfportrear0_0 for sc0_0
5332024-01-19T06:54:54.050Zcreating port tfportqsfp0_0 for sc0_1
5342024-01-19T06:54:54.050Zsoftnpu zone setup
5352024-01-19T06:54:54.050Zconfigure zone
5362024-01-19T06:54:54.089Zinstall zone
5372024-01-19T06:54:55.086Zboot zone
5382024-01-19T06:54:58.195Z[sidecar_softnpu] svcs milestone/network
5392024-01-19T06:54:58.237ZSTATE STIME FMRI
5402024-01-19T06:54:58.237Zonline 6:54:56 svc:/milestone/network:default
5412024-01-19T06:54:58.237Z[sidecar_softnpu] /softnpu/npu start
5422024-01-19T06:54:58.277Z
5432024-01-19T06:54:58.277Z+ /opt/oxide/work/tools/scrimlet/softnpu-init.sh
5442024-01-19T06:54:58.277Z+ GATEWAY_IP=10.151.1.100
5452024-01-19T06:54:58.277Z+ echo 'Using 10.151.1.100 as gateway ip'
5462024-01-19T06:54:58.277ZUsing 10.151.1.100 as gateway ip
5472024-01-19T06:54:58.277Z+ [[ ! -v GATEWAY_MAC ]]
5482024-01-19T06:54:58.277Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
5492024-01-19T06:54:58.277Z++ grep -c -E '^([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}$' /dev/fd/63
5502024-01-19T06:54:58.277Z+++ echo 18:c0:4d:0d:9f:b2
5512024-01-19T06:54:58.277Z+ COUNT=1
5522024-01-19T06:54:58.277Z+ [[ 1 -ne 1 ]]
5532024-01-19T06:54:58.277Z+ echo 'Using 18:c0:4d:0d:9f:b2 as gateway mac'
5542024-01-19T06:54:58.277ZUsing 18:c0:4d:0d:9f:b2 as gateway mac
5552024-01-19T06:54:58.277Z+ z_scadm add-arp-entry 10.151.1.100 18:c0:4d:0d:9f:b2
5562024-01-19T06:54:58.277Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-arp-entry 10.151.1.100 18:c0:4d:0d:9f:b2
5572024-01-19T06:54:58.317Z+ PXA_MAC=a8:e1:de:01:70:1d
5582024-01-19T06:54:58.317Z+ [[ -v PXA_START ]]
5592024-01-19T06:54:58.317Z+ z_scadm add-proxy-arp 10.151.1.164 10.151.1.227 a8:e1:de:01:70:1d
5602024-01-19T06:54:58.317Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-proxy-arp 10.151.1.164 10.151.1.227 a8:e1:de:01:70:1d
5612024-01-19T06:54:58.317Z+ z_scadm dump-state
5622024-01-19T06:54:58.317Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
5632024-01-19T06:54:58.357Zlocal v6:
5642024-01-19T06:54:58.357Zlocal v4:
5652024-01-19T06:54:58.357Zrouter v6:
5662024-01-19T06:54:58.357Zrouter v4_idx:
5672024-01-19T06:54:58.357Zrouter v4_routes:
5682024-01-19T06:54:58.357Zresolver v4:
5692024-01-19T06:54:58.357Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
5702024-01-19T06:54:58.357Zresolver v6:
5712024-01-19T06:54:58.357Znat v4:
5722024-01-19T06:54:58.357Znat_v6:
5732024-01-19T06:54:58.357Zport_mac:
5742024-01-19T06:54:58.357Zproxy arp:
5752024-01-19T06:54:58.357Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
5762024-01-19T06:54:58.357Z+ success 'softnpu zone exists'
5772024-01-19T06:54:58.357Z+ set +x
5782024-01-19T06:54:58.358Zsoftnpu zone exists
5792024-01-19T06:54:58.358Z
5802024-01-19T06:54:58.358Zreal 27.608069396
5812024-01-19T06:54:58.358Zuser 0.720068552
5822024-01-19T06:54:58.358Zsys 1.566808739
5832024-01-19T06:54:58.358Ztrap 0.000093995
5842024-01-19T06:54:58.358Ztflt 0.000328967
5852024-01-19T06:54:58.358Zdflt 0.000963473
5862024-01-19T06:54:58.358Zkflt 0.000000000
5872024-01-19T06:54:58.358Zlock 21:24.539918091
5882024-01-19T06:54:58.358Zslp 1:22.753888532
5892024-01-19T06:54:58.358Zlat 0.060983185
5902024-01-19T06:54:58.358Zstop 0.353261074
5912024-01-19T06:54:58.358Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml
5922024-01-19T06:54:58.398Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
5932024-01-19T06:54:58.398Z+ SILO_NAME=recovery
5942024-01-19T06:54:58.398Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
5952024-01-19T06:54:58.398Z+ EXTERNAL_DNS_DOMAIN=oxide.test
5962024-01-19T06:54:58.398Z+ sed -i~ '
5972024-01-19T06:54:58.398Z /^external_dns_ips/c\
5982024-01-19T06:54:58.398Zexternal_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
5992024-01-19T06:54:58.398Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
6002024-01-19T06:54:58.399Z /^first/c\
6012024-01-19T06:54:58.399Zfirst = "10.151.1.164"
6022024-01-19T06:54:58.399Z /^last/c\
6032024-01-19T06:54:58.399Zlast = "10.151.1.173"
6042024-01-19T06:54:58.399Z }
6052024-01-19T06:54:58.399Z /^\[rack_network_config/,/^$/ {
6062024-01-19T06:54:58.399Z /^infra_ip_first/c\
6072024-01-19T06:54:58.399Zinfra_ip_first = "10.151.1.174"
6082024-01-19T06:54:58.399Z /^infra_ip_last/c\
6092024-01-19T06:54:58.399Zinfra_ip_last = "10.151.1.174"
6102024-01-19T06:54:58.399Z }
6112024-01-19T06:54:58.399Z /^\[\[rack_network_config.ports/,/^$/ {
6122024-01-19T06:54:58.399Z /^routes/c\
6132024-01-19T06:54:58.399Zroutes = \[{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}\]
6142024-01-19T06:54:58.399Z /^addresses/c\
6152024-01-19T06:54:58.399Zaddresses = \["10.151.1.174/32"\]
6162024-01-19T06:54:58.399Z }
6172024-01-19T06:54:58.399Z' pkg/config-rss.toml
6182024-01-19T06:54:58.399Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
6192024-01-19T06:54:58.400Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
6202024-01-19T06:54:58.400Z+++ pkg/config-rss.toml Fri Jan 19 06:54:58 2024
6212024-01-19T06:54:58.400Z@@ -41,7 +41,7 @@
6222024-01-19T06:54:58.400Z # the DNS domain delegated to the rack by the customer. Each of these addresses
6232024-01-19T06:54:58.400Z # must be contained in one of the "internal services" IP Pool ranges listed
6242024-01-19T06:54:58.400Z # below.
6252024-01-19T06:54:58.400Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
6262024-01-19T06:54:58.400Z+external_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
6272024-01-19T06:54:58.400Z
6282024-01-19T06:54:58.400Z # Initial TLS certificates for the external API
6292024-01-19T06:54:58.400Z #
6302024-01-19T06:54:58.400Z@@ -77,8 +77,8 @@
6312024-01-19T06:54:58.400Z #
6322024-01-19T06:54:58.400Z # For more on this and what to put here, see docs/how-to-run.adoc.
6332024-01-19T06:54:58.400Z [[internal_services_ip_pool_ranges]]
6342024-01-19T06:54:58.400Z-first = "192.168.1.20"
6352024-01-19T06:54:58.400Z-last = "192.168.1.29"
6362024-01-19T06:54:58.400Z+first = "10.151.1.164"
6372024-01-19T06:54:58.400Z+last = "10.151.1.173"
6382024-01-19T06:54:58.400Z
6392024-01-19T06:54:58.400Z # TODO - this configuration is subject to change going forward. Ultimately these
6402024-01-19T06:54:58.400Z # parameters should be provided to the control plane via wicket, but we need to
6412024-01-19T06:54:58.400Z@@ -92,8 +92,8 @@
6422024-01-19T06:54:58.401Z # A range of IP addresses used by Boundary Services on the external network. In
6432024-01-19T06:54:58.401Z # a real system, these would be addresses of the uplink ports on the Sidecar.
6442024-01-19T06:54:58.401Z # With softnpu, only one address is used.
6452024-01-19T06:54:58.401Z-infra_ip_first = "192.168.1.30"
6462024-01-19T06:54:58.401Z-infra_ip_last = "192.168.1.30"
6472024-01-19T06:54:58.401Z+infra_ip_first = "10.151.1.174"
6482024-01-19T06:54:58.401Z+infra_ip_last = "10.151.1.174"
6492024-01-19T06:54:58.401Z
6502024-01-19T06:54:58.401Z # Configurations for BGP routers to run on the scrimlets.
6512024-01-19T06:54:58.401Z bgp = []
6522024-01-19T06:54:58.401Z@@ -101,9 +101,9 @@
6532024-01-19T06:54:58.401Z # You can configure multiple uplinks by repeating the following stanza
6542024-01-19T06:54:58.401Z [[rack_network_config.ports]]
6552024-01-19T06:54:58.401Z # Routes associated with this port.
6562024-01-19T06:54:58.401Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
6572024-01-19T06:54:58.401Z+routes = [{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}]
6582024-01-19T06:54:58.401Z # Addresses associated with this port.
6592024-01-19T06:54:58.401Z-addresses = ["192.168.1.30/32"]
6602024-01-19T06:54:58.401Z+addresses = ["10.151.1.174/32"]
6612024-01-19T06:54:58.401Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
6622024-01-19T06:54:58.401Z port = "qsfp0"
6632024-01-19T06:54:58.401Z # The speed of this port.
6642024-01-19T06:54:58.401Z+ true
6652024-01-19T06:54:58.402Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml
6662024-01-19T06:54:58.402Za pkg/config-rss.toml 7K
6672024-01-19T06:54:58.402Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~
6682024-01-19T06:54:58.402Z+ TLS_NAME=recovery.sys.oxide.test
6692024-01-19T06:54:58.402Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
6702024-01-19T06:54:58.932Z.........+...+....+.....+.+.....+.+...+.....+.+......+......+...............+..+...+...+.+......+...+.....+......+....+......+.........+.........+...+..+.+...........+...+.+......+......+.....+.........+...+......+.+.....+......+...+.........+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+.......+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+...+............+.......+.........+...........+...+...+...+....+....................+.......+.........+..............+.+..+...+................+.....+.+............+...+........+.+......+.....+..........+.........+.....+...+......+.+.........+.........+......+.................+...+...............+.+..................+..+.............+........+............+......+.......+..+.+..............+.......+.....+......+....+.....+.................................+..................+...+....+...+..+.+......+.....+.........+......+......+...+..........+.....+....+...+...+..+............+.......+...+........+.......+............+......................................................+..+...........................+...+...+.........+.+...........+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6712024-01-19T06:54:59.147Z...+.........+.....+..........+.........+..+.........+...+......+.+...+...+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*......+...+..+.+..+...+...+......+.............+..+.+.........+.........+..+....+........+............+.+..+......+.........+......+.+...............+...+..+.+.................+.+...............+.....+.+......+.....+...+.........+.+......+......+...+..+....+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...........+....+......+.....+...+......+....+...+............+............+...+.....+.............+...............+...............+...+.....+.+...+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6722024-01-19T06:54:59.187Z-----
6732024-01-19T06:54:59.187Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6742024-01-19T06:54:59.187Za pkg/initial-tls-cert.pem 2K
6752024-01-19T06:54:59.187Za pkg/initial-tls-key.pem 4K
6762024-01-19T06:54:59.187Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6772024-01-19T06:54:59.187Z+ rmdir pkg
6782024-01-19T06:54:59.187Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
6792024-01-19T06:54:59.187Z+ pfexec mkdir -p /usr/oxide
6802024-01-19T06:54:59.187Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0
6812024-01-19T06:54:59.915Z+ pfexec chmod +x /usr/oxide/oxide
6822024-01-19T06:54:59.956Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
6832024-01-19T06:55:18.215Z+ OMICRON_NO_UNINSTALL=1
6842024-01-19T06:55:18.215Z+ ptime -m pfexec ./target/release/omicron-package -t test install
6852024-01-19T06:55:18.255ZJan 19 06:55:18.306 DEBG target[test]: Target({"image": "standard", "machine": "non-gimlet", "rack-topology": "single-sled", "switch": "softnpu"})
6862024-01-19T06:55:18.255ZJan 19 06:55:18.308 INFO Installing service, dst: /opt/oxide/clickhouse.tar.gz, src: out/clickhouse.tar.gz
6872024-01-19T06:55:18.255ZJan 19 06:55:18.308 INFO Installing service, dst: /opt/oxide/clickhouse_keeper.tar.gz, src: out/clickhouse_keeper.tar.gz
6882024-01-19T06:55:18.255ZJan 19 06:55:18.308 INFO Installing service, dst: /opt/oxide/crucible_pantry.tar.gz, src: out/crucible-pantry.tar.gz
6892024-01-19T06:55:18.255ZJan 19 06:55:18.308 INFO Installing service, dst: /opt/oxide/nexus.tar.gz, src: out/omicron-nexus.tar.gz
6902024-01-19T06:55:18.255ZJan 19 06:55:18.308 INFO Installing service, dst: /opt/oxide/overlay.tar.gz, src: out/overlay.tar.gz
6912024-01-19T06:55:18.256ZJan 19 06:55:18.308 INFO Installing service, dst: /opt/oxide/external_dns.tar.gz, src: out/external-dns.tar.gz
6922024-01-19T06:55:18.256ZJan 19 06:55:18.308 INFO Installing service, dst: /opt/oxide/internal_dns.tar.gz, src: out/internal-dns.tar.gz
6932024-01-19T06:55:18.256ZJan 19 06:55:18.308 INFO Installing service, dst: /opt/oxide/cockroachdb.tar.gz, src: out/cockroachdb.tar.gz
6942024-01-19T06:55:18.256ZJan 19 06:55:18.308 INFO Installing service, dst: /opt/oxide/sled-agent.tar, src: out/omicron-sled-agent.tar
6952024-01-19T06:55:18.256ZJan 19 06:55:18.308 INFO Installing service, dst: /opt/oxide/oximeter.tar.gz, src: out/oximeter-collector.tar.gz
6962024-01-19T06:55:18.256ZJan 19 06:55:18.308 INFO Installing service, dst: /opt/oxide/crucible.tar.gz, src: out/crucible.tar.gz
6972024-01-19T06:55:18.256ZJan 19 06:55:18.309 INFO Installing service, dst: /opt/oxide/switch.tar.gz, src: out/switch-softnpu.tar.gz
6982024-01-19T06:55:18.256ZJan 19 06:55:18.309 INFO Installing service, dst: /opt/oxide/ntp.tar.gz, src: out/ntp.tar.gz
6992024-01-19T06:55:18.256ZJan 19 06:55:18.309 INFO Installing service, dst: /opt/oxide/propolis-server.tar.gz, src: out/propolis-server.tar.gz
7002024-01-19T06:55:18.256ZJan 19 06:55:18.309 INFO Installing service, dst: /opt/oxide/mg-ddm.tar, src: out/mg-ddm-gz.tar
7012024-01-19T06:55:18.256ZJan 19 06:55:18.309 INFO Installing service, dst: /opt/oxide/mgs.tar.gz, src: out/omicron-gateway-softnpu.tar.gz
7022024-01-19T06:55:18.491ZJan 19 06:55:18.566 INFO Unpacking service tarball, service_path: /opt/oxide/mg-ddm, tar_path: /opt/oxide/mg-ddm.tar
7032024-01-19T06:55:18.532ZJan 19 06:55:18.608 INFO Unpacking service tarball, service_path: /opt/oxide/sled-agent, tar_path: /opt/oxide/sled-agent.tar
7042024-01-19T06:55:18.600ZJan 19 06:55:18.676 INFO Installing bootstrap service from /opt/oxide/sled-agent/pkg/manifest.xml
7052024-01-19T06:55:18.730Z
7062024-01-19T06:55:18.835Zreal 0.509434140
7072024-01-19T06:55:18.835Zuser 0.101876894
7082024-01-19T06:55:18.835Zsys 1.941078721
7092024-01-19T06:55:18.836Ztrap 0.000004160
7102024-01-19T06:55:18.836Ztflt 0.000014579
7112024-01-19T06:55:18.836Zdflt 0.001075557
7122024-01-19T06:55:18.836Zkflt 0.000000000
7132024-01-19T06:55:18.836Zlock 45.880331085
7142024-01-19T06:55:18.836Zslp 0.723876077
7152024-01-19T06:55:18.836Zlat 0.002924715
7162024-01-19T06:55:18.836Zstop 0.201596061
7172024-01-19T06:55:18.836Z+ retry=0
7182024-01-19T06:55:18.836Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7192024-01-19T06:55:18.836Z+ [[ 0 -gt 30 ]]
7202024-01-19T06:55:18.836Z+ sleep 1
7212024-01-19T06:55:19.745Z+ retry=1
7222024-01-19T06:55:19.745Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7232024-01-19T06:55:19.785Z+ [[ 1 -gt 30 ]]
7242024-01-19T06:55:19.785Z+ sleep 1
7252024-01-19T06:55:20.759Z+ retry=2
7262024-01-19T06:55:20.759Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7272024-01-19T06:55:20.799Z+ [[ 2 -gt 30 ]]
7282024-01-19T06:55:20.799Z+ sleep 1
7292024-01-19T06:55:21.773Z+ retry=3
7302024-01-19T06:55:21.773Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7312024-01-19T06:55:21.812Z+ [[ 3 -gt 30 ]]
7322024-01-19T06:55:21.812Z+ sleep 1
7332024-01-19T06:55:22.786Z+ retry=4
7342024-01-19T06:55:22.786Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7352024-01-19T06:55:42.703Z+ [[ 4 -gt 30 ]]
7362024-01-19T06:55:42.703Z+ sleep 1
7372024-01-19T06:55:43.706Z+ retry=5
7382024-01-19T06:55:43.706Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7392024-01-19T06:55:43.745Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
7402024-01-19T06:55:43.746Zlocal v6:
7412024-01-19T06:55:43.746Zfe80::c8ec:81ff:feaf:762
7422024-01-19T06:55:43.746Zfd00:99::1
7432024-01-19T06:55:43.746Zlocal v4:
7442024-01-19T06:55:43.746Z10.151.1.174
7452024-01-19T06:55:43.746Zrouter v6:
7462024-01-19T06:55:43.746Zrouter v4_idx:
7472024-01-19T06:55:43.746Z0.0.0.0/0 -> 0
7482024-01-19T06:55:43.746Zrouter v4_routes:
7492024-01-19T06:55:43.746Z0 -> 10.151.1.100 (2)
7502024-01-19T06:55:43.746Zresolver v4:
7512024-01-19T06:55:43.746Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
7522024-01-19T06:55:43.746Zresolver v6:
7532024-01-19T06:55:43.746Zfe80::c8ec:81ff:feaf:762 -> ca:ec:81:af:07:62
7542024-01-19T06:55:43.746Zfe80::c8ac:a9ff:feb9:2ee5 -> ca:ac:a9:b9:2e:e5
7552024-01-19T06:55:43.746Znat v4:
7562024-01-19T06:55:43.746Znat_v6:
7572024-01-19T06:55:43.746Zport_mac:
7582024-01-19T06:55:43.746Z1000: a8:40:25:47:6e:51
7592024-01-19T06:55:43.746Z1: a8:40:25:47:6e:52
7602024-01-19T06:55:43.746Zproxy arp:
7612024-01-19T06:55:43.747Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
7622024-01-19T06:55:43.786Z+ retry=0
7632024-01-19T06:55:43.786Z+++ zoneadm list -n
7642024-01-19T06:55:43.786Z+++ grep oxz_ntp
7652024-01-19T06:55:43.787Z++ pfexec svcs -z -Hostate oxide/ntp
7662024-01-19T06:55:43.787Zsvcs: zone '-Hostate': entity not found
7672024-01-19T06:55:43.787Z++ true
7682024-01-19T06:55:43.787Z+ [[ '' != online ]]
7692024-01-19T06:55:43.787Z+ [[ 0 -gt 60 ]]
7702024-01-19T06:55:43.787Z+ sleep 1
7712024-01-19T06:55:44.769Z+ retry=1
7722024-01-19T06:55:44.809Z+++ zoneadm list -n
7732024-01-19T06:55:44.809Z+++ grep oxz_ntp
7742024-01-19T06:55:44.809Z++ pfexec svcs -z -Hostate oxide/ntp
7752024-01-19T06:55:44.809Zsvcs: zone '-Hostate': entity not found
7762024-01-19T06:55:44.809Z++ true
7772024-01-19T06:55:44.809Z+ [[ '' != online ]]
7782024-01-19T06:55:44.809Z+ [[ 1 -gt 60 ]]
7792024-01-19T06:55:44.809Z+ sleep 1
7802024-01-19T06:55:45.792Z+ retry=2
7812024-01-19T06:55:45.831Z+++ zoneadm list -n
7822024-01-19T06:55:45.831Z+++ grep oxz_ntp
7832024-01-19T06:55:45.831Z++ pfexec svcs -z -Hostate oxide/ntp
7842024-01-19T06:55:45.831Zsvcs: zone '-Hostate': entity not found
7852024-01-19T06:55:45.831Z++ true
7862024-01-19T06:55:45.831Z+ [[ '' != online ]]
7872024-01-19T06:55:45.831Z+ [[ 2 -gt 60 ]]
7882024-01-19T06:55:45.832Z+ sleep 1
7892024-01-19T06:55:46.814Z+ retry=3
7902024-01-19T06:55:46.853Z+++ zoneadm list -n
7912024-01-19T06:55:46.853Z+++ grep oxz_ntp
7922024-01-19T06:55:46.854Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
7932024-01-19T06:55:46.893Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7942024-01-19T06:55:46.893Z++ true
7952024-01-19T06:55:46.893Z+ [[ '' != online ]]
7962024-01-19T06:55:46.893Z+ [[ 3 -gt 60 ]]
7972024-01-19T06:55:46.893Z+ sleep 1
7982024-01-19T06:55:47.859Z+ retry=4
7992024-01-19T06:55:47.899Z+++ zoneadm list -n
8002024-01-19T06:55:47.899Z+++ grep oxz_ntp
8012024-01-19T06:55:47.899Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8022024-01-19T06:55:47.899Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
8032024-01-19T06:55:47.900Z++ true
8042024-01-19T06:55:47.900Z+ [[ '' != online ]]
8052024-01-19T06:55:47.900Z+ [[ 4 -gt 60 ]]
8062024-01-19T06:55:47.900Z+ sleep 1
8072024-01-19T06:55:48.891Z+ retry=5
8082024-01-19T06:55:48.993Z+++ zoneadm list -n
8092024-01-19T06:55:48.993Z+++ grep oxz_ntp
8102024-01-19T06:55:48.993Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8112024-01-19T06:55:48.994Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
8122024-01-19T06:55:48.994Z++ true
8132024-01-19T06:55:48.994Z+ [[ '' != online ]]
8142024-01-19T06:55:48.994Z+ [[ 5 -gt 60 ]]
8152024-01-19T06:55:48.994Z+ sleep 1
8162024-01-19T06:55:49.920Z+ retry=6
8172024-01-19T06:55:49.963Z+++ zoneadm list -n
8182024-01-19T06:55:49.963Z+++ grep oxz_ntp
8192024-01-19T06:55:49.963Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8202024-01-19T06:55:49.963Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
8212024-01-19T06:55:49.963Z++ true
8222024-01-19T06:55:49.963Z+ [[ '' != online ]]
8232024-01-19T06:55:49.963Z+ [[ 6 -gt 60 ]]
8242024-01-19T06:55:49.963Z+ sleep 1
8252024-01-19T06:55:50.950Z+ retry=7
8262024-01-19T06:55:50.990Z+++ zoneadm list -n
8272024-01-19T06:55:50.990Z+++ grep oxz_ntp
8282024-01-19T06:55:50.990Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8292024-01-19T06:55:50.990Z+ [[ disabled != online ]]
8302024-01-19T06:55:50.990Z+ [[ 7 -gt 60 ]]
8312024-01-19T06:55:50.990Z+ sleep 1
8322024-01-19T06:55:51.974Z+ retry=8
8332024-01-19T06:55:52.013Z+++ zoneadm list -n
8342024-01-19T06:55:52.013Z+++ grep oxz_ntp
8352024-01-19T06:55:52.014Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8362024-01-19T06:55:52.014Z+ [[ disabled != online ]]
8372024-01-19T06:55:52.014Z+ [[ 8 -gt 60 ]]
8382024-01-19T06:55:52.014Z+ sleep 1
8392024-01-19T06:55:52.997Z+ retry=9
8402024-01-19T06:55:53.037Z+++ zoneadm list -n
8412024-01-19T06:55:53.037Z+++ grep oxz_ntp
8422024-01-19T06:55:53.037Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8432024-01-19T06:55:53.037Z+ [[ disabled != online ]]
8442024-01-19T06:55:53.037Z+ [[ 9 -gt 60 ]]
8452024-01-19T06:55:53.037Z+ sleep 1
8462024-01-19T06:55:54.019Z+ retry=10
8472024-01-19T06:55:54.060Z+++ zoneadm list -n
8482024-01-19T06:55:54.060Z+++ grep oxz_ntp
8492024-01-19T06:55:54.060Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8502024-01-19T06:55:54.060Z+ [[ disabled != online ]]
8512024-01-19T06:55:54.060Z+ [[ 10 -gt 60 ]]
8522024-01-19T06:55:54.060Z+ sleep 1
8532024-01-19T06:55:55.041Z+ retry=11
8542024-01-19T06:55:55.080Z+++ zoneadm list -n
8552024-01-19T06:55:55.080Z+++ grep oxz_ntp
8562024-01-19T06:55:55.080Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8572024-01-19T06:55:55.080Z+ [[ disabled != online ]]
8582024-01-19T06:55:55.080Z+ [[ 11 -gt 60 ]]
8592024-01-19T06:55:55.080Z+ sleep 1
8602024-01-19T06:55:56.063Z+ retry=12
8612024-01-19T06:55:56.103Z+++ zoneadm list -n
8622024-01-19T06:55:56.103Z+++ grep oxz_ntp
8632024-01-19T06:55:56.103Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8642024-01-19T06:55:56.103Z+ [[ disabled != online ]]
8652024-01-19T06:55:56.103Z+ [[ 12 -gt 60 ]]
8662024-01-19T06:55:56.103Z+ sleep 1
8672024-01-19T06:55:57.086Z+ retry=13
8682024-01-19T06:55:57.126Z+++ zoneadm list -n
8692024-01-19T06:55:57.126Z+++ grep oxz_ntp
8702024-01-19T06:55:57.126Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8712024-01-19T06:55:57.126Z+ [[ disabled != online ]]
8722024-01-19T06:55:57.126Z+ [[ 13 -gt 60 ]]
8732024-01-19T06:55:57.127Z+ sleep 1
8742024-01-19T06:55:58.108Z+ retry=14
8752024-01-19T06:55:58.148Z+++ zoneadm list -n
8762024-01-19T06:55:58.148Z+++ grep oxz_ntp
8772024-01-19T06:55:58.148Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8782024-01-19T06:55:58.148Z+ [[ disabled != online ]]
8792024-01-19T06:55:58.148Z+ [[ 14 -gt 60 ]]
8802024-01-19T06:55:58.148Z+ sleep 1
8812024-01-19T06:55:59.131Z+ retry=15
8822024-01-19T06:55:59.171Z+++ zoneadm list -n
8832024-01-19T06:55:59.171Z+++ grep oxz_ntp
8842024-01-19T06:55:59.171Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8852024-01-19T06:55:59.171Z+ [[ disabled != online ]]
8862024-01-19T06:55:59.171Z+ [[ 15 -gt 60 ]]
8872024-01-19T06:55:59.171Z+ sleep 1
 
8882024-01-19T06:55:59.245Zbuskin console login: Jan 19 06:55:58 buskin ip: ip: joining multicasts failed (4) on vopte0 - will use link layer broadcasts for multicast
 
8892024-01-19T06:56:00.153Z+ retry=16
8902024-01-19T06:56:00.193Z+++ zoneadm list -n
8912024-01-19T06:56:00.193Z+++ grep oxz_ntp
8922024-01-19T06:56:00.193Z++ pfexec svcs -z oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b -Hostate oxide/ntp
8932024-01-19T06:56:00.193Z+ [[ online != online ]]
8942024-01-19T06:56:00.193Z+ echo 'Waited for chrony: 16s'
8952024-01-19T06:56:00.193Z+ export RUST_BACKTRACE=1
8962024-01-19T06:56:00.193Z+ RUST_BACKTRACE=1
8972024-01-19T06:56:00.193ZWaited for chrony: 16s
8982024-01-19T06:56:00.193Z+ export E2E_TLS_CERT IPPOOL_START IPPOOL_END
8992024-01-19T06:56:00.193Z++ ./tests/bootstrap
9002024-01-19T06:56:00.232ZUsing external DNS server at 10.151.1.164:53
9012024-01-19T06:56:00.232Z2024-01-19 06:56:00.279914386 UTC: attempting to log into API
9022024-01-19T06:56:15.269Z2024-01-19 06:56:15.287992996 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9032024-01-19T06:56:16.242Z2024-01-19 06:56:16.286069583 UTC: attempting to log into API
9042024-01-19T06:56:31.276Z2024-01-19 06:56:31.277593065 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9052024-01-19T06:56:32.293Z2024-01-19 06:56:32.276246790 UTC: attempting to log into API
9062024-01-19T06:56:47.389Z2024-01-19 06:56:47.353210349 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9072024-01-19T06:56:48.377Z2024-01-19 06:56:48.354959438 UTC: attempting to log into API
9082024-01-19T06:57:03.420Z2024-01-19 06:57:03.394549007 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9092024-01-19T06:57:04.460Z2024-01-19 06:57:04.438952868 UTC: attempting to log into API
9102024-01-19T06:57:19.540Z2024-01-19 06:57:19.513877138 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9112024-01-19T06:57:20.548Z2024-01-19 06:57:20.514085109 UTC: attempting to log into API
 
9122024-01-19T06:57:21.952ZJan 19 06:57:21 buskin ip: ip: joining multicasts failed (4) on vopte1 - will use link layer broadcasts for multicast
9132024-01-19T06:57:27.897ZJan 19 06:57:27 buskin ip: ip: joining multicasts failed (4) on vopte4 - will use link layer broadcasts for multicast
 
9142024-01-19T06:57:35.570Z2024-01-19 06:57:35.547209525 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
 
9152024-01-19T06:57:35.574ZJan 19 06:57:35 buskin ip: ip: joining multicasts failed (4) on vopte5 - will use link layer broadcasts for multicast
 
9162024-01-19T06:57:36.571Z2024-01-19 06:57:36.548215755 UTC: attempting to log into API
 
9172024-01-19T06:57:43.793ZJan 19 06:57:43 buskin ip: ip: joining multicasts failed (4) on vopte2 - will use link layer broadcasts for multicast
9182024-01-19T06:57:50.978ZJan 19 06:57:50 buskin ip: ip: joining multicasts failed (4) on vopte3 - will use link layer broadcasts for multicast
 
9192024-01-19T06:57:51.605Z2024-01-19 06:57:51.584699110 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9202024-01-19T06:57:52.610Z2024-01-19 06:57:52.586922962 UTC: attempting to log into API
9212024-01-19T06:58:07.640Z2024-01-19 06:58:07.622046232 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
9222024-01-19T06:58:08.666Z2024-01-19 06:58:08.622787508 UTC: attempting to log into API
9232024-01-19T06:58:14.414Z2024-01-19 06:58:14.395675420 UTC: login succeeded
9242024-01-19T06:58:14.462Zwaiting for nexus to come up...
9252024-01-19T06:58:14.514Zcreating IP pool... 10.151.1.175 - 10.151.1.227
9262024-01-19T06:58:14.685Zsetting up quotas...
9272024-01-19T06:58:14.728Zensuring datasets are ready...
9282024-01-19T06:58:16.926Zlet's roll.
9292024-01-19T06:58:16.966Z+ eval 'OXIDE_HOST=https://recovery.sys.oxide.test
9302024-01-19T06:58:16.967ZOXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
9312024-01-19T06:58:16.967ZOXIDE_TOKEN=oxide-token-f71b2f1f056eedcae417f0ce50c4fb1293548953'
9322024-01-19T06:58:16.967Z++ OXIDE_HOST=https://recovery.sys.oxide.test
9332024-01-19T06:58:16.967Z++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
9342024-01-19T06:58:16.967Z++ OXIDE_TOKEN=oxide-token-f71b2f1f056eedcae417f0ce50c4fb1293548953
9352024-01-19T06:58:16.967Z+ export OXIDE_HOST OXIDE_TOKEN
9362024-01-19T06:58:16.967Z+ retry=0
9372024-01-19T06:58:16.967Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9382024-01-19T06:58:17.018Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images'
9392024-01-19T06:58:17.278Z{"status":"ok"}success
9402024-01-19T06:58:17.279ZProject {
9412024-01-19T06:58:17.318Z description: "some images",
9422024-01-19T06:58:17.318Z id: 8e1709d9-7316-45fe-ac6e-c2a104fb3285,
9432024-01-19T06:58:17.318Z name: Name(
9442024-01-19T06:58:17.319Z "images",
9452024-01-19T06:58:17.319Z ),
9462024-01-19T06:58:17.319Z time_created: 2024-01-19T06:58:17.070406Z,
9472024-01-19T06:58:17.319Z time_modified: 2024-01-19T06:58:17.070406Z,
9482024-01-19T06:58:17.319Z}
9492024-01-19T06:58:17.319Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11
9502024-01-19T07:03:49.442Zdone!
9512024-01-19T07:03:49.539Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11
9522024-01-19T07:03:49.653Zsuccess
9532024-01-19T07:03:49.653ZImage {
9542024-01-19T07:03:49.653Z block_size: ByteCount(
9552024-01-19T07:03:49.654Z 512,
9562024-01-19T07:03:49.654Z ),
9572024-01-19T07:03:49.698Z description: "debian 11 original base image",
9582024-01-19T07:03:49.698Z digest: None,
9592024-01-19T07:03:49.699Z id: f4a52280-f664-40b7-80d5-a9463880a711,
9602024-01-19T07:03:49.699Z name: Name(
9612024-01-19T07:03:49.699Z "debian11",
9622024-01-19T07:03:49.699Z ),
9632024-01-19T07:03:49.699Z os: "debian",
9642024-01-19T07:03:49.699Z project_id: None,
9652024-01-19T07:03:49.699Z size: ByteCount(
9662024-01-19T07:03:49.700Z 2147483648,
9672024-01-19T07:03:49.700Z ),
9682024-01-19T07:03:49.700Z time_created: 2024-01-19T07:03:49.381136Z,
9692024-01-19T07:03:49.700Z time_modified: 2024-01-19T07:03:49.612270Z,
9702024-01-19T07:03:49.700Z url: None,
9712024-01-19T07:03:49.700Z version: "11",
9722024-01-19T07:03:49.701Z}
9732024-01-19T07:03:49.701Z+ rm ./tests/bootstrap
9742024-01-19T07:03:49.750Z+ for test_bin in tests/*
9752024-01-19T07:03:49.750Z+ ./tests/bootstrap-d8e74790c9e6e17d
9762024-01-19T07:03:49.750Z
9772024-01-19T07:03:49.750Zrunning 0 tests
9782024-01-19T07:03:49.750Z
9792024-01-19T07:03:49.751Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
9802024-01-19T07:03:49.751Z
9812024-01-19T07:03:49.752Z+ for test_bin in tests/*
9822024-01-19T07:03:49.752Z+ ./tests/end_to_end_tests-4b1652e1f243c546
9832024-01-19T07:03:49.798Z
9842024-01-19T07:03:49.799Zrunning 2 tests
9852024-01-19T07:03:49.984Ztest no_spoof::no_spoof_here ... ok
9862024-01-19T07:04:25.029Ztest instance_launch::instance_launch ... ok
9872024-01-19T07:04:25.032Z
9882024-01-19T07:04:25.032Ztest result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 35.25s
9892024-01-19T07:04:25.032Z
9902024-01-19T07:04:25.072Z+ _exit_trap
9912024-01-19T07:04:25.072Z+ local status=0
9922024-01-19T07:04:25.072Z+ [[ 0 -eq 0 ]]
9932024-01-19T07:04:25.072Z+ exit 0
9942024-01-19T07:04:25.072Zprocess exited: duration 602682 ms, exit code 0
 
9952024-01-19T07:04:25.230Zfound 111 output files
9962024-01-19T07:04:25.230Zuploading: /var/svc/log/oxide-sled-agent:default.log (682685 bytes)
9972024-01-19T07:04:27.411Zupload warning: file "/var/svc/log/oxide-sled-agent:default.log" changed size mid upload: 682685 -> 683084
9982024-01-19T07:04:28.929Zuploaded: /var/svc/log/oxide-sled-agent:default.log
9992024-01-19T07:04:28.930Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_93801f24-441f-40db-b506-a0161b1a0586/root/var/svc/log/oxide-clickhouse:default.log (452952 bytes)
10002024-01-19T07:04:30.335Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_93801f24-441f-40db-b506-a0161b1a0586/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 452952 -> 464932
10012024-01-19T07:04:31.458Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_93801f24-441f-40db-b506-a0161b1a0586/root/var/svc/log/oxide-clickhouse:default.log
10022024-01-19T07:04:31.505Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_93801f24-441f-40db-b506-a0161b1a0586/root/var/svc/log/oxide-clickhouse:default.log.0 (306522 bytes)
10032024-01-19T07:04:33.648Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_93801f24-441f-40db-b506-a0161b1a0586/root/var/svc/log/oxide-clickhouse:default.log.0
10042024-01-19T07:04:33.648Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_93801f24-441f-40db-b506-a0161b1a0586/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
10052024-01-19T07:04:34.766Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_93801f24-441f-40db-b506-a0161b1a0586/root/var/svc/log/oxide-zone-network-setup:default.log
10062024-01-19T07:04:34.766Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_93801f24-441f-40db-b506-a0161b1a0586/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1709 bytes)
10072024-01-19T07:04:35.934Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_93801f24-441f-40db-b506-a0161b1a0586/root/var/svc/log/oxide-zone-network-setup:default.log.0
10082024-01-19T07:04:35.934Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_a0272bd0-99af-492b-87ba-d5b3bbea723d/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
10092024-01-19T07:04:37.470Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_a0272bd0-99af-492b-87ba-d5b3bbea723d/root/var/svc/log/oxide-cockroachdb:default.log
10102024-01-19T07:04:37.470Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_a0272bd0-99af-492b-87ba-d5b3bbea723d/root/var/svc/log/oxide-cockroachdb:default.log.0 (4404 bytes)
10112024-01-19T07:04:38.277Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_a0272bd0-99af-492b-87ba-d5b3bbea723d/root/var/svc/log/oxide-cockroachdb:default.log.0
10122024-01-19T07:04:38.278Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_a0272bd0-99af-492b-87ba-d5b3bbea723d/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
10132024-01-19T07:04:39.410Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_a0272bd0-99af-492b-87ba-d5b3bbea723d/root/var/svc/log/oxide-zone-network-setup:default.log
10142024-01-19T07:04:39.410Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_a0272bd0-99af-492b-87ba-d5b3bbea723d/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1706 bytes)
10152024-01-19T07:04:40.997Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_a0272bd0-99af-492b-87ba-d5b3bbea723d/root/var/svc/log/oxide-zone-network-setup:default.log.0
10162024-01-19T07:04:40.998Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6bc279f3-14f4-430c-bd1f-52077875bdc2/root/var/svc/log/oxide-crucible-agent:default.log (0 bytes)
10172024-01-19T07:04:42.114Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6bc279f3-14f4-430c-bd1f-52077875bdc2/root/var/svc/log/oxide-crucible-agent:default.log
10182024-01-19T07:04:42.155Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6bc279f3-14f4-430c-bd1f-52077875bdc2/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
10192024-01-19T07:04:43.274Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6bc279f3-14f4-430c-bd1f-52077875bdc2/root/var/svc/log/oxide-crucible-agent:default.log.0
10202024-01-19T07:04:43.276Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6bc279f3-14f4-430c-bd1f-52077875bdc2/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
10212024-01-19T07:04:44.451Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6bc279f3-14f4-430c-bd1f-52077875bdc2/root/var/svc/log/oxide-crucible-downstairs:default.log
10222024-01-19T07:04:44.451Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6bc279f3-14f4-430c-bd1f-52077875bdc2/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
10232024-01-19T07:04:45.612Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6bc279f3-14f4-430c-bd1f-52077875bdc2/root/var/svc/log/oxide-crucible-downstairs:default.log.0
10242024-01-19T07:04:45.612Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_78ffc6e0-c07c-424f-8bee-f4ada3a7fb1d/root/var/svc/log/oxide-external_dns:default.log (15099 bytes)
10252024-01-19T07:04:46.835Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_78ffc6e0-c07c-424f-8bee-f4ada3a7fb1d/root/var/svc/log/oxide-external_dns:default.log
10262024-01-19T07:04:46.836Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_78ffc6e0-c07c-424f-8bee-f4ada3a7fb1d/root/var/svc/log/oxide-external_dns:default.log.0 (11839 bytes)
10272024-01-19T07:04:48.046Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_78ffc6e0-c07c-424f-8bee-f4ada3a7fb1d/root/var/svc/log/oxide-external_dns:default.log.0
10282024-01-19T07:04:48.046Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_f0fbba6c-c906-47d4-8108-6326746b1253/root/var/svc/log/oxide-internal_dns:default.log (15023 bytes)
10292024-01-19T07:04:49.265Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_f0fbba6c-c906-47d4-8108-6326746b1253/root/var/svc/log/oxide-internal_dns:default.log
10302024-01-19T07:04:49.312Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_f0fbba6c-c906-47d4-8108-6326746b1253/root/var/svc/log/oxide-internal_dns:default.log.0 (98684 bytes)
10312024-01-19T07:04:50.630Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_f0fbba6c-c906-47d4-8108-6326746b1253/root/var/svc/log/oxide-internal_dns:default.log.0
10322024-01-19T07:04:50.630Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_91c51792-c9e1-4311-915c-4b52aee81a5f/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
10332024-01-19T07:04:51.825Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_91c51792-c9e1-4311-915c-4b52aee81a5f/root/var/svc/log/oxide-cockroachdb:default.log
10342024-01-19T07:04:51.826Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_91c51792-c9e1-4311-915c-4b52aee81a5f/root/var/svc/log/oxide-cockroachdb:default.log.0 (4378 bytes)
10352024-01-19T07:04:53.011Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_91c51792-c9e1-4311-915c-4b52aee81a5f/root/var/svc/log/oxide-cockroachdb:default.log.0
10362024-01-19T07:04:53.011Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_91c51792-c9e1-4311-915c-4b52aee81a5f/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
10372024-01-19T07:04:54.125Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_91c51792-c9e1-4311-915c-4b52aee81a5f/root/var/svc/log/oxide-zone-network-setup:default.log
10382024-01-19T07:04:54.125Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_91c51792-c9e1-4311-915c-4b52aee81a5f/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1706 bytes)
10392024-01-19T07:04:55.278Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_91c51792-c9e1-4311-915c-4b52aee81a5f/root/var/svc/log/oxide-zone-network-setup:default.log.0
10402024-01-19T07:04:55.317Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-agent:default.log (23938 bytes)
10412024-01-19T07:04:56.969Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-agent:default.log
10422024-01-19T07:04:56.969Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-agent:default.log.0 (9367 bytes)
10432024-01-19T07:04:58.172Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-agent:default.log.0
10442024-01-19T07:04:58.175Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
10452024-01-19T07:04:59.289Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:default.log
10462024-01-19T07:04:59.289Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
10472024-01-19T07:05:00.451Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:default.log.0
10482024-01-19T07:05:00.499Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:downstairs-0ff2b836-e4ec-4de5-accc-b41c2af81fdf.log (1092 bytes)
10492024-01-19T07:05:01.631Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:downstairs-0ff2b836-e4ec-4de5-accc-b41c2af81fdf.log
10502024-01-19T07:05:01.672Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:downstairs-0ff2b836-e4ec-4de5-accc-b41c2af81fdf.log.0 (4523 bytes)
10512024-01-19T07:05:02.791Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:downstairs-0ff2b836-e4ec-4de5-accc-b41c2af81fdf.log.0
10522024-01-19T07:05:02.834Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:downstairs-79942032-3b6a-45cd-9066-ad633bf29c3f.log (5512 bytes)
10532024-01-19T07:05:03.955Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:downstairs-79942032-3b6a-45cd-9066-ad633bf29c3f.log
10542024-01-19T07:05:03.957Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:downstairs-e9307d1e-32fe-4261-b8ec-5867635e4d79.log (2933 bytes)
10552024-01-19T07:05:05.128Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:downstairs-e9307d1e-32fe-4261-b8ec-5867635e4d79.log
10562024-01-19T07:05:05.128Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:snapshot-0ff2b836-e4ec-4de5-accc-b41c2af81fdf-4c6503a7-91a8-48e5-9b76-f92a92e131ac.log (5158 bytes)
10572024-01-19T07:05:06.318Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_872b2217-ca64-4d16-851c-d07acdccefa0/root/var/svc/log/oxide-crucible-downstairs:snapshot-0ff2b836-e4ec-4de5-accc-b41c2af81fdf-4c6503a7-91a8-48e5-9b76-f92a92e131ac.log
10582024-01-19T07:05:06.318Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_d2b4ac6c-21da-4f5d-9bca-be8de66ed364/root/var/svc/log/oxide-external_dns:default.log (15104 bytes)
10592024-01-19T07:05:06.472Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_d2b4ac6c-21da-4f5d-9bca-be8de66ed364/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 15104 -> 18878
10602024-01-19T07:05:07.612Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_d2b4ac6c-21da-4f5d-9bca-be8de66ed364/root/var/svc/log/oxide-external_dns:default.log
10612024-01-19T07:05:07.612Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_d2b4ac6c-21da-4f5d-9bca-be8de66ed364/root/var/svc/log/oxide-external_dns:default.log.0 (11829 bytes)
10622024-01-19T07:05:09.279Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_d2b4ac6c-21da-4f5d-9bca-be8de66ed364/root/var/svc/log/oxide-external_dns:default.log.0
10632024-01-19T07:05:09.279Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_785acdbc-a6de-4f53-8b0c-61b8ab566671/root/var/svc/log/oxide-internal_dns:default.log (15022 bytes)
10642024-01-19T07:05:09.497Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_785acdbc-a6de-4f53-8b0c-61b8ab566671/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 15022 -> 18777
10652024-01-19T07:05:10.616Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_785acdbc-a6de-4f53-8b0c-61b8ab566671/root/var/svc/log/oxide-internal_dns:default.log
10662024-01-19T07:05:10.617Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_785acdbc-a6de-4f53-8b0c-61b8ab566671/root/var/svc/log/oxide-internal_dns:default.log.0 (106405 bytes)
10672024-01-19T07:05:12.682Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_785acdbc-a6de-4f53-8b0c-61b8ab566671/root/var/svc/log/oxide-internal_dns:default.log.0
10682024-01-19T07:05:12.683Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_cc18a6fc-e263-4088-9f48-920cc531805a/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
10692024-01-19T07:05:13.405Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_cc18a6fc-e263-4088-9f48-920cc531805a/root/var/svc/log/oxide-cockroachdb:default.log
10702024-01-19T07:05:13.405Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_cc18a6fc-e263-4088-9f48-920cc531805a/root/var/svc/log/oxide-cockroachdb:default.log.0 (4406 bytes)
10712024-01-19T07:05:14.592Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_cc18a6fc-e263-4088-9f48-920cc531805a/root/var/svc/log/oxide-cockroachdb:default.log.0
10722024-01-19T07:05:14.592Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_cc18a6fc-e263-4088-9f48-920cc531805a/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
10732024-01-19T07:05:15.775Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_cc18a6fc-e263-4088-9f48-920cc531805a/root/var/svc/log/oxide-zone-network-setup:default.log
10742024-01-19T07:05:15.775Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_cc18a6fc-e263-4088-9f48-920cc531805a/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1706 bytes)
10752024-01-19T07:05:16.933Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_cc18a6fc-e263-4088-9f48-920cc531805a/root/var/svc/log/oxide-zone-network-setup:default.log.0
10762024-01-19T07:05:16.933Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-agent:default.log (6353 bytes)
10772024-01-19T07:05:18.101Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-agent:default.log
10782024-01-19T07:05:18.101Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-agent:default.log.0 (14114 bytes)
10792024-01-19T07:05:19.330Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-agent:default.log.0
10802024-01-19T07:05:19.379Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
10812024-01-19T07:05:20.472Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-downstairs:default.log
10822024-01-19T07:05:20.472Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
10832024-01-19T07:05:21.646Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-downstairs:default.log.0
10842024-01-19T07:05:21.646Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-downstairs:downstairs-6da60c6b-0b48-4b19-8b77-738f7b656dee.log (0 bytes)
10852024-01-19T07:05:22.766Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-downstairs:downstairs-6da60c6b-0b48-4b19-8b77-738f7b656dee.log
10862024-01-19T07:05:22.766Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-downstairs:downstairs-6da60c6b-0b48-4b19-8b77-738f7b656dee.log.0 (3048 bytes)
10872024-01-19T07:05:23.922Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-downstairs:downstairs-6da60c6b-0b48-4b19-8b77-738f7b656dee.log.0
10882024-01-19T07:05:23.922Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-downstairs:downstairs-ca1cd54b-1289-430f-9bad-5789340a9685.log (2933 bytes)
10892024-01-19T07:05:25.076Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_0c5c976c-8fc5-45e8-9bbc-7b79d53c6222/root/var/svc/log/oxide-crucible-downstairs:downstairs-ca1cd54b-1289-430f-9bad-5789340a9685.log
10902024-01-19T07:05:25.078Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_789f114d-a4a8-468c-9f80-5773189053c1/root/var/svc/log/oxide-internal_dns:default.log (15025 bytes)
10912024-01-19T07:05:25.306Zupload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_789f114d-a4a8-468c-9f80-5773189053c1/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 15025 -> 18782
10922024-01-19T07:05:26.424Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_789f114d-a4a8-468c-9f80-5773189053c1/root/var/svc/log/oxide-internal_dns:default.log
10932024-01-19T07:05:26.424Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_789f114d-a4a8-468c-9f80-5773189053c1/root/var/svc/log/oxide-internal_dns:default.log.0 (102347 bytes)
10942024-01-19T07:05:28.029Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_789f114d-a4a8-468c-9f80-5773189053c1/root/var/svc/log/oxide-internal_dns:default.log.0
10952024-01-19T07:05:28.029Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_ac85c3e5-1bae-4359-8b4e-a11ad3285bfa/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
10962024-01-19T07:05:29.152Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_ac85c3e5-1bae-4359-8b4e-a11ad3285bfa/root/var/svc/log/oxide-cockroachdb:default.log
10972024-01-19T07:05:29.152Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_ac85c3e5-1bae-4359-8b4e-a11ad3285bfa/root/var/svc/log/oxide-cockroachdb:default.log.0 (4406 bytes)
10982024-01-19T07:05:30.308Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_ac85c3e5-1bae-4359-8b4e-a11ad3285bfa/root/var/svc/log/oxide-cockroachdb:default.log.0
10992024-01-19T07:05:30.309Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_ac85c3e5-1bae-4359-8b4e-a11ad3285bfa/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
11002024-01-19T07:05:31.849Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_ac85c3e5-1bae-4359-8b4e-a11ad3285bfa/root/var/svc/log/oxide-zone-network-setup:default.log
11012024-01-19T07:05:31.850Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_ac85c3e5-1bae-4359-8b4e-a11ad3285bfa/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1706 bytes)
11022024-01-19T07:05:33.025Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_ac85c3e5-1bae-4359-8b4e-a11ad3285bfa/root/var/svc/log/oxide-zone-network-setup:default.log.0
11032024-01-19T07:05:33.026Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_d7692f2f-53e5-4a67-9a89-08a02855aa18/root/var/svc/log/oxide-crucible-agent:default.log (0 bytes)
11042024-01-19T07:05:34.142Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_d7692f2f-53e5-4a67-9a89-08a02855aa18/root/var/svc/log/oxide-crucible-agent:default.log
11052024-01-19T07:05:34.142Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_d7692f2f-53e5-4a67-9a89-08a02855aa18/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
11062024-01-19T07:05:35.321Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_d7692f2f-53e5-4a67-9a89-08a02855aa18/root/var/svc/log/oxide-crucible-agent:default.log.0
11072024-01-19T07:05:35.373Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_d7692f2f-53e5-4a67-9a89-08a02855aa18/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11082024-01-19T07:05:36.438Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_d7692f2f-53e5-4a67-9a89-08a02855aa18/root/var/svc/log/oxide-crucible-downstairs:default.log
11092024-01-19T07:05:36.439Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_d7692f2f-53e5-4a67-9a89-08a02855aa18/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11102024-01-19T07:05:37.603Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_d7692f2f-53e5-4a67-9a89-08a02855aa18/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11112024-01-19T07:05:37.603Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_pantry_11d446c8-f0df-4903-a4e6-7fafc3c74f11/root/var/svc/log/oxide-crucible-pantry:default.log (810196 bytes)
11122024-01-19T07:05:38.753Zupload warning: file "/pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_pantry_11d446c8-f0df-4903-a4e6-7fafc3c74f11/root/var/svc/log/oxide-crucible-pantry:default.log" changed size mid upload: 810196 -> 841897
11132024-01-19T07:05:39.885Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_pantry_11d446c8-f0df-4903-a4e6-7fafc3c74f11/root/var/svc/log/oxide-crucible-pantry:default.log
11142024-01-19T07:05:39.885Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_pantry_11d446c8-f0df-4903-a4e6-7fafc3c74f11/root/var/svc/log/oxide-crucible-pantry:default.log.0 (403531 bytes)
11152024-01-19T07:05:41.361Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_pantry_11d446c8-f0df-4903-a4e6-7fafc3c74f11/root/var/svc/log/oxide-crucible-pantry:default.log.0
11162024-01-19T07:05:41.361Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_76abbc1f-1010-48ea-b984-f4da8239d28b/root/var/svc/log/oxide-nexus:default.log (23503580 bytes)
11172024-01-19T07:05:49.457Zupload warning: file "/pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_76abbc1f-1010-48ea-b984-f4da8239d28b/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 23503580 -> 23580335
11182024-01-19T07:05:50.577Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_76abbc1f-1010-48ea-b984-f4da8239d28b/root/var/svc/log/oxide-nexus:default.log
11192024-01-19T07:05:50.577Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_76abbc1f-1010-48ea-b984-f4da8239d28b/root/var/svc/log/oxide-nexus:default.log.0 (12544875 bytes)
11202024-01-19T07:05:55.568Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_76abbc1f-1010-48ea-b984-f4da8239d28b/root/var/svc/log/oxide-nexus:default.log.0
11212024-01-19T07:05:55.568Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_a67c5b14-ac0e-4d63-a0e7-80da50f5daf5/root/var/svc/log/oxide-nexus:default.log (297622 bytes)
11222024-01-19T07:05:55.994Zupload warning: file "/pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_a67c5b14-ac0e-4d63-a0e7-80da50f5daf5/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 297622 -> 434643
11232024-01-19T07:05:57.132Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_a67c5b14-ac0e-4d63-a0e7-80da50f5daf5/root/var/svc/log/oxide-nexus:default.log
11242024-01-19T07:05:57.187Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_a67c5b14-ac0e-4d63-a0e7-80da50f5daf5/root/var/svc/log/oxide-nexus:default.log.0 (471065 bytes)
11252024-01-19T07:05:58.644Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_nexus_a67c5b14-ac0e-4d63-a0e7-80da50f5daf5/root/var/svc/log/oxide-nexus:default.log.0
11262024-01-19T07:05:58.644Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_bae4670d-b179-48e2-82c8-f55a25165cc8/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
11272024-01-19T07:05:59.764Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_bae4670d-b179-48e2-82c8-f55a25165cc8/root/var/svc/log/oxide-cockroachdb:default.log
11282024-01-19T07:05:59.806Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_bae4670d-b179-48e2-82c8-f55a25165cc8/root/var/svc/log/oxide-cockroachdb:default.log.0 (4407 bytes)
11292024-01-19T07:06:00.917Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_bae4670d-b179-48e2-82c8-f55a25165cc8/root/var/svc/log/oxide-cockroachdb:default.log.0
11302024-01-19T07:06:00.963Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_bae4670d-b179-48e2-82c8-f55a25165cc8/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
11312024-01-19T07:06:02.056Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_bae4670d-b179-48e2-82c8-f55a25165cc8/root/var/svc/log/oxide-zone-network-setup:default.log
11322024-01-19T07:06:02.056Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_bae4670d-b179-48e2-82c8-f55a25165cc8/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1706 bytes)
11332024-01-19T07:06:03.237Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_bae4670d-b179-48e2-82c8-f55a25165cc8/root/var/svc/log/oxide-zone-network-setup:default.log.0
11342024-01-19T07:06:03.238Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-agent:default.log (17035 bytes)
11352024-01-19T07:06:04.484Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-agent:default.log
11362024-01-19T07:06:04.484Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-agent:default.log.0 (14117 bytes)
11372024-01-19T07:06:05.777Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-agent:default.log.0
11382024-01-19T07:06:05.818Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11392024-01-19T07:06:06.890Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-downstairs:default.log
11402024-01-19T07:06:06.930Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11412024-01-19T07:06:08.052Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11422024-01-19T07:06:08.052Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-downstairs:downstairs-0d9811b6-ecf7-4951-9299-1c8e199cddda.log (0 bytes)
11432024-01-19T07:06:09.172Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-downstairs:downstairs-0d9811b6-ecf7-4951-9299-1c8e199cddda.log
11442024-01-19T07:06:09.212Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-downstairs:downstairs-0d9811b6-ecf7-4951-9299-1c8e199cddda.log.0 (3048 bytes)
11452024-01-19T07:06:10.337Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-downstairs:downstairs-0d9811b6-ecf7-4951-9299-1c8e199cddda.log.0
11462024-01-19T07:06:10.337Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-downstairs:downstairs-87a204f7-a1a0-435a-86ee-f4cc6cc20f09.log (5514 bytes)
11472024-01-19T07:06:11.509Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-downstairs:downstairs-87a204f7-a1a0-435a-86ee-f4cc6cc20f09.log
11482024-01-19T07:06:11.551Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-downstairs:downstairs-f7affbaa-4815-4077-ac15-db51fd1a5172.log (2930 bytes)
11492024-01-19T07:06:12.663Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_99cd078d-484e-42d9-847b-9d5d873c3ce5/root/var/svc/log/oxide-crucible-downstairs:downstairs-f7affbaa-4815-4077-ac15-db51fd1a5172.log
11502024-01-19T07:06:12.663Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_7ee1ed73-a778-43e1-8b3c-f2e271d25241/root/var/svc/log/oxide-crucible-pantry:default.log (0 bytes)
11512024-01-19T07:06:13.803Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_7ee1ed73-a778-43e1-8b3c-f2e271d25241/root/var/svc/log/oxide-crucible-pantry:default.log
11522024-01-19T07:06:13.803Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_7ee1ed73-a778-43e1-8b3c-f2e271d25241/root/var/svc/log/oxide-crucible-pantry:default.log.0 (2333 bytes)
11532024-01-19T07:06:14.970Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_7ee1ed73-a778-43e1-8b3c-f2e271d25241/root/var/svc/log/oxide-crucible-pantry:default.log.0
11542024-01-19T07:06:14.970Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_9a1623d9-a4c5-41cf-a8b9-d611e26c66d5/root/var/svc/log/oxide-crucible-agent:default.log (11143 bytes)
11552024-01-19T07:06:16.187Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_9a1623d9-a4c5-41cf-a8b9-d611e26c66d5/root/var/svc/log/oxide-crucible-agent:default.log
11562024-01-19T07:06:16.187Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_9a1623d9-a4c5-41cf-a8b9-d611e26c66d5/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
11572024-01-19T07:06:17.351Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_9a1623d9-a4c5-41cf-a8b9-d611e26c66d5/root/var/svc/log/oxide-crucible-agent:default.log.0
11582024-01-19T07:06:17.351Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_9a1623d9-a4c5-41cf-a8b9-d611e26c66d5/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11592024-01-19T07:06:18.469Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_9a1623d9-a4c5-41cf-a8b9-d611e26c66d5/root/var/svc/log/oxide-crucible-downstairs:default.log
11602024-01-19T07:06:18.472Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_9a1623d9-a4c5-41cf-a8b9-d611e26c66d5/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11612024-01-19T07:06:19.633Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_9a1623d9-a4c5-41cf-a8b9-d611e26c66d5/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11622024-01-19T07:06:19.633Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_9a1623d9-a4c5-41cf-a8b9-d611e26c66d5/root/var/svc/log/oxide-crucible-downstairs:downstairs-7b7f154a-f0aa-4787-a6bb-11b639f9397f.log (5515 bytes)
11632024-01-19T07:06:20.788Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_9a1623d9-a4c5-41cf-a8b9-d611e26c66d5/root/var/svc/log/oxide-crucible-downstairs:downstairs-7b7f154a-f0aa-4787-a6bb-11b639f9397f.log
11642024-01-19T07:06:20.827Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/svc/log/oxide-ntp:default.log (0 bytes)
11652024-01-19T07:06:21.904Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/svc/log/oxide-ntp:default.log
11662024-01-19T07:06:21.904Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/svc/log/oxide-ntp:default.log.0 (919 bytes)
11672024-01-19T07:06:23.074Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/svc/log/oxide-ntp:default.log.0
11682024-01-19T07:06:23.076Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-agent:default.log (10357 bytes)
11692024-01-19T07:06:24.276Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-agent:default.log
11702024-01-19T07:06:24.277Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-agent:default.log.0 (9359 bytes)
11712024-01-19T07:06:25.477Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-agent:default.log.0
11722024-01-19T07:06:25.477Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11732024-01-19T07:06:26.604Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-downstairs:default.log
11742024-01-19T07:06:26.605Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11752024-01-19T07:06:27.762Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11762024-01-19T07:06:27.763Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-downstairs:downstairs-6b920101-9ac7-4e32-bd07-48bed36b3b60.log (1092 bytes)
11772024-01-19T07:06:28.922Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-downstairs:downstairs-6b920101-9ac7-4e32-bd07-48bed36b3b60.log
11782024-01-19T07:06:28.967Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-downstairs:downstairs-6b920101-9ac7-4e32-bd07-48bed36b3b60.log.0 (4526 bytes)
11792024-01-19T07:06:30.098Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-downstairs:downstairs-6b920101-9ac7-4e32-bd07-48bed36b3b60.log.0
11802024-01-19T07:06:30.148Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-downstairs:snapshot-6b920101-9ac7-4e32-bd07-48bed36b3b60-4c6503a7-91a8-48e5-9b76-f92a92e131ac.log (5157 bytes)
11812024-01-19T07:06:31.271Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_c8b8fee3-4f71-4cd6-900b-c6931b77877c/root/var/svc/log/oxide-crucible-downstairs:snapshot-6b920101-9ac7-4e32-bd07-48bed36b3b60-4c6503a7-91a8-48e5-9b76-f92a92e131ac.log
11822024-01-19T07:06:31.272Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-agent:default.log (7758 bytes)
11832024-01-19T07:06:32.502Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-agent:default.log
11842024-01-19T07:06:32.502Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-agent:default.log.0 (9794 bytes)
11852024-01-19T07:06:33.707Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-agent:default.log.0
11862024-01-19T07:06:33.707Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11872024-01-19T07:06:34.885Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-downstairs:default.log
11882024-01-19T07:06:35.334Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11892024-01-19T07:06:36.067Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11902024-01-19T07:06:36.068Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-downstairs:downstairs-d2518a85-a382-4425-a66b-d39170fa55a9.log (1092 bytes)
11912024-01-19T07:06:37.255Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-downstairs:downstairs-d2518a85-a382-4425-a66b-d39170fa55a9.log
11922024-01-19T07:06:37.255Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-downstairs:downstairs-d2518a85-a382-4425-a66b-d39170fa55a9.log.0 (4529 bytes)
11932024-01-19T07:06:38.431Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-downstairs:downstairs-d2518a85-a382-4425-a66b-d39170fa55a9.log.0
11942024-01-19T07:06:38.484Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-downstairs:snapshot-d2518a85-a382-4425-a66b-d39170fa55a9-4c6503a7-91a8-48e5-9b76-f92a92e131ac.log (5157 bytes)
11952024-01-19T07:06:39.606Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6c13e0af-591f-40e3-b554-05da995650b5/root/var/svc/log/oxide-crucible-downstairs:snapshot-d2518a85-a382-4425-a66b-d39170fa55a9-4c6503a7-91a8-48e5-9b76-f92a92e131ac.log
11962024-01-19T07:06:39.607Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f8c70014-26f6-40d0-b3c7-1c84106a6c48/root/var/svc/log/oxide-crucible-agent:default.log (0 bytes)
11972024-01-19T07:06:40.728Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f8c70014-26f6-40d0-b3c7-1c84106a6c48/root/var/svc/log/oxide-crucible-agent:default.log
11982024-01-19T07:06:40.778Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f8c70014-26f6-40d0-b3c7-1c84106a6c48/root/var/svc/log/oxide-crucible-agent:default.log.0 (13683 bytes)
11992024-01-19T07:06:41.951Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f8c70014-26f6-40d0-b3c7-1c84106a6c48/root/var/svc/log/oxide-crucible-agent:default.log.0
12002024-01-19T07:06:41.951Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f8c70014-26f6-40d0-b3c7-1c84106a6c48/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
12012024-01-19T07:06:43.076Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f8c70014-26f6-40d0-b3c7-1c84106a6c48/root/var/svc/log/oxide-crucible-downstairs:default.log
12022024-01-19T07:06:43.076Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f8c70014-26f6-40d0-b3c7-1c84106a6c48/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
12032024-01-19T07:06:44.267Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f8c70014-26f6-40d0-b3c7-1c84106a6c48/root/var/svc/log/oxide-crucible-downstairs:default.log.0
12042024-01-19T07:06:44.267Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f8c70014-26f6-40d0-b3c7-1c84106a6c48/root/var/svc/log/oxide-crucible-downstairs:downstairs-d57e79b1-c451-448d-9f40-d230fec81ef7.log (0 bytes)
12052024-01-19T07:06:45.397Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f8c70014-26f6-40d0-b3c7-1c84106a6c48/root/var/svc/log/oxide-crucible-downstairs:downstairs-d57e79b1-c451-448d-9f40-d230fec81ef7.log
12062024-01-19T07:06:45.397Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f8c70014-26f6-40d0-b3c7-1c84106a6c48/root/var/svc/log/oxide-crucible-downstairs:downstairs-d57e79b1-c451-448d-9f40-d230fec81ef7.log.0 (3049 bytes)
12072024-01-19T07:06:47.030Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_f8c70014-26f6-40d0-b3c7-1c84106a6c48/root/var/svc/log/oxide-crucible-downstairs:downstairs-d57e79b1-c451-448d-9f40-d230fec81ef7.log.0
12082024-01-19T07:06:47.030Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_21612b06-d2cc-4332-83f8-c973b756923e/root/var/svc/log/oxide-crucible-pantry:default.log (0 bytes)
12092024-01-19T07:06:48.145Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_21612b06-d2cc-4332-83f8-c973b756923e/root/var/svc/log/oxide-crucible-pantry:default.log
12102024-01-19T07:06:48.185Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_21612b06-d2cc-4332-83f8-c973b756923e/root/var/svc/log/oxide-crucible-pantry:default.log.0 (2338 bytes)
12112024-01-19T07:06:49.298Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_21612b06-d2cc-4332-83f8-c973b756923e/root/var/svc/log/oxide-crucible-pantry:default.log.0
12122024-01-19T07:06:49.298Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_9701e47d-adef-4306-bc88-8d87db93a05b/root/var/svc/log/oxide-nexus:default.log (294748 bytes)
12132024-01-19T07:06:50.565Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_9701e47d-adef-4306-bc88-8d87db93a05b/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 294748 -> 439876
12142024-01-19T07:06:51.685Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_9701e47d-adef-4306-bc88-8d87db93a05b/root/var/svc/log/oxide-nexus:default.log
12152024-01-19T07:06:51.685Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_9701e47d-adef-4306-bc88-8d87db93a05b/root/var/svc/log/oxide-nexus:default.log.0 (599157 bytes)
12162024-01-19T07:06:54.217Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_9701e47d-adef-4306-bc88-8d87db93a05b/root/var/svc/log/oxide-nexus:default.log.0
12172024-01-19T07:06:54.265Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_be51095f-e3d0-48fc-9880-c4272a8e1aff/root/var/svc/log/oxide-oximeter:default.log (151451 bytes)
12182024-01-19T07:06:55.081Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_be51095f-e3d0-48fc-9880-c4272a8e1aff/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 151451 -> 233585
12192024-01-19T07:06:56.207Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_be51095f-e3d0-48fc-9880-c4272a8e1aff/root/var/svc/log/oxide-oximeter:default.log
12202024-01-19T07:06:56.256Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_be51095f-e3d0-48fc-9880-c4272a8e1aff/root/var/svc/log/oxide-oximeter:default.log.0 (79938 bytes)
12212024-01-19T07:06:57.844Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_be51095f-e3d0-48fc-9880-c4272a8e1aff/root/var/svc/log/oxide-oximeter:default.log.0
12222024-01-19T07:06:57.844Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/log/chrony/measurements.log (18906 bytes)
12232024-01-19T07:06:58.070Zupload warning: file "/pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/log/chrony/measurements.log" changed size mid upload: 18906 -> 25208
12242024-01-19T07:06:59.182Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/log/chrony/measurements.log
12252024-01-19T07:06:59.182Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/log/chrony/statistics.log (11934 bytes)
12262024-01-19T07:06:59.331Zupload warning: file "/pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/log/chrony/statistics.log" changed size mid upload: 11934 -> 16965
12272024-01-19T07:07:00.448Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/log/chrony/statistics.log
12282024-01-19T07:07:00.448Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/log/chrony/tracking.log (1718 bytes)
12292024-01-19T07:07:00.524Zupload warning: file "/pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/log/chrony/tracking.log" changed size mid upload: 1718 -> 2114
12302024-01-19T07:07:01.641Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_ntp_b6eab424-549d-4f15-be63-0662c67dc28b/root/var/log/chrony/tracking.log