01J9SRFWZ54XKJEXFXK085SQ0B: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log*",
        "%/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
        "/tmp/kstat/*.kstat",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.33",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01J9SRGAYXAVDQFEK7VS9RTXXR

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12024-10-09T23:05:57.089Zjob dependencies complete; ready to run (waiting for 0 s)
22024-10-09T23:08:06.354Zjob assigned to worker 01J9SRGCA5P8B3W9X7PHBHT74H (queued for 2 m 9 s)
32024-10-09T23:08:06.455Zhttp://172.20.0.5:9969/ipxe/buskin... ok
42024-10-09T23:08:06.497Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/kernel/amd64/unix... ok
52024-10-09T23:08:06.531Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive... ok
62024-10-09T23:08:06.565Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive.hash... ok
72024-10-09T23:08:06.599Zhttp://172.20.0.5:9969/postboot/buskin... ok
82024-10-09T23:08:06.632ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
92024-10-09T23:08:06.666Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d5
102024-10-09T23:08:06.699ZOxide Helios Version helios-2.0.22827 64-bit
112024-10-09T23:08:06.733ZConfiguring devices.
122024-10-09T23:08:06.767Z##################################################
132024-10-09T23:08:06.801Z#### ############# ##
142024-10-09T23:08:06.834Z### ### ############ ##
152024-10-09T23:08:06.868Z## ### # ## ### ## Oxide Computer Company ##
162024-10-09T23:08:06.901Z## ## ## ### # ### ##
172024-10-09T23:08:06.935Z## # ### #### #### This Station Under ##
182024-10-09T23:08:06.968Z### ### #### # ### Computer Control ##
192024-10-09T23:08:07.002Z#### #### ### ## ##
202024-10-09T23:08:07.036Z##################################################
212024-10-09T23:08:07.069ZHostname: buskin
222024-10-09T23:08:07.103ZLoading smf(7) service descriptions:
232024-10-09T23:08:07.724Zdownloading input: /input/package/work/package.tar.gz
242024-10-09T23:12:47.992Zdownloaded input: /input/package/work/package.tar.gz
 
252024-10-09T23:12:48.017Zstarting task 0: "setup"
262024-10-09T23:12:48.017Z++ uname -s
272024-10-09T23:12:48.017Z+ kern=SunOS
282024-10-09T23:12:48.017Z+ case "$kern" in
292024-10-09T23:12:48.017Z+ groupadd -g 12345 build
302024-10-09T23:12:48.017Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
312024-10-09T23:12:50.010Z+ zfs create -o mountpoint=/work rpool/work
322024-10-09T23:12:50.055Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
332024-10-09T23:12:50.079Z+ home_fs=zfs
342024-10-09T23:12:50.079Z+ [[ zfs == autofs ]]
352024-10-09T23:12:50.079Z+ mkdir -p /home/build
362024-10-09T23:12:50.079Z+ chown build:build /home/build /work
372024-10-09T23:12:52.063Z+ chmod 0700 /home/build /work
382024-10-09T23:12:52.086Zprocess exited: duration 4074 ms, exit code 0
 
392024-10-09T23:12:52.134Zstarting task 1: "authentication"
402024-10-09T23:12:52.177Zprocess exited: duration 43 ms, exit code 0
 
412024-10-09T23:12:52.226Zstarting task 2: "build"
422024-10-09T23:12:52.251Z+ trap _exit_trap EXIT
432024-10-09T23:12:52.251Z+ OPTE_COMMIT=f3002b356da7d0e4ca15beb66a5566a92919baaa
442024-10-09T23:12:52.251Z+ [[ xf3002b356da7d0e4ca15beb66a5566a92919baaa != \x ]]
452024-10-09T23:12:52.251Z+ curl -sSfOL https://buildomat.eng.oxide.computer/public/file/oxidecomputer/opte/module/f3002b356da7d0e4ca15beb66a5566a92919baaa/xde
462024-10-09T23:13:03.068Z+ pfexec rem_drv xde
472024-10-09T23:13:03.094Z+ pfexec mv xde /kernel/drv/amd64/xde
482024-10-09T23:13:03.158Z+ pfexec add_drv xde
492024-10-09T23:13:03.265Z+ curl -sSfOL https://buildomat.eng.oxide.computer/public/file/oxidecomputer/opte/release/f3002b356da7d0e4ca15beb66a5566a92919baaa/opteadm
502024-10-09T23:13:13.356Z+ chmod +x opteadm
512024-10-09T23:13:13.380Z+ cp opteadm /tmp/opteadm
522024-10-09T23:13:13.409Z+ pfexec mv opteadm /opt/oxide/opte/bin/opteadm
532024-10-09T23:13:13.474Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
542024-10-09T23:13:13.605Zsecpolicy_fs_linkdir: 0x55 = 0x55
552024-10-09T23:13:13.606Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
562024-10-09T23:13:13.606Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
572024-10-09T23:13:13.606Zsecpolicy_fs_linkdir+3: 0 = 0xe5
582024-10-09T23:13:13.606Zsecpolicy_fs_linkdir+4: 0 = 0xb8
592024-10-09T23:13:13.606Zsecpolicy_fs_linkdir+5: 0 = 0x1
602024-10-09T23:13:13.606Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
612024-10-09T23:13:13.606Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
622024-10-09T23:13:13.631Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
632024-10-09T23:13:13.631Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
642024-10-09T23:13:13.631Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
652024-10-09T23:13:13.631Z+ [[ -d /opt/oxide ]]
662024-10-09T23:13:13.631Z+ pfexec mv /opt/oxide /opt/oxide-underneath
672024-10-09T23:13:13.631Z+ pfexec mkdir /opt/oxide
682024-10-09T23:13:13.631Z+ pfexec mount -F tmpfs -O swap /opt/oxide
692024-10-09T23:13:13.631Z+ [[ -d /opt/oxide-underneath ]]
702024-10-09T23:13:13.631Z+ cd /opt/oxide-underneath
712024-10-09T23:13:13.631Z+ pfexec tar ceEp@/f - .
722024-10-09T23:13:13.631Z+ cd /opt/oxide
732024-10-09T23:13:13.631Z+ pfexec tar xveEp@/f -
742024-10-09T23:13:13.670Zx ., 0 bytes, 0 tape blocks
752024-10-09T23:13:13.670Zx ./opte, 0 bytes, 0 tape blocks
762024-10-09T23:13:13.670Zx ./opte/bin, 0 bytes, 0 tape blocks
772024-10-09T23:13:13.670Zx ./opte/bin/opteadm, 55763696 bytes, 108914 tape blocks
782024-10-09T23:13:13.695Z+ pfexec rm -rf /opt/oxide-underneath
792024-10-09T23:13:13.695Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
802024-10-09T23:13:13.720Z+ pfexec mkdir /opt/oxide/work
812024-10-09T23:13:13.720Z+ pfexec chown build:build /opt/oxide/work
822024-10-09T23:13:13.720Z+ cd /opt/oxide/work
832024-10-09T23:13:13.720Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
842024-10-09T23:13:26.133ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
852024-10-09T23:13:26.133Zx .github/buildomat/ci-env.sh, 196 bytes, 1 tape blocks
862024-10-09T23:13:26.133Zx out/target/test, 107 bytes, 1 tape blocks
872024-10-09T23:13:26.133Zx out/npuzone/npuzone, 23699288 bytes, 46288 tape blocks
882024-10-09T23:13:26.133Zx package-manifest.toml, 32084 bytes, 63 tape blocks
892024-10-09T23:13:26.133Zx smf/sled-agent/non-gimlet/config.toml, 3569 bytes, 7 tape blocks
902024-10-09T23:13:26.133Zx target/release/omicron-package, 33591856 bytes, 65610 tape blocks
912024-10-09T23:13:26.133Zx target/release/xtask, 12351008 bytes, 24124 tape blocks
922024-10-09T23:13:26.133Zx target/debug/bootstrap, 203705608 bytes, 397863 tape blocks
932024-10-09T23:13:26.133Zx tests/bootstrap-efd25e8616779453, 24349840 bytes, 47559 tape blocks
942024-10-09T23:13:26.134Zx tests/commtest-eaa404bccd4b5540, 24582256 bytes, 48013 tape blocks
952024-10-09T23:13:26.134Zx tests/dhcp_server-3126bc2315c994d1, 24349856 bytes, 47559 tape blocks
962024-10-09T23:13:26.134Zx tests/end_to_end_tests-f5a0d93c9d75949a, 221606912 bytes, 432826 tape blocks
972024-10-09T23:13:26.134Zx out/clickhouse.tar.gz, 239390429 bytes, 467560 tape blocks
982024-10-09T23:13:26.134Zx out/clickhouse_keeper.tar.gz, 239368292 bytes, 467517 tape blocks
992024-10-09T23:13:26.134Zx out/clickhouse_server.tar.gz, 239364424 bytes, 467509 tape blocks
1002024-10-09T23:13:26.134Zx out/cockroachdb.tar.gz, 135371642 bytes, 264398 tape blocks
1012024-10-09T23:13:26.134Zx out/crucible-dtrace.tar, 83456 bytes, 163 tape blocks
1022024-10-09T23:13:26.134Zx out/crucible-pantry-zone.tar.gz, 27607889 bytes, 53922 tape blocks
1032024-10-09T23:13:26.134Zx out/crucible-zone.tar.gz, 35987434 bytes, 70288 tape blocks
1042024-10-09T23:13:26.134Zx out/external-dns.tar.gz, 25821783 bytes, 50434 tape blocks
1052024-10-09T23:13:26.134Zx out/internal-dns.tar.gz, 25821172 bytes, 50432 tape blocks
1062024-10-09T23:13:26.134Zx out/mg-ddm-gz.tar, 64164864 bytes, 125322 tape blocks
1072024-10-09T23:13:26.134Zx out/nexus.tar.gz, 97408700 bytes, 190252 tape blocks
1082024-10-09T23:13:26.134Zx out/ntp.tar.gz, 9316399 bytes, 18197 tape blocks
1092024-10-09T23:13:26.134Zx out/omicron-gateway-softnpu.tar.gz, 21586298 bytes, 42161 tape blocks
1102024-10-09T23:13:26.134Zx out/omicron-sled-agent.tar, 104879616 bytes, 204843 tape blocks
1112024-10-09T23:13:26.134Zx out/overlay.tar.gz, 2622 bytes, 6 tape blocks
1122024-10-09T23:13:26.134Zx out/oximeter.tar.gz, 44809863 bytes, 87520 tape blocks
1132024-10-09T23:13:26.134Zx out/oxlog.tar, 3110400 bytes, 6075 tape blocks
1142024-10-09T23:13:26.134Zx out/probe.tar.gz, 2829470 bytes, 5527 tape blocks
1152024-10-09T23:13:26.134Zx out/propolis-server.tar.gz, 68085710 bytes, 132980 tape blocks
1162024-10-09T23:13:26.134Zx out/pumpkind-gz.tar, 27304960 bytes, 53330 tape blocks
1172024-10-09T23:13:26.135Zx out/switch-softnpu.tar.gz, 226585571 bytes, 442550 tape blocks
1182024-10-09T23:13:26.159Z
1192024-10-09T23:13:26.159Zreal 12.399449497
1202024-10-09T23:13:26.160Zuser 10.958748458
1212024-10-09T23:13:26.160Zsys 2.493200045
1222024-10-09T23:13:26.160Ztrap 0.000409606
1232024-10-09T23:13:26.160Ztflt 0.000000000
1242024-10-09T23:13:26.160Zdflt 0.000000000
1252024-10-09T23:13:26.160Zkflt 0.000000000
1262024-10-09T23:13:26.160Zlock 0.000000000
1272024-10-09T23:13:26.160Zslp 11.034117103
1282024-10-09T23:13:26.160Zlat 0.310057845
1292024-10-09T23:13:26.160Zstop 0.000137322
1302024-10-09T23:13:26.160Z+ source .github/buildomat/ci-env.sh
1312024-10-09T23:13:26.160Z++ export CARGO_TERM_COLOR=always
1322024-10-09T23:13:26.160Z++ CARGO_TERM_COLOR=always
1332024-10-09T23:13:26.160Z+ bmat address ls
1342024-10-09T23:13:26.160ZNAME CIDR FIRST LAST COUNT
1352024-10-09T23:13:26.160Zextra 10.151.1.0/24 10.151.1.164 10.151.1.227 64
1362024-10-09T23:13:26.160Z++ bmat address ls -f extra -Ho first,last,count
1372024-10-09T23:13:26.160Z+ set -- 10.151.1.164 10.151.1.227 64
1382024-10-09T23:13:26.160Z+ EXTRA_IP_START=10.151.1.164
1392024-10-09T23:13:26.160Z+ EXTRA_IP_END=10.151.1.227
1402024-10-09T23:13:26.160Z+ EXTRA_IP_COUNT=64
1412024-10-09T23:13:26.160Z+ (( EXTRA_IP_COUNT >= 32 ))
1422024-10-09T23:13:26.160Z+ EXTRA_IP_BASE=10.151.1
1432024-10-09T23:13:26.160Z+ EXTRA_IP_FOCTET=164
1442024-10-09T23:13:26.160Z+ EXTRA_IP_LOCTET=227
1452024-10-09T23:13:26.160Z+ SERVICE_IP_POOL_START=10.151.1.164
1462024-10-09T23:13:26.160Z+ SERVICE_IP_POOL_END=10.151.1.173
1472024-10-09T23:13:26.160Z+ DNS_IP1=10.151.1.164
1482024-10-09T23:13:26.160Z+ DNS_IP2=10.151.1.165
1492024-10-09T23:13:26.160Z+ UPLINK_IP=10.151.1.174
1502024-10-09T23:13:26.160Z+ IPPOOL_START=10.151.1.175
1512024-10-09T23:13:26.160Z+ IPPOOL_END=10.151.1.227
1522024-10-09T23:13:26.160Z++ ipadm show-addr -po type,addr
1532024-10-09T23:13:26.160Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
1542024-10-09T23:13:26.160Z+ GATEWAY_IP=10.151.1.100
1552024-10-09T23:13:26.160Z+ [[ -n 10.151.1.100 ]]
1562024-10-09T23:13:26.160Z+ ping -s 10.151.1.100 56 1
1572024-10-09T23:13:26.160ZPING 10.151.1.100: 56 data bytes
1582024-10-09T23:13:26.161Z64 bytes from 10.151.1.100: icmp_seq=0. time=0.011 ms
1592024-10-09T23:13:26.161Z
1602024-10-09T23:13:26.161Z----10.151.1.100 PING Statistics----
1612024-10-09T23:13:26.161Z1 packets transmitted, 1 packets received, 0% packet loss
1622024-10-09T23:13:26.161Zround-trip (ms) min/avg/max/stddev = 0.011/0.011/0.011/-nan
1632024-10-09T23:13:26.161Z++ arp -an
1642024-10-09T23:13:26.161Z++ awk -vgw=10.151.1.100 '$2 == gw {print $NF}'
1652024-10-09T23:13:26.161Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
1662024-10-09T23:13:26.161Z+ routeadm -e ipv4-forwarding -u
1672024-10-09T23:13:26.270Z+ PXA_START=10.151.1.164
1682024-10-09T23:13:26.270Z+ PXA_END=10.151.1.227
1692024-10-09T23:13:26.270Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
1702024-10-09T23:13:26.465Z+ ptime -m pfexec ./target/release/xtask virtual-hardware --vdev-dir /scratch create --gateway-ip 10.151.1.100 --gateway-mac 18:c0:4d:0d:9f:b2 --pxa-start 10.151.1.164 --pxa-end 10.151.1.227
1712024-10-09T23:13:26.490ZCouldn't find Cargo.toml, using /opt/oxide/work as workspace root
1722024-10-09T23:13:26.490Zcreating virtual hardware
1732024-10-09T23:13:26.490Zcreating /scratch/m2_0.vdev
1742024-10-09T23:13:26.490Zcreating /scratch/m2_1.vdev
1752024-10-09T23:13:26.490Zcreating /scratch/u2_0.vdev
1762024-10-09T23:13:26.490Zcreating /scratch/u2_1.vdev
1772024-10-09T23:13:26.490Zcreating /scratch/u2_2.vdev
1782024-10-09T23:13:26.490Zcreating /scratch/u2_3.vdev
1792024-10-09T23:13:26.490Zcreating /scratch/u2_4.vdev
1802024-10-09T23:13:26.490Zcreating /scratch/u2_5.vdev
1812024-10-09T23:13:26.491Zcreating /scratch/u2_6.vdev
1822024-10-09T23:13:26.491Zcreating /scratch/u2_7.vdev
1832024-10-09T23:13:26.491Zcreating /scratch/u2_8.vdev
1842024-10-09T23:13:26.516ZSimnet net0/sc0_0 exists
1852024-10-09T23:13:26.540ZSimnet net1/sc1_0 exists
1862024-10-09T23:13:26.540ZVnic sc0_1 exists
1872024-10-09T23:13:39.109ZUsing 10.151.1.100 as gateway ip
1882024-10-09T23:13:39.109Zusing 18:C0:4D:0D:9F:B2 as gateway mac
1892024-10-09T23:13:39.109Zconfiguring SoftNPU ARP entry
1902024-10-09T23:13:39.137Zconfiguring SoftNPU proxy ARP
1912024-10-09T23:13:39.190ZSoftNPU state:
1922024-10-09T23:13:39.190Z local v6:
1932024-10-09T23:13:39.190Z local v4:
1942024-10-09T23:13:39.190Z router v6:
1952024-10-09T23:13:39.190Z router v4_idx:
1962024-10-09T23:13:39.190Z router v4_routes:
1972024-10-09T23:13:39.190Z resolver v4:
1982024-10-09T23:13:39.190Z 10.151.1.100 -> 18:c0:4d:0d:9f:b2
1992024-10-09T23:13:39.190Z resolver v6:
2002024-10-09T23:13:39.190Z nat v4:
2012024-10-09T23:13:39.190Z nat_v6:
2022024-10-09T23:13:39.190Z port_mac:
2032024-10-09T23:13:39.190Z proxy arp:
2042024-10-09T23:13:39.190Z 10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
2052024-10-09T23:13:39.190Zcreated virtual hardware
2062024-10-09T23:13:39.214Z
2072024-10-09T23:13:39.214Zreal 12.722315982
2082024-10-09T23:13:39.214Zuser 0.671177225
2092024-10-09T23:13:39.214Zsys 3.217747033
2102024-10-09T23:13:39.214Ztrap 0.000068537
2112024-10-09T23:13:39.214Ztflt 0.000272196
2122024-10-09T23:13:39.214Zdflt 0.000432068
2132024-10-09T23:13:39.214Zkflt 0.000002320
2142024-10-09T23:13:39.214Zlock 9:53.980667061
2152024-10-09T23:13:39.214Zslp 38.423653342
2162024-10-09T23:13:39.214Zlat 0.079918327
2172024-10-09T23:13:39.214Zstop 0.362360601
2182024-10-09T23:13:39.215Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml pkg/config.toml
2192024-10-09T23:13:39.261Z+ sed -E -i~ 's/(m2|u2)(.*\.vdev)/\/scratch\/\1\2/g' pkg/config.toml
2202024-10-09T23:13:39.261Z+ diff -u pkg/config.toml~ pkg/config.toml
2212024-10-09T23:13:39.261Z--- pkg/config.toml~ Mon Jul 24 01:21:28 2006
2222024-10-09T23:13:39.261Z+++ pkg/config.toml Wed Oct 9 23:13:39 2024
2232024-10-09T23:13:39.261Z@@ -29,18 +29,18 @@
2242024-10-09T23:13:39.261Z # These paths have the prefix of either "u2" or "m2", followed by an underscore,
2252024-10-09T23:13:39.261Z # followed by a string that is embedded into their fake serial values.
2262024-10-09T23:13:39.261Z vdevs = [
2272024-10-09T23:13:39.262Z- "m2_0.vdev",
2282024-10-09T23:13:39.262Z- "m2_1.vdev",
2292024-10-09T23:13:39.262Z+ "/scratch/m2_0.vdev",
2302024-10-09T23:13:39.262Z+ "/scratch/m2_1.vdev",
2312024-10-09T23:13:39.262Z
2322024-10-09T23:13:39.262Z- "u2_0.vdev",
2332024-10-09T23:13:39.262Z- "u2_1.vdev",
2342024-10-09T23:13:39.262Z- "u2_2.vdev",
2352024-10-09T23:13:39.262Z- "u2_3.vdev",
2362024-10-09T23:13:39.262Z- "u2_4.vdev",
2372024-10-09T23:13:39.262Z- "u2_5.vdev",
2382024-10-09T23:13:39.262Z- "u2_6.vdev",
2392024-10-09T23:13:39.262Z- "u2_7.vdev",
2402024-10-09T23:13:39.262Z- "u2_8.vdev",
2412024-10-09T23:13:39.262Z+ "/scratch/u2_0.vdev",
2422024-10-09T23:13:39.262Z+ "/scratch/u2_1.vdev",
2432024-10-09T23:13:39.262Z+ "/scratch/u2_2.vdev",
2442024-10-09T23:13:39.262Z+ "/scratch/u2_3.vdev",
2452024-10-09T23:13:39.262Z+ "/scratch/u2_4.vdev",
2462024-10-09T23:13:39.262Z+ "/scratch/u2_5.vdev",
2472024-10-09T23:13:39.262Z+ "/scratch/u2_6.vdev",
2482024-10-09T23:13:39.262Z+ "/scratch/u2_7.vdev",
2492024-10-09T23:13:39.262Z+ "/scratch/u2_8.vdev",
2502024-10-09T23:13:39.262Z ]
2512024-10-09T23:13:39.262Z
2522024-10-09T23:13:39.262Z # Percentage of usable physical DRAM to use for the VMM reservoir, which
2532024-10-09T23:13:39.262Z+ true
2542024-10-09T23:13:39.262Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
2552024-10-09T23:13:39.262Z+ SILO_NAME=recovery
2562024-10-09T23:13:39.262Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
2572024-10-09T23:13:39.262Z+ EXTERNAL_DNS_DOMAIN=oxide.test
2582024-10-09T23:13:39.262Z+ sed -i~ '
2592024-10-09T23:13:39.262Z /^external_dns_ips/c\
2602024-10-09T23:13:39.262Zexternal_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
2612024-10-09T23:13:39.262Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
2622024-10-09T23:13:39.263Z /^first/c\
2632024-10-09T23:13:39.263Zfirst = "10.151.1.164"
2642024-10-09T23:13:39.263Z /^last/c\
2652024-10-09T23:13:39.263Zlast = "10.151.1.173"
2662024-10-09T23:13:39.263Z }
2672024-10-09T23:13:39.263Z /^infra_ip_first/c\
2682024-10-09T23:13:39.263Zinfra_ip_first = "10.151.1.174"
2692024-10-09T23:13:39.263Z /^infra_ip_last/c\
2702024-10-09T23:13:39.263Zinfra_ip_last = "10.151.1.174"
2712024-10-09T23:13:39.263Z /^\[\[rack_network_config.ports/,/^$/ {
2722024-10-09T23:13:39.263Z /^routes/c\
2732024-10-09T23:13:39.263Zroutes = \[{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}\]
2742024-10-09T23:13:39.263Z /^addresses/c\
2752024-10-09T23:13:39.263Zaddresses = \[{address = "10.151.1.174/24"} \]
2762024-10-09T23:13:39.263Z }
2772024-10-09T23:13:39.263Z' pkg/config-rss.toml
2782024-10-09T23:13:39.263Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
2792024-10-09T23:13:39.263Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
2802024-10-09T23:13:39.263Z+++ pkg/config-rss.toml Wed Oct 9 23:13:39 2024
2812024-10-09T23:13:39.263Z@@ -33,7 +33,7 @@
2822024-10-09T23:13:39.263Z # the DNS domain delegated to the rack by the customer. Each of these addresses
2832024-10-09T23:13:39.263Z # must be contained in one of the "internal services" IP Pool ranges listed
2842024-10-09T23:13:39.263Z # below.
2852024-10-09T23:13:39.263Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
2862024-10-09T23:13:39.263Z+external_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
2872024-10-09T23:13:39.263Z
2882024-10-09T23:13:39.263Z # Initial TLS certificates for the external API
2892024-10-09T23:13:39.263Z #
2902024-10-09T23:13:39.263Z@@ -69,8 +69,8 @@
2912024-10-09T23:13:39.263Z #
2922024-10-09T23:13:39.263Z # For more on this and what to put here, see docs/how-to-run.adoc.
2932024-10-09T23:13:39.263Z [[internal_services_ip_pool_ranges]]
2942024-10-09T23:13:39.263Z-first = "192.168.1.20"
2952024-10-09T23:13:39.263Z-last = "192.168.1.29"
2962024-10-09T23:13:39.263Z+first = "10.151.1.164"
2972024-10-09T23:13:39.263Z+last = "10.151.1.173"
2982024-10-09T23:13:39.263Z
2992024-10-09T23:13:39.263Z # TODO - this configuration is subject to change going forward. Ultimately these
3002024-10-09T23:13:39.264Z # parameters should be provided to the control plane via wicket, but we need to
3012024-10-09T23:13:39.264Z@@ -91,8 +91,8 @@
3022024-10-09T23:13:39.264Z # A range of IP addresses used by Boundary Services on the external network. In
3032024-10-09T23:13:39.264Z # a real system, these would be addresses of the uplink ports on the Sidecar.
3042024-10-09T23:13:39.264Z # With softnpu, only one address is used.
3052024-10-09T23:13:39.264Z-infra_ip_first = "192.168.1.30"
3062024-10-09T23:13:39.264Z-infra_ip_last = "192.168.1.30"
3072024-10-09T23:13:39.264Z+infra_ip_first = "10.151.1.174"
3082024-10-09T23:13:39.264Z+infra_ip_last = "10.151.1.174"
3092024-10-09T23:13:39.264Z
3102024-10-09T23:13:39.264Z # Configurations for BGP routers to run on the scrimlets.
3112024-10-09T23:13:39.264Z bgp = []
3122024-10-09T23:13:39.264Z@@ -100,9 +100,9 @@
3132024-10-09T23:13:39.264Z # You can configure multiple uplinks by repeating the following stanza
3142024-10-09T23:13:39.264Z [[rack_network_config.ports]]
3152024-10-09T23:13:39.264Z # Routes associated with this port.
3162024-10-09T23:13:39.264Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
3172024-10-09T23:13:39.264Z+routes = [{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}]
3182024-10-09T23:13:39.264Z # Addresses associated with this port.
3192024-10-09T23:13:39.288Z-addresses = [{address = "192.168.1.30/24"}]
3202024-10-09T23:13:39.288Z+addresses = [{address = "10.151.1.174/24"} ]
3212024-10-09T23:13:39.288Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
3222024-10-09T23:13:39.288Z port = "qsfp0"
3232024-10-09T23:13:39.289Z # The speed of this port.
3242024-10-09T23:13:39.289Z+ true
3252024-10-09T23:13:39.289Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml pkg/config.toml
3262024-10-09T23:13:39.289Za pkg/config-rss.toml 8K
3272024-10-09T23:13:39.289Za pkg/config.toml 4K
3282024-10-09T23:13:39.289Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~ pkg/config.toml pkg/config.toml~
3292024-10-09T23:13:39.289Z+ TLS_NAME=recovery.sys.oxide.test
3302024-10-09T23:13:39.289Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
3312024-10-09T23:13:39.792Z......+.+.........+..+...+......+.+.....+.+...+..+.........+..........+......+...+..+.........+.+.....+.............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+.........+......+.........+..+.......+...+........+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+.........+.+...+.....+...+...................+.....+.+.....+...+....+..................+...+...........+.+.........+.....+..................+.......+........+......+......+.+..+.+..+....+...+..+.+..+......+...............+....+..+......+....+.....+.........+....+...+..+......+...............+......+.+..+.......+..................+...+........+....+...............+....................+.......+.......................+.........................+...+...+..+.........+....+...........+.........+..........+...............+......+.........+.....+.......+.....+...+......+.........+................+........+......+....+...+.........+..............+.......+...+......+.....+......+.........+....+..+.............+..+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3322024-10-09T23:13:40.621Z...+..+....+.........+..+..........+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+.........+.........+..+....+......+...+..+.....................+...+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+.....+............+.....................+.......+..+.......+.........+.........+........+.............+..+.......+...+.....+.......+...+..+......+.......+........+.+...............+...+........+...+.........+..........+.....+.+.....+................+........+.+............+..+.....................+.......+...+..+..........+...+......+...+...................................+.......+.........+.....+............+...............+...+..........+.........+..+.........+.......+.........+......+........+......+.+..+.............+.....+.........+......+.......+..+....+..............+................+.....................+........+......+....+..+.+..+............+............................+...........+....+.....+.......+.....+.+.........+........+.+..+....+...........+.......+..+................+..+.......+...........+....+...+...+...+.....+.+.....+....+......+........+.+.........+.....................+.....+...................+..+...................+..+...+.+........+......+.......+....................+.......+..+....+..........................+.........+....+...............+..+.........+.......+..+.+............+........+.+..............+.......+..+...+.........................+........+......+.........+.......+...+......+...........+.......+...+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3332024-10-09T23:13:40.645Z-----
3342024-10-09T23:13:40.669Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
3352024-10-09T23:13:40.669Za pkg/initial-tls-cert.pem 2K
3362024-10-09T23:13:40.669Za pkg/initial-tls-key.pem 4K
3372024-10-09T23:13:40.669Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
3382024-10-09T23:13:40.669Z+ rmdir pkg
3392024-10-09T23:13:40.669Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
3402024-10-09T23:13:40.669Z+ pfexec mkdir -p /usr/oxide
3412024-10-09T23:13:40.669Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0
3422024-10-09T23:13:41.287Z+ pfexec chmod +x /usr/oxide/oxide
3432024-10-09T23:13:41.313Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
3442024-10-09T23:13:59.599Z+ OMICRON_NO_UNINSTALL=1
3452024-10-09T23:13:59.599Z+ ptime -m pfexec ./target/release/omicron-package -t test install
3462024-10-09T23:13:59.623ZLogging to: /opt/oxide/work/out/LOG
3472024-10-09T23:14:00.167Z
3482024-10-09T23:14:00.192Zreal 0.563775211
3492024-10-09T23:14:00.192Zuser 0.133043262
3502024-10-09T23:14:00.192Zsys 2.817161818
3512024-10-09T23:14:00.192Ztrap 0.000002079
3522024-10-09T23:14:00.192Ztflt 0.000014989
3532024-10-09T23:14:00.192Zdflt 0.000419350
3542024-10-09T23:14:00.192Zkflt 0.000000000
3552024-10-09T23:14:00.192Zlock 50.703713878
3562024-10-09T23:14:00.192Zslp 0.748737467
3572024-10-09T23:14:00.192Zlat 0.003048702
3582024-10-09T23:14:00.192Zstop 0.209293644
3592024-10-09T23:14:00.192Z+ retry=0
3602024-10-09T23:14:00.192Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3612024-10-09T23:14:00.192Z+ [[ 0 -gt 30 ]]
3622024-10-09T23:14:00.192Z+ sleep 1
3632024-10-09T23:14:01.182Z+ retry=1
3642024-10-09T23:14:01.182Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3652024-10-09T23:14:01.207Z+ [[ 1 -gt 30 ]]
3662024-10-09T23:14:01.207Z+ sleep 1
3672024-10-09T23:14:02.197Z+ retry=2
3682024-10-09T23:14:02.197Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3692024-10-09T23:14:02.222Z+ [[ 2 -gt 30 ]]
3702024-10-09T23:14:02.222Z+ sleep 1
3712024-10-09T23:14:03.211Z+ retry=3
3722024-10-09T23:14:03.212Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3732024-10-09T23:14:23.180Z+ [[ 3 -gt 30 ]]
3742024-10-09T23:14:23.181Z+ sleep 1
3752024-10-09T23:14:24.184Z+ retry=4
3762024-10-09T23:14:24.184Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3772024-10-09T23:14:24.208Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
3782024-10-09T23:14:24.233Zlocal v6:
3792024-10-09T23:14:24.233Zfe80::889e:d5ff:feb6:f42a
3802024-10-09T23:14:24.233Zfd01:d005:1fd4:9229::1
3812024-10-09T23:14:24.233Zlocal v4:
3822024-10-09T23:14:24.233Z10.151.1.174
3832024-10-09T23:14:24.233Zrouter v6:
3842024-10-09T23:14:24.233Zrouter v4_idx:
3852024-10-09T23:14:24.233Zrouter v4_routes:
3862024-10-09T23:14:24.233Zresolver v4:
3872024-10-09T23:14:24.233Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
3882024-10-09T23:14:24.233Zresolver v6:
3892024-10-09T23:14:24.233Zfe80::f46a:93ff:fe0b:cc4e -> f6:6a:93:0b:cc:4e
3902024-10-09T23:14:24.233Zfe80::889e:d5ff:feb6:f42a -> 8a:9e:d5:b6:f4:2a
3912024-10-09T23:14:24.234Znat v4:
3922024-10-09T23:14:24.234Znat_v6:
3932024-10-09T23:14:24.234Zport_mac:
3942024-10-09T23:14:24.234Z1000: a8:40:25:04:15:ac
3952024-10-09T23:14:24.234Z1: a8:40:25:04:15:ad
3962024-10-09T23:14:24.234Z2: a8:40:25:04:15:cd
3972024-10-09T23:14:24.234Zproxy arp:
3982024-10-09T23:14:24.234Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
3992024-10-09T23:14:24.234Z+ retry=0
4002024-10-09T23:14:24.234Z+++ zoneadm list -n
4012024-10-09T23:14:24.234Z+++ grep oxz_ntp
4022024-10-09T23:14:24.258Z++ pfexec svcs -z -Hostate oxide/ntp
4032024-10-09T23:14:24.258Zsvcs: zone '-Hostate': entity not found
4042024-10-09T23:14:24.258Z++ true
4052024-10-09T23:14:24.258Z+ [[ '' != online ]]
4062024-10-09T23:14:24.258Z+ [[ 0 -gt 60 ]]
4072024-10-09T23:14:24.258Z+ sleep 1
4082024-10-09T23:14:25.249Z+ retry=1
4092024-10-09T23:14:25.273Z+++ zoneadm list -n
4102024-10-09T23:14:25.273Z+++ grep oxz_ntp
4112024-10-09T23:14:25.273Z++ pfexec svcs -z -Hostate oxide/ntp
4122024-10-09T23:14:25.274Zsvcs: zone '-Hostate': entity not found
4132024-10-09T23:14:25.274Z++ true
4142024-10-09T23:14:25.274Z+ [[ '' != online ]]
4152024-10-09T23:14:25.274Z+ [[ 1 -gt 60 ]]
4162024-10-09T23:14:25.274Z+ sleep 1
4172024-10-09T23:14:26.273Z+ retry=2
4182024-10-09T23:14:26.297Z+++ zoneadm list -n
4192024-10-09T23:14:26.297Z+++ grep oxz_ntp
4202024-10-09T23:14:26.297Z++ pfexec svcs -z oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3 -Hostate oxide/ntp
4212024-10-09T23:14:26.346Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
4222024-10-09T23:14:26.370Z++ true
4232024-10-09T23:14:26.370Z+ [[ '' != online ]]
4242024-10-09T23:14:26.370Z+ [[ 2 -gt 60 ]]
4252024-10-09T23:14:26.370Z+ sleep 1
4262024-10-09T23:14:27.352Z+ retry=3
4272024-10-09T23:14:27.376Z+++ zoneadm list -n
4282024-10-09T23:14:27.376Z+++ grep oxz_ntp
4292024-10-09T23:14:27.376Z++ pfexec svcs -z oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3 -Hostate oxide/ntp
4302024-10-09T23:14:27.400Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
4312024-10-09T23:14:27.400Z++ true
4322024-10-09T23:14:27.400Z+ [[ '' != online ]]
4332024-10-09T23:14:27.400Z+ [[ 3 -gt 60 ]]
4342024-10-09T23:14:27.401Z+ sleep 1
4352024-10-09T23:14:28.383Z+ retry=4
4362024-10-09T23:14:28.407Z+++ zoneadm list -n
4372024-10-09T23:14:28.407Z+++ grep oxz_ntp
4382024-10-09T23:14:28.407Z++ pfexec svcs -z oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3 -Hostate oxide/ntp
4392024-10-09T23:14:28.431Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
4402024-10-09T23:14:28.431Z++ true
4412024-10-09T23:14:28.431Z+ [[ '' != online ]]
4422024-10-09T23:14:28.431Z+ [[ 4 -gt 60 ]]
4432024-10-09T23:14:28.431Z+ sleep 1
4442024-10-09T23:14:29.413Z+ retry=5
4452024-10-09T23:14:29.437Z+++ zoneadm list -n
4462024-10-09T23:14:29.437Z+++ grep oxz_ntp
4472024-10-09T23:14:29.437Z++ pfexec svcs -z oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3 -Hostate oxide/ntp
4482024-10-09T23:14:29.461Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
4492024-10-09T23:14:29.461Z++ true
4502024-10-09T23:14:29.461Z+ [[ '' != online ]]
4512024-10-09T23:14:29.461Z+ [[ 5 -gt 60 ]]
4522024-10-09T23:14:29.461Z+ sleep 1
4532024-10-09T23:14:30.442Z+ retry=6
4542024-10-09T23:14:30.466Z+++ zoneadm list -n
4552024-10-09T23:14:30.466Z+++ grep oxz_ntp
4562024-10-09T23:14:30.466Z++ pfexec svcs -z oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3 -Hostate oxide/ntp
4572024-10-09T23:14:30.466Z+ [[ offline != online ]]
4582024-10-09T23:14:30.466Z+ [[ 6 -gt 60 ]]
4592024-10-09T23:14:30.466Z+ sleep 1
4602024-10-09T23:14:31.463Z+ retry=7
4612024-10-09T23:14:31.489Z+++ zoneadm list -n
4622024-10-09T23:14:31.489Z+++ grep oxz_ntp
4632024-10-09T23:14:31.489Z++ pfexec svcs -z oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3 -Hostate oxide/ntp
4642024-10-09T23:14:31.489Z+ [[ offline != online ]]
4652024-10-09T23:14:31.489Z+ [[ 7 -gt 60 ]]
4662024-10-09T23:14:31.489Z+ sleep 1
4672024-10-09T23:14:32.485Z+ retry=8
4682024-10-09T23:14:32.509Z+++ zoneadm list -n
4692024-10-09T23:14:32.509Z+++ grep oxz_ntp
4702024-10-09T23:14:32.509Z++ pfexec svcs -z oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3 -Hostate oxide/ntp
4712024-10-09T23:14:32.509Z+ [[ offline != online ]]
4722024-10-09T23:14:32.509Z+ [[ 8 -gt 60 ]]
4732024-10-09T23:14:32.509Z+ sleep 1
4742024-10-09T23:14:33.506Z+ retry=9
4752024-10-09T23:14:33.531Z+++ zoneadm list -n
4762024-10-09T23:14:33.531Z+++ grep oxz_ntp
4772024-10-09T23:14:33.531Z++ pfexec svcs -z oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3 -Hostate oxide/ntp
4782024-10-09T23:14:33.531Z+ [[ offline != online ]]
4792024-10-09T23:14:33.531Z+ [[ 9 -gt 60 ]]
4802024-10-09T23:14:33.531Z+ sleep 1
4812024-10-09T23:14:34.528Z+ retry=10
4822024-10-09T23:14:34.552Z+++ zoneadm list -n
4832024-10-09T23:14:34.553Z+++ grep oxz_ntp
4842024-10-09T23:14:34.553Z++ pfexec svcs -z oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3 -Hostate oxide/ntp
4852024-10-09T23:14:34.553Z+ [[ offline != online ]]
4862024-10-09T23:14:34.553Z+ [[ 10 -gt 60 ]]
4872024-10-09T23:14:34.553Z+ sleep 1
4882024-10-09T23:14:35.549Z+ retry=11
4892024-10-09T23:14:35.574Z+++ zoneadm list -n
4902024-10-09T23:14:35.574Z+++ grep oxz_ntp
4912024-10-09T23:14:35.574Z++ pfexec svcs -z oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3 -Hostate oxide/ntp
4922024-10-09T23:14:35.574Z+ [[ offline != online ]]
4932024-10-09T23:14:35.574Z+ [[ 11 -gt 60 ]]
4942024-10-09T23:14:35.574Z+ sleep 1
 
4952024-10-09T23:14:36.235Zbuskin console login: Oct 9 23:14:35 buskin ip: ip: joining multicasts failed (4) on opte0 - will use link layer broadcasts for multicast
 
4962024-10-09T23:14:36.571Z+ retry=12
4972024-10-09T23:14:36.596Z+++ zoneadm list -n
4982024-10-09T23:14:36.596Z+++ grep oxz_ntp
4992024-10-09T23:14:36.596Z++ pfexec svcs -z oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3 -Hostate oxide/ntp
5002024-10-09T23:14:36.596Z+ [[ online != online ]]
5012024-10-09T23:14:36.596Z+ echo 'Waited for chrony: 12s'
5022024-10-09T23:14:36.596ZWaited for chrony: 12s
5032024-10-09T23:14:36.596Z+ retry=0
5042024-10-09T23:14:36.596Z+ zoneadm list
5052024-10-09T23:14:36.596Z+ grep nexus
5062024-10-09T23:14:36.620Z+ [[ 0 -gt 300 ]]
5072024-10-09T23:14:36.620Z+ sleep 1
5082024-10-09T23:14:37.600Z+ retry=1
5092024-10-09T23:14:37.625Z+ zoneadm list
5102024-10-09T23:14:37.625Z+ grep nexus
5112024-10-09T23:14:37.625Z+ [[ 1 -gt 300 ]]
5122024-10-09T23:14:37.625Z+ sleep 1
5132024-10-09T23:14:38.610Z+ retry=2
5142024-10-09T23:14:38.635Z+ zoneadm list
5152024-10-09T23:14:38.635Z+ grep nexus
5162024-10-09T23:14:38.635Z+ [[ 2 -gt 300 ]]
5172024-10-09T23:14:38.635Z+ sleep 1
5182024-10-09T23:14:39.623Z+ retry=3
5192024-10-09T23:14:39.647Z+ zoneadm list
5202024-10-09T23:14:39.647Z+ grep nexus
5212024-10-09T23:14:39.647Z+ [[ 3 -gt 300 ]]
5222024-10-09T23:14:39.647Z+ sleep 1
5232024-10-09T23:14:40.636Z+ retry=4
5242024-10-09T23:14:40.660Z+ zoneadm list
5252024-10-09T23:14:40.660Z+ grep nexus
5262024-10-09T23:14:40.660Z+ [[ 4 -gt 300 ]]
5272024-10-09T23:14:40.660Z+ sleep 1
5282024-10-09T23:14:41.649Z+ retry=5
5292024-10-09T23:14:41.673Z+ zoneadm list
5302024-10-09T23:14:41.673Z+ grep nexus
5312024-10-09T23:14:41.673Z+ [[ 5 -gt 300 ]]
5322024-10-09T23:14:41.673Z+ sleep 1
5332024-10-09T23:14:42.663Z+ retry=6
5342024-10-09T23:14:42.713Z+ zoneadm list
5352024-10-09T23:14:42.713Z+ grep nexus
5362024-10-09T23:14:42.713Z+ [[ 6 -gt 300 ]]
5372024-10-09T23:14:42.713Z+ sleep 1
5382024-10-09T23:14:43.676Z+ retry=7
5392024-10-09T23:14:43.699Z+ zoneadm list
5402024-10-09T23:14:43.699Z+ grep nexus
5412024-10-09T23:14:43.699Z+ [[ 7 -gt 300 ]]
5422024-10-09T23:14:43.699Z+ sleep 1
5432024-10-09T23:14:44.691Z+ retry=8
5442024-10-09T23:14:44.714Z+ zoneadm list
5452024-10-09T23:14:44.714Z+ grep nexus
5462024-10-09T23:14:44.714Z+ [[ 8 -gt 300 ]]
5472024-10-09T23:14:44.714Z+ sleep 1
5482024-10-09T23:14:45.706Z+ retry=9
5492024-10-09T23:14:45.729Z+ zoneadm list
5502024-10-09T23:14:45.729Z+ grep nexus
5512024-10-09T23:14:45.729Z+ [[ 9 -gt 300 ]]
5522024-10-09T23:14:45.729Z+ sleep 1
5532024-10-09T23:14:46.730Z+ retry=10
5542024-10-09T23:14:46.797Z+ zoneadm list
5552024-10-09T23:14:46.884Z+ grep nexus
5562024-10-09T23:14:47.001Z+ [[ 10 -gt 300 ]]
5572024-10-09T23:14:47.030Z+ sleep 1
5582024-10-09T23:14:48.651Z+ retry=11
5592024-10-09T23:14:48.685Z+ zoneadm list
5602024-10-09T23:14:48.712Z+ grep nexus
5612024-10-09T23:14:48.761Z+ [[ 11 -gt 300 ]]
5622024-10-09T23:14:48.761Z+ sleep 1
5632024-10-09T23:14:49.780Z+ retry=12
5642024-10-09T23:14:49.803Z+ zoneadm list
5652024-10-09T23:14:49.803Z+ grep nexus
5662024-10-09T23:14:49.803Z+ [[ 12 -gt 300 ]]
5672024-10-09T23:14:49.803Z+ sleep 1
5682024-10-09T23:14:50.794Z+ retry=13
5692024-10-09T23:14:50.817Z+ zoneadm list
5702024-10-09T23:14:50.817Z+ grep nexus
5712024-10-09T23:14:50.817Z+ [[ 13 -gt 300 ]]
5722024-10-09T23:14:50.817Z+ sleep 1
5732024-10-09T23:14:51.809Z+ retry=14
5742024-10-09T23:14:51.833Z+ zoneadm list
5752024-10-09T23:14:51.833Z+ grep nexus
5762024-10-09T23:14:51.833Z+ [[ 14 -gt 300 ]]
5772024-10-09T23:14:51.833Z+ sleep 1
5782024-10-09T23:14:52.824Z+ retry=15
5792024-10-09T23:14:52.847Z+ zoneadm list
5802024-10-09T23:14:52.847Z+ grep nexus
5812024-10-09T23:14:52.847Z+ [[ 15 -gt 300 ]]
5822024-10-09T23:14:52.847Z+ sleep 1
5832024-10-09T23:14:53.839Z+ retry=16
5842024-10-09T23:14:53.862Z+ zoneadm list
5852024-10-09T23:14:53.862Z+ grep nexus
5862024-10-09T23:14:53.862Z+ [[ 16 -gt 300 ]]
5872024-10-09T23:14:53.863Z+ sleep 1
5882024-10-09T23:14:54.854Z+ retry=17
5892024-10-09T23:14:54.877Z+ zoneadm list
5902024-10-09T23:14:54.877Z+ grep nexus
5912024-10-09T23:14:54.877Z+ [[ 17 -gt 300 ]]
5922024-10-09T23:14:54.877Z+ sleep 1
5932024-10-09T23:14:55.870Z+ retry=18
5942024-10-09T23:14:55.894Z+ zoneadm list
5952024-10-09T23:14:55.894Z+ grep nexus
5962024-10-09T23:14:55.894Z+ [[ 18 -gt 300 ]]
5972024-10-09T23:14:55.894Z+ sleep 1
5982024-10-09T23:14:56.885Z+ retry=19
5992024-10-09T23:14:56.908Z+ zoneadm list
6002024-10-09T23:14:56.908Z+ grep nexus
6012024-10-09T23:14:56.908Z+ [[ 19 -gt 300 ]]
6022024-10-09T23:14:56.908Z+ sleep 1
6032024-10-09T23:14:57.901Z+ retry=20
6042024-10-09T23:14:57.924Z+ zoneadm list
6052024-10-09T23:14:57.924Z+ grep nexus
6062024-10-09T23:14:57.924Z+ [[ 20 -gt 300 ]]
6072024-10-09T23:14:57.924Z+ sleep 1
6082024-10-09T23:14:58.917Z+ retry=21
6092024-10-09T23:14:58.940Z+ zoneadm list
6102024-10-09T23:14:58.940Z+ grep nexus
6112024-10-09T23:14:58.940Z+ [[ 21 -gt 300 ]]
6122024-10-09T23:14:58.940Z+ sleep 1
6132024-10-09T23:14:59.931Z+ retry=22
6142024-10-09T23:14:59.955Z+ zoneadm list
6152024-10-09T23:14:59.955Z+ grep nexus
6162024-10-09T23:14:59.955Z+ [[ 22 -gt 300 ]]
6172024-10-09T23:14:59.955Z+ sleep 1
6182024-10-09T23:15:00.948Z+ retry=23
6192024-10-09T23:15:00.972Z+ zoneadm list
6202024-10-09T23:15:00.972Z+ grep nexus
6212024-10-09T23:15:00.972Z+ [[ 23 -gt 300 ]]
6222024-10-09T23:15:00.972Z+ sleep 1
6232024-10-09T23:15:01.962Z+ retry=24
6242024-10-09T23:15:01.985Z+ zoneadm list
6252024-10-09T23:15:01.985Z+ grep nexus
6262024-10-09T23:15:01.985Z+ [[ 24 -gt 300 ]]
6272024-10-09T23:15:01.985Z+ sleep 1
6282024-10-09T23:15:02.974Z+ retry=25
6292024-10-09T23:15:02.997Z+ zoneadm list
6302024-10-09T23:15:02.997Z+ grep nexus
6312024-10-09T23:15:02.997Z+ [[ 25 -gt 300 ]]
6322024-10-09T23:15:02.997Z+ sleep 1
6332024-10-09T23:15:04.027Z+ retry=26
6342024-10-09T23:15:04.027Z+ zoneadm list
6352024-10-09T23:15:04.051Z+ grep nexus
6362024-10-09T23:15:04.051Z+ [[ 26 -gt 300 ]]
6372024-10-09T23:15:04.051Z+ sleep 1
6382024-10-09T23:15:05.067Z+ retry=27
6392024-10-09T23:15:05.173Z+ zoneadm list
6402024-10-09T23:15:05.200Z+ grep nexus
6412024-10-09T23:15:05.227Z+ [[ 27 -gt 300 ]]
6422024-10-09T23:15:05.227Z+ sleep 1
6432024-10-09T23:15:06.234Z+ retry=28
6442024-10-09T23:15:06.258Z+ zoneadm list
6452024-10-09T23:15:06.258Z+ grep nexus
6462024-10-09T23:15:06.258Z+ [[ 28 -gt 300 ]]
6472024-10-09T23:15:06.258Z+ sleep 1
6482024-10-09T23:15:07.252Z+ retry=29
6492024-10-09T23:15:07.275Z+ zoneadm list
6502024-10-09T23:15:07.275Z+ grep nexus
6512024-10-09T23:15:07.275Z+ [[ 29 -gt 300 ]]
6522024-10-09T23:15:07.275Z+ sleep 1
6532024-10-09T23:15:08.264Z+ retry=30
6542024-10-09T23:15:08.288Z+ zoneadm list
6552024-10-09T23:15:08.288Z+ grep nexus
6562024-10-09T23:15:08.288Z+ [[ 30 -gt 300 ]]
6572024-10-09T23:15:08.288Z+ sleep 1
6582024-10-09T23:15:09.276Z+ retry=31
6592024-10-09T23:15:09.299Z+ zoneadm list
6602024-10-09T23:15:09.299Z+ grep nexus
6612024-10-09T23:15:09.299Z+ [[ 31 -gt 300 ]]
6622024-10-09T23:15:09.299Z+ sleep 1
6632024-10-09T23:15:10.289Z+ retry=32
6642024-10-09T23:15:10.312Z+ zoneadm list
6652024-10-09T23:15:10.312Z+ grep nexus
6662024-10-09T23:15:10.312Z+ [[ 32 -gt 300 ]]
6672024-10-09T23:15:10.312Z+ sleep 1
6682024-10-09T23:15:11.304Z+ retry=33
6692024-10-09T23:15:11.327Z+ zoneadm list
6702024-10-09T23:15:11.328Z+ grep nexus
6712024-10-09T23:15:11.328Z+ [[ 33 -gt 300 ]]
6722024-10-09T23:15:11.328Z+ sleep 1
6732024-10-09T23:15:12.317Z+ retry=34
6742024-10-09T23:15:12.341Z+ zoneadm list
6752024-10-09T23:15:12.341Z+ grep nexus
6762024-10-09T23:15:12.341Z+ [[ 34 -gt 300 ]]
6772024-10-09T23:15:12.341Z+ sleep 1
6782024-10-09T23:15:13.330Z+ retry=35
6792024-10-09T23:15:13.353Z+ zoneadm list
6802024-10-09T23:15:13.353Z+ grep nexus
6812024-10-09T23:15:13.353Z+ [[ 35 -gt 300 ]]
6822024-10-09T23:15:13.353Z+ sleep 1
6832024-10-09T23:15:14.343Z+ retry=36
6842024-10-09T23:15:14.366Z+ zoneadm list
6852024-10-09T23:15:14.366Z+ grep nexus
6862024-10-09T23:15:14.366Z+ [[ 36 -gt 300 ]]
6872024-10-09T23:15:14.366Z+ sleep 1
6882024-10-09T23:15:15.356Z+ retry=37
6892024-10-09T23:15:15.379Z+ zoneadm list
6902024-10-09T23:15:15.379Z+ grep nexus
6912024-10-09T23:15:15.379Z+ [[ 37 -gt 300 ]]
6922024-10-09T23:15:15.379Z+ sleep 1
6932024-10-09T23:15:16.370Z+ retry=38
6942024-10-09T23:15:16.393Z+ zoneadm list
6952024-10-09T23:15:16.393Z+ grep nexus
6962024-10-09T23:15:16.393Z+ [[ 38 -gt 300 ]]
6972024-10-09T23:15:16.393Z+ sleep 1
6982024-10-09T23:15:17.383Z+ retry=39
6992024-10-09T23:15:17.405Z+ zoneadm list
7002024-10-09T23:15:17.406Z+ grep nexus
7012024-10-09T23:15:17.406Z+ [[ 39 -gt 300 ]]
7022024-10-09T23:15:17.406Z+ sleep 1
7032024-10-09T23:15:18.394Z+ retry=40
7042024-10-09T23:15:18.417Z+ zoneadm list
7052024-10-09T23:15:18.417Z+ grep nexus
7062024-10-09T23:15:18.417Z+ [[ 40 -gt 300 ]]
7072024-10-09T23:15:18.417Z+ sleep 1
7082024-10-09T23:15:19.406Z+ retry=41
7092024-10-09T23:15:19.429Z+ zoneadm list
7102024-10-09T23:15:19.429Z+ grep nexus
7112024-10-09T23:15:19.429Z+ [[ 41 -gt 300 ]]
7122024-10-09T23:15:19.429Z+ sleep 1
7132024-10-09T23:15:20.418Z+ retry=42
7142024-10-09T23:15:20.441Z+ zoneadm list
7152024-10-09T23:15:20.441Z+ grep nexus
7162024-10-09T23:15:20.441Z+ [[ 42 -gt 300 ]]
7172024-10-09T23:15:20.441Z+ sleep 1
7182024-10-09T23:15:21.429Z+ retry=43
7192024-10-09T23:15:21.452Z+ zoneadm list
7202024-10-09T23:15:21.452Z+ grep nexus
7212024-10-09T23:15:21.452Z+ [[ 43 -gt 300 ]]
7222024-10-09T23:15:21.452Z+ sleep 1
7232024-10-09T23:15:22.441Z+ retry=44
7242024-10-09T23:15:22.464Z+ zoneadm list
7252024-10-09T23:15:22.464Z+ grep nexus
7262024-10-09T23:15:22.464Z+ [[ 44 -gt 300 ]]
7272024-10-09T23:15:22.464Z+ sleep 1
7282024-10-09T23:15:23.453Z+ retry=45
7292024-10-09T23:15:23.476Z+ zoneadm list
7302024-10-09T23:15:23.476Z+ grep nexus
7312024-10-09T23:15:23.476Z+ [[ 45 -gt 300 ]]
7322024-10-09T23:15:23.476Z+ sleep 1
7332024-10-09T23:15:24.465Z+ retry=46
7342024-10-09T23:15:24.488Z+ zoneadm list
7352024-10-09T23:15:24.488Z+ grep nexus
7362024-10-09T23:15:24.488Z+ [[ 46 -gt 300 ]]
7372024-10-09T23:15:24.488Z+ sleep 1
7382024-10-09T23:15:25.478Z+ retry=47
7392024-10-09T23:15:25.502Z+ zoneadm list
7402024-10-09T23:15:25.502Z+ grep nexus
7412024-10-09T23:15:25.502Z+ [[ 47 -gt 300 ]]
7422024-10-09T23:15:25.502Z+ sleep 1
7432024-10-09T23:15:26.490Z+ retry=48
7442024-10-09T23:15:26.513Z+ zoneadm list
7452024-10-09T23:15:26.513Z+ grep nexus
7462024-10-09T23:15:26.513Z+ [[ 48 -gt 300 ]]
7472024-10-09T23:15:26.513Z+ sleep 1
7482024-10-09T23:15:27.502Z+ retry=49
7492024-10-09T23:15:27.525Z+ zoneadm list
7502024-10-09T23:15:27.525Z+ grep nexus
7512024-10-09T23:15:27.525Z+ [[ 49 -gt 300 ]]
7522024-10-09T23:15:27.525Z+ sleep 1
7532024-10-09T23:15:28.515Z+ retry=50
7542024-10-09T23:15:28.538Z+ zoneadm list
7552024-10-09T23:15:28.539Z+ grep nexus
7562024-10-09T23:15:28.539Z+ [[ 50 -gt 300 ]]
7572024-10-09T23:15:28.539Z+ sleep 1
7582024-10-09T23:15:29.526Z+ retry=51
7592024-10-09T23:15:29.549Z+ zoneadm list
7602024-10-09T23:15:29.549Z+ grep nexus
7612024-10-09T23:15:29.549Z+ [[ 51 -gt 300 ]]
7622024-10-09T23:15:29.549Z+ sleep 1
7632024-10-09T23:15:30.547Z+ retry=52
7642024-10-09T23:15:30.571Z+ zoneadm list
7652024-10-09T23:15:30.571Z+ grep nexus
7662024-10-09T23:15:30.594Z+ [[ 52 -gt 300 ]]
7672024-10-09T23:15:30.594Z+ sleep 1
7682024-10-09T23:15:31.579Z+ retry=53
7692024-10-09T23:15:31.603Z+ zoneadm list
7702024-10-09T23:15:31.603Z+ grep nexus
7712024-10-09T23:15:31.603Z+ [[ 53 -gt 300 ]]
7722024-10-09T23:15:31.603Z+ sleep 1
7732024-10-09T23:15:32.596Z+ retry=54
7742024-10-09T23:15:32.619Z+ zoneadm list
7752024-10-09T23:15:32.619Z+ grep nexus
7762024-10-09T23:15:32.642Z+ [[ 54 -gt 300 ]]
7772024-10-09T23:15:32.642Z+ sleep 1
7782024-10-09T23:15:34.486Z+ retry=55
7792024-10-09T23:15:34.518Z+ zoneadm list
7802024-10-09T23:15:34.518Z+ grep nexus
7812024-10-09T23:15:34.571Z+ [[ 55 -gt 300 ]]
7822024-10-09T23:15:34.571Z+ sleep 1
7832024-10-09T23:15:35.577Z+ retry=56
7842024-10-09T23:15:35.601Z+ zoneadm list
7852024-10-09T23:15:35.601Z+ grep nexus
7862024-10-09T23:15:35.601Z+ [[ 56 -gt 300 ]]
7872024-10-09T23:15:35.601Z+ sleep 1
7882024-10-09T23:15:36.600Z+ retry=57
7892024-10-09T23:15:36.624Z+ zoneadm list
7902024-10-09T23:15:36.624Z+ grep nexus
7912024-10-09T23:15:36.624Z+ [[ 57 -gt 300 ]]
7922024-10-09T23:15:36.624Z+ sleep 1
7932024-10-09T23:15:37.620Z+ retry=58
7942024-10-09T23:15:37.643Z+ zoneadm list
7952024-10-09T23:15:37.643Z+ grep nexus
7962024-10-09T23:15:37.643Zoxz_nexus_780e1728-4013-4877-b14f-85b67afdf49d
7972024-10-09T23:15:37.643Zoxz_nexus_efde9fa7-2c58-47bd-9c83-7e8e861f5541
7982024-10-09T23:15:37.643Z+ echo 'Waited for nexus: 58s'
7992024-10-09T23:15:37.643ZWaited for nexus: 58s
8002024-10-09T23:15:37.643Z+ export RUST_BACKTRACE=1
8012024-10-09T23:15:37.643Z+ RUST_BACKTRACE=1
8022024-10-09T23:15:37.643Z+ export E2E_TLS_CERT IPPOOL_START IPPOOL_END
8032024-10-09T23:15:37.643Z++ ./target/debug/bootstrap
8042024-10-09T23:15:38.667ZUsing external DNS server at 10.151.1.164:53
8052024-10-09T23:15:38.690Z2024-10-09 23:15:38.672383458 UTC: attempting to log into API
 
8062024-10-09T23:15:44.092ZOct 9 23:15:43 buskin xde: WARNING: failed to generate descriptor for stateful action: ICMP:172.30.3.5:0:134.215.155.177:0 Unexpected { msg: "Expected ICMP Echo Request, found: destination unreachable" }
 
8072024-10-09T23:15:53.882Z2024-10-09 23:15:53.869796629 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local)
8082024-10-09T23:15:54.884Z2024-10-09 23:15:54.871181464 UTC: attempting to log into API
 
8092024-10-09T23:16:09.623ZOct 9 23:15:47 buskin last message repeated 1 time
8102024-10-09T23:16:09.655ZOct 9 23:16:09 buskin ip: ip: joining multicasts failed (4) on opte4 - will use link layer broadcasts for multicast
8112024-10-09T23:16:09.689ZOct 9 23:16:09 buskin ip: ip: joining multicasts failed (4) on opte1 - will use link layer broadcasts for multicast
8122024-10-09T23:16:09.721ZOct 9 23:16:09 buskin ip: ip: joining multicasts failed (4) on opte5 - will use link layer broadcasts for multicast
 
8132024-10-09T23:16:09.929Z2024-10-09 23:16:09.917140199 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local)
 
8142024-10-09T23:16:10.652ZOct 9 23:16:10 buskin ip: ip: joining multicasts failed (4) on opte3 - will use link layer broadcasts for multicast
 
8152024-10-09T23:16:10.930Z2024-10-09 23:16:10.918251088 UTC: attempting to log into API
 
8162024-10-09T23:16:11.275ZOct 9 23:16:10 buskin ip: ip: joining multicasts failed (4) on opte2 - will use link layer broadcasts for multicast
 
8172024-10-09T23:16:25.963Z2024-10-09 23:16:25.951109745 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local)
8182024-10-09T23:16:26.965Z2024-10-09 23:16:26.952485950 UTC: attempting to log into API
 
8192024-10-09T23:16:41.939ZOct 9 23:16:41 buskin xde: WARNING: failed to generate descriptor for stateful action: ICMP:172.30.3.5:0:209.51.161.238:0 Unexpected { msg: "Expected ICMP Echo Request, found: destination unreachable" }
 
8202024-10-09T23:16:41.998Z2024-10-09 23:16:41.985775005 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local)
8212024-10-09T23:16:42.999Z2024-10-09 23:16:42.986963816 UTC: attempting to log into API
8222024-10-09T23:16:53.045Z2024-10-09 23:16:53.031808391 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local)
8232024-10-09T23:16:54.045Z2024-10-09 23:16:54.033114475 UTC: attempting to log into API
8242024-10-09T23:16:54.846Z2024-10-09 23:16:54.833542675 UTC: login succeeded
8252024-10-09T23:16:54.878Zwaiting for nexus to come up...
8262024-10-09T23:16:54.963Zcreating IP pool... 10.151.1.175 - 10.151.1.227
8272024-10-09T23:16:55.214Zsetting up quotas...
8282024-10-09T23:16:55.267Zensuring datasets are ready...
8292024-10-09T23:16:58.166Zlet's roll.
8302024-10-09T23:16:58.189Z+ eval 'OXIDE_HOST=https://recovery.sys.oxide.test
8312024-10-09T23:16:58.189ZOXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
8322024-10-09T23:16:58.189ZOXIDE_TOKEN=oxide-token-d384c0f54a83e6e2421c7d47fc4824c918d1c5e0'
8332024-10-09T23:16:58.189Z++ OXIDE_HOST=https://recovery.sys.oxide.test
8342024-10-09T23:16:58.189Z++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
8352024-10-09T23:16:58.189Z++ OXIDE_TOKEN=oxide-token-d384c0f54a83e6e2421c7d47fc4824c918d1c5e0
8362024-10-09T23:16:58.189Z+ export OXIDE_HOST OXIDE_TOKEN
8372024-10-09T23:16:58.189Z+ retry=0
8382024-10-09T23:16:58.189Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8392024-10-09T23:16:58.213Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8402024-10-09T23:16:58.213Z+ [[ 0 -gt 60 ]]
8412024-10-09T23:16:58.213Z+ sleep 1
8422024-10-09T23:16:59.198Z+ retry=1
8432024-10-09T23:16:59.221Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8442024-10-09T23:16:59.221Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8452024-10-09T23:16:59.222Z+ [[ 1 -gt 60 ]]
8462024-10-09T23:16:59.222Z+ sleep 1
8472024-10-09T23:17:00.213Z+ retry=2
8482024-10-09T23:17:00.213Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8492024-10-09T23:17:00.237Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8502024-10-09T23:17:00.237Z+ [[ 2 -gt 60 ]]
8512024-10-09T23:17:00.237Z+ sleep 1
8522024-10-09T23:17:01.230Z+ retry=3
8532024-10-09T23:17:01.230Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8542024-10-09T23:17:01.254Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8552024-10-09T23:17:01.254Z+ [[ 3 -gt 60 ]]
8562024-10-09T23:17:01.254Z+ sleep 1
8572024-10-09T23:17:02.245Z+ retry=4
8582024-10-09T23:17:02.245Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8592024-10-09T23:17:02.268Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8602024-10-09T23:17:02.268Z+ [[ 4 -gt 60 ]]
8612024-10-09T23:17:02.268Z+ sleep 1
8622024-10-09T23:17:03.260Z+ retry=5
8632024-10-09T23:17:03.260Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8642024-10-09T23:17:03.283Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8652024-10-09T23:17:03.283Z+ [[ 5 -gt 60 ]]
8662024-10-09T23:17:03.283Z+ sleep 1
8672024-10-09T23:17:04.311Z+ retry=6
8682024-10-09T23:17:04.312Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8692024-10-09T23:17:04.381Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8702024-10-09T23:17:04.381Z+ [[ 6 -gt 60 ]]
8712024-10-09T23:17:04.382Z+ sleep 1
8722024-10-09T23:17:05.291Z+ retry=7
8732024-10-09T23:17:05.291Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8742024-10-09T23:17:05.314Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8752024-10-09T23:17:05.314Z+ [[ 7 -gt 60 ]]
8762024-10-09T23:17:05.314Z+ sleep 1
8772024-10-09T23:17:06.306Z+ retry=8
8782024-10-09T23:17:06.306Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8792024-10-09T23:17:06.330Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8802024-10-09T23:17:06.330Z+ [[ 8 -gt 60 ]]
8812024-10-09T23:17:06.330Z+ sleep 1
8822024-10-09T23:17:07.322Z+ retry=9
8832024-10-09T23:17:07.322Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8842024-10-09T23:17:07.346Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8852024-10-09T23:17:07.346Z+ [[ 9 -gt 60 ]]
8862024-10-09T23:17:07.346Z+ sleep 1
8872024-10-09T23:17:08.338Z+ retry=10
8882024-10-09T23:17:08.338Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8892024-10-09T23:17:08.361Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8902024-10-09T23:17:08.361Z+ [[ 10 -gt 60 ]]
8912024-10-09T23:17:08.361Z+ sleep 1
8922024-10-09T23:17:09.354Z+ retry=11
8932024-10-09T23:17:09.354Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8942024-10-09T23:17:09.377Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8952024-10-09T23:17:09.377Z+ [[ 11 -gt 60 ]]
8962024-10-09T23:17:09.378Z+ sleep 1
8972024-10-09T23:17:10.370Z+ retry=12
8982024-10-09T23:17:10.370Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8992024-10-09T23:17:10.393Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9002024-10-09T23:17:10.393Z+ [[ 12 -gt 60 ]]
9012024-10-09T23:17:10.394Z+ sleep 1
9022024-10-09T23:17:11.385Z+ retry=13
9032024-10-09T23:17:11.385Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9042024-10-09T23:17:11.409Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9052024-10-09T23:17:11.409Z+ [[ 13 -gt 60 ]]
9062024-10-09T23:17:11.409Z+ sleep 1
9072024-10-09T23:17:12.401Z+ retry=14
9082024-10-09T23:17:12.401Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9092024-10-09T23:17:12.424Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9102024-10-09T23:17:12.424Z+ [[ 14 -gt 60 ]]
9112024-10-09T23:17:12.424Z+ sleep 1
9122024-10-09T23:17:13.417Z+ retry=15
9132024-10-09T23:17:13.417Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9142024-10-09T23:17:13.440Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9152024-10-09T23:17:13.441Z+ [[ 15 -gt 60 ]]
9162024-10-09T23:17:13.441Z+ sleep 1
9172024-10-09T23:17:14.434Z+ retry=16
9182024-10-09T23:17:14.434Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9192024-10-09T23:17:14.457Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9202024-10-09T23:17:14.457Z+ [[ 16 -gt 60 ]]
9212024-10-09T23:17:14.457Z+ sleep 1
9222024-10-09T23:17:15.447Z+ retry=17
9232024-10-09T23:17:15.447Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9242024-10-09T23:17:15.471Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9252024-10-09T23:17:15.471Z+ [[ 17 -gt 60 ]]
9262024-10-09T23:17:15.471Z+ sleep 1
9272024-10-09T23:17:16.464Z+ retry=18
9282024-10-09T23:17:16.464Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9292024-10-09T23:17:16.487Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9302024-10-09T23:17:16.488Z+ [[ 18 -gt 60 ]]
9312024-10-09T23:17:16.488Z+ sleep 1
9322024-10-09T23:17:17.480Z+ retry=19
9332024-10-09T23:17:17.480Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9342024-10-09T23:17:17.503Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9352024-10-09T23:17:17.503Z+ [[ 19 -gt 60 ]]
9362024-10-09T23:17:17.504Z+ sleep 1
9372024-10-09T23:17:18.497Z+ retry=20
9382024-10-09T23:17:18.497Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9392024-10-09T23:17:18.585Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images'
9402024-10-09T23:17:19.370Z{"status":"ok"}success
9412024-10-09T23:17:19.370ZProject {
9422024-10-09T23:17:19.370Z description: "some images",
9432024-10-09T23:17:19.370Z id: a7f92590-5ced-4816-9784-92859b956120,
9442024-10-09T23:17:19.370Z name: Name(
9452024-10-09T23:17:19.370Z "images",
9462024-10-09T23:17:19.370Z ),
9472024-10-09T23:17:19.393Z time_created: 2024-10-09T23:17:18.765653Z,
9482024-10-09T23:17:19.393Z time_modified: 2024-10-09T23:17:18.765653Z,
9492024-10-09T23:17:19.393Z}
9502024-10-09T23:17:19.393Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11
9512024-10-09T23:20:44.383Zdone!
9522024-10-09T23:20:44.483Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11
9532024-10-09T23:20:44.602Zsuccess
9542024-10-09T23:20:44.602ZImage {
9552024-10-09T23:20:44.602Z block_size: ByteCount(
9562024-10-09T23:20:44.602Z 512,
9572024-10-09T23:20:44.602Z ),
9582024-10-09T23:20:44.625Z description: "debian 11 original base image",
9592024-10-09T23:20:44.625Z digest: None,
9602024-10-09T23:20:44.625Z id: 10fdf1d0-c96f-47a2-b82f-6c1c8ea64fb2,
9612024-10-09T23:20:44.625Z name: Name(
9622024-10-09T23:20:44.625Z "debian11",
9632024-10-09T23:20:44.625Z ),
9642024-10-09T23:20:44.625Z os: "debian",
9652024-10-09T23:20:44.625Z project_id: None,
9662024-10-09T23:20:44.625Z size: ByteCount(
9672024-10-09T23:20:44.625Z 2147483648,
9682024-10-09T23:20:44.625Z ),
9692024-10-09T23:20:44.625Z time_created: 2024-10-09T23:20:44.292073Z,
9702024-10-09T23:20:44.625Z time_modified: 2024-10-09T23:20:44.577158Z,
9712024-10-09T23:20:44.625Z url: None,
9722024-10-09T23:20:44.625Z version: "11",
9732024-10-09T23:20:44.625Z}
9742024-10-09T23:20:44.625Z+ for test_bin in tests/*
9752024-10-09T23:20:44.625Z+ ./tests/bootstrap-efd25e8616779453
9762024-10-09T23:20:44.625Z
9772024-10-09T23:20:44.625Zrunning 0 tests
9782024-10-09T23:20:44.625Z
9792024-10-09T23:20:44.625Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
9802024-10-09T23:20:44.625Z
9812024-10-09T23:20:44.626Z+ for test_bin in tests/*
9822024-10-09T23:20:44.626Z+ ./tests/commtest-eaa404bccd4b5540
9832024-10-09T23:20:44.649Z
9842024-10-09T23:20:44.649Zrunning 0 tests
9852024-10-09T23:20:44.649Z
9862024-10-09T23:20:44.649Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
9872024-10-09T23:20:44.649Z
9882024-10-09T23:20:44.649Z+ for test_bin in tests/*
9892024-10-09T23:20:44.649Z+ ./tests/dhcp_server-3126bc2315c994d1
9902024-10-09T23:20:44.649Z
9912024-10-09T23:20:44.649Zrunning 0 tests
9922024-10-09T23:20:44.649Z
9932024-10-09T23:20:44.649Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
9942024-10-09T23:20:44.649Z
9952024-10-09T23:20:44.649Z+ for test_bin in tests/*
9962024-10-09T23:20:44.649Z+ ./tests/end_to_end_tests-f5a0d93c9d75949a
9972024-10-09T23:20:44.716Z
9982024-10-09T23:20:44.716Zrunning 2 tests
9992024-10-09T23:20:44.956Ztest no_spoof::no_spoof_here ... ok
10002024-10-09T23:21:15.414Ztest instance_launch::instance_launch ... ok
10012024-10-09T23:21:15.414Z
10022024-10-09T23:21:15.414Ztest result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 30.70s
10032024-10-09T23:21:15.414Z
10042024-10-09T23:21:15.437Z+ _exit_trap
10052024-10-09T23:21:15.437Z+ local status=0
10062024-10-09T23:21:15.437Z+ set +o errexit
10072024-10-09T23:21:15.437Z+ [[ xf3002b356da7d0e4ca15beb66a5566a92919baaa != \x ]]
10082024-10-09T23:21:15.437Z+ pfexec cp /tmp/opteadm /opt/oxide/opte/bin/opteadm
10092024-10-09T23:21:16.014Z+ pfexec svcadm -Z disable -s cron
10102024-10-09T23:21:17.877Z+ [[ 0 -eq 0 ]]
10112024-10-09T23:21:17.877Z+ exit 0
10122024-10-09T23:21:17.900Zprocess exited: duration 505648 ms, exit code 0
 
10132024-10-09T23:21:18.007Zfound 129 output files
10142024-10-09T23:21:18.007Zuploading: /var/svc/log/oxide-sled-agent:default.log (906244 bytes)
10152024-10-09T23:21:19.284Zuploaded: /var/svc/log/oxide-sled-agent:default.log
10162024-10-09T23:21:19.311Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log (3467352 bytes)
10172024-10-09T23:21:19.663Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log" changed size mid upload: 3467352 -> 3499947
10182024-10-09T23:21:20.733Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log
10192024-10-09T23:21:20.757Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log.0 (399046 bytes)
10202024-10-09T23:21:20.797ZOct 9 23:16:43 buskin last message repeated 2 times
10212024-10-09T23:21:21.995Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log.0
10222024-10-09T23:21:22.020Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log (2855 bytes)
10232024-10-09T23:21:22.045Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log" changed size mid upload: 2855 -> 3152
10242024-10-09T23:21:23.108Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log
10252024-10-09T23:21:23.108Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log.0 (33147 bytes)
10262024-10-09T23:21:24.240Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log.0
10272024-10-09T23:21:24.240Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log (119300 bytes)
10282024-10-09T23:21:24.392Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log" changed size mid upload: 119300 -> 121676
10292024-10-09T23:21:25.458Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log
10302024-10-09T23:21:25.458Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log.0 (12384 bytes)
10312024-10-09T23:21:26.570Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log.0
10322024-10-09T23:21:26.570Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log (170436 bytes)
10332024-10-09T23:21:26.743Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log" changed size mid upload: 170436 -> 179598
10342024-10-09T23:21:27.951Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log
10352024-10-09T23:21:27.951Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log.0 (18717 bytes)
10362024-10-09T23:21:29.063Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log.0
10372024-10-09T23:21:29.063Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log (1014985 bytes)
10382024-10-09T23:21:29.331Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log" changed size mid upload: 1014985 -> 1053593
10392024-10-09T23:21:30.398Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log
10402024-10-09T23:21:30.398Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log.0 (130371 bytes)
10412024-10-09T23:21:31.616Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log.0
10422024-10-09T23:21:31.616Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log (2475646 bytes)
10432024-10-09T23:21:31.935Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log" changed size mid upload: 2475646 -> 2567296
10442024-10-09T23:21:33.003Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log
10452024-10-09T23:21:33.003Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log.0 (254916 bytes)
10462024-10-09T23:21:34.221Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log.0
10472024-10-09T23:21:34.245Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log (0 bytes)
10482024-10-09T23:21:35.289Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log
10492024-10-09T23:21:35.289Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log.0 (3337 bytes)
10502024-10-09T23:21:36.384Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log.0
10512024-10-09T23:21:36.384Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log (188430 bytes)
10522024-10-09T23:21:36.568Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log" changed size mid upload: 188430 -> 197234
10532024-10-09T23:21:37.639Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log
10542024-10-09T23:21:37.663Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log.0 (19941 bytes)
10552024-10-09T23:21:38.779Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log.0
10562024-10-09T23:21:38.804Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log (4680 bytes)
10572024-10-09T23:21:38.828Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log" changed size mid upload: 4680 -> 5265
10582024-10-09T23:21:39.896Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log
10592024-10-09T23:21:39.896Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log.0 (1151 bytes)
10602024-10-09T23:21:40.992Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log.0
10612024-10-09T23:21:41.016Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log (0 bytes)
10622024-10-09T23:21:42.064Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log
10632024-10-09T23:21:42.064Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log.0 (3814 bytes)
10642024-10-09T23:21:43.159Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log.0
10652024-10-09T23:21:43.159Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
10662024-10-09T23:21:44.231Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-zone-network-setup:default.log
10672024-10-09T23:21:44.232Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-zone-network-setup:default.log.0 (9140 bytes)
10682024-10-09T23:21:45.348Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-zone-network-setup:default.log.0
10692024-10-09T23:21:45.348Zuploading: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_cockroachdb_296d28c1-6a09-46dc-a901-111c91db6c76/root/var/svc/log/oxide-cockroach-admin:default.log (4703 bytes)
10702024-10-09T23:21:46.443Zuploaded: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_cockroachdb_296d28c1-6a09-46dc-a901-111c91db6c76/root/var/svc/log/oxide-cockroach-admin:default.log
10712024-10-09T23:21:46.443Zuploading: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_cockroachdb_296d28c1-6a09-46dc-a901-111c91db6c76/root/var/svc/log/oxide-cockroachdb:default.log (4290 bytes)
10722024-10-09T23:21:47.539Zuploaded: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_cockroachdb_296d28c1-6a09-46dc-a901-111c91db6c76/root/var/svc/log/oxide-cockroachdb:default.log
10732024-10-09T23:21:47.539Zuploading: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_cockroachdb_296d28c1-6a09-46dc-a901-111c91db6c76/root/var/svc/log/oxide-zone-network-setup:default.log (2100 bytes)
10742024-10-09T23:21:48.637Zuploaded: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_cockroachdb_296d28c1-6a09-46dc-a901-111c91db6c76/root/var/svc/log/oxide-zone-network-setup:default.log
10752024-10-09T23:21:48.637Zuploading: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_crucible_4977ea60-05fd-4cd5-b6dc-758bfcf15337/root/var/svc/log/oxide-crucible-agent:default.log (12074 bytes)
10762024-10-09T23:21:49.755Zuploaded: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_crucible_4977ea60-05fd-4cd5-b6dc-758bfcf15337/root/var/svc/log/oxide-crucible-agent:default.log
10772024-10-09T23:21:49.755Zuploading: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_crucible_4977ea60-05fd-4cd5-b6dc-758bfcf15337/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10782024-10-09T23:21:50.850Zuploaded: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_crucible_4977ea60-05fd-4cd5-b6dc-758bfcf15337/root/var/svc/log/oxide-crucible-downstairs:default.log
10792024-10-09T23:21:50.850Zuploading: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_crucible_4977ea60-05fd-4cd5-b6dc-758bfcf15337/root/var/svc/log/oxide-crucible-downstairs:downstairs-c801f119-f695-4456-b198-9943479b2084.log (3125 bytes)
10802024-10-09T23:21:51.945Zuploaded: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_crucible_4977ea60-05fd-4cd5-b6dc-758bfcf15337/root/var/svc/log/oxide-crucible-downstairs:downstairs-c801f119-f695-4456-b198-9943479b2084.log
10812024-10-09T23:21:51.945Zuploading: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_crucible_4977ea60-05fd-4cd5-b6dc-758bfcf15337/root/var/svc/log/oxide-zone-network-setup:default.log (2082 bytes)
10822024-10-09T23:21:53.042Zuploaded: /pool/ext/0f1674be-fe99-41ac-8e3d-cac6ad2b4244/crypt/zone/oxz_crucible_4977ea60-05fd-4cd5-b6dc-758bfcf15337/root/var/svc/log/oxide-zone-network-setup:default.log
10832024-10-09T23:21:53.042Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_98956a7e-50cc-4627-a19d-2c0c4594bd80/root/var/svc/log/oxide-crucible-agent:default.log (20180 bytes)
10842024-10-09T23:21:54.159Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_98956a7e-50cc-4627-a19d-2c0c4594bd80/root/var/svc/log/oxide-crucible-agent:default.log
10852024-10-09T23:21:54.159Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_98956a7e-50cc-4627-a19d-2c0c4594bd80/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10862024-10-09T23:21:55.254Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_98956a7e-50cc-4627-a19d-2c0c4594bd80/root/var/svc/log/oxide-crucible-downstairs:default.log
10872024-10-09T23:21:55.278Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_98956a7e-50cc-4627-a19d-2c0c4594bd80/root/var/svc/log/oxide-crucible-downstairs:downstairs-a375b62e-4c45-489e-9e11-0d29f000a9ff.log (6079 bytes)
10882024-10-09T23:21:56.350Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_98956a7e-50cc-4627-a19d-2c0c4594bd80/root/var/svc/log/oxide-crucible-downstairs:downstairs-a375b62e-4c45-489e-9e11-0d29f000a9ff.log
10892024-10-09T23:21:56.350Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_98956a7e-50cc-4627-a19d-2c0c4594bd80/root/var/svc/log/oxide-crucible-downstairs:snapshot-a375b62e-4c45-489e-9e11-0d29f000a9ff-c171cea4-eebc-44fb-83e6-14ac51e1d1ff.log (5688 bytes)
10902024-10-09T23:21:57.456Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_98956a7e-50cc-4627-a19d-2c0c4594bd80/root/var/svc/log/oxide-crucible-downstairs:snapshot-a375b62e-4c45-489e-9e11-0d29f000a9ff-c171cea4-eebc-44fb-83e6-14ac51e1d1ff.log
10912024-10-09T23:21:57.457Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_98956a7e-50cc-4627-a19d-2c0c4594bd80/root/var/svc/log/oxide-zone-network-setup:default.log (2083 bytes)
10922024-10-09T23:21:58.549Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_98956a7e-50cc-4627-a19d-2c0c4594bd80/root/var/svc/log/oxide-zone-network-setup:default.log
10932024-10-09T23:21:58.550Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_pantry_21466580-0e6d-4fd0-90cb-e87b1775c87d/root/var/svc/log/oxide-crucible-pantry:default.log (765675 bytes)
10942024-10-09T23:21:59.875Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_pantry_21466580-0e6d-4fd0-90cb-e87b1775c87d/root/var/svc/log/oxide-crucible-pantry:default.log
10952024-10-09T23:21:59.898Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_pantry_21466580-0e6d-4fd0-90cb-e87b1775c87d/root/var/svc/log/oxide-zone-network-setup:default.log (2137 bytes)
10962024-10-09T23:22:00.970Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_crucible_pantry_21466580-0e6d-4fd0-90cb-e87b1775c87d/root/var/svc/log/oxide-zone-network-setup:default.log
10972024-10-09T23:22:00.971Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-chrony-setup:default.log (0 bytes)
10982024-10-09T23:22:02.044Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-chrony-setup:default.log
10992024-10-09T23:22:02.044Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-chrony-setup:default.log.0 (14495 bytes)
11002024-10-09T23:22:03.162Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-chrony-setup:default.log.0
11012024-10-09T23:22:03.162Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-ntp:default.log (1332 bytes)
11022024-10-09T23:22:04.258Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-ntp:default.log
11032024-10-09T23:22:04.282Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-ntp:default.log.0 (788 bytes)
11042024-10-09T23:22:05.355Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-ntp:default.log.0
11052024-10-09T23:22:05.355Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-opte-interface-setup:default.log (0 bytes)
11062024-10-09T23:22:06.426Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-opte-interface-setup:default.log
11072024-10-09T23:22:06.426Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-opte-interface-setup:default.log.0 (1125 bytes)
11082024-10-09T23:22:07.522Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-opte-interface-setup:default.log.0
11092024-10-09T23:22:07.522Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
11102024-10-09T23:22:08.594Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-zone-network-setup:default.log
11112024-10-09T23:22:08.594Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-zone-network-setup:default.log.0 (2031 bytes)
11122024-10-09T23:22:09.689Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/svc/log/oxide-zone-network-setup:default.log.0
11132024-10-09T23:22:09.689Zuploading: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_crucible_730f9ba3-b591-4d14-871d-a36777ecd376/root/var/svc/log/oxide-crucible-agent:default.log (8485 bytes)
11142024-10-09T23:22:10.807Zuploaded: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_crucible_730f9ba3-b591-4d14-871d-a36777ecd376/root/var/svc/log/oxide-crucible-agent:default.log
11152024-10-09T23:22:10.807Zuploading: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_crucible_730f9ba3-b591-4d14-871d-a36777ecd376/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11162024-10-09T23:22:11.901Zuploaded: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_crucible_730f9ba3-b591-4d14-871d-a36777ecd376/root/var/svc/log/oxide-crucible-downstairs:default.log
11172024-10-09T23:22:11.925Zuploading: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_crucible_730f9ba3-b591-4d14-871d-a36777ecd376/root/var/svc/log/oxide-crucible-downstairs:downstairs-854b3848-814c-4c5e-981c-134ddfe536ad.log (3019 bytes)
11182024-10-09T23:22:12.999Zuploaded: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_crucible_730f9ba3-b591-4d14-871d-a36777ecd376/root/var/svc/log/oxide-crucible-downstairs:downstairs-854b3848-814c-4c5e-981c-134ddfe536ad.log
11192024-10-09T23:22:12.999Zuploading: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_crucible_730f9ba3-b591-4d14-871d-a36777ecd376/root/var/svc/log/oxide-zone-network-setup:default.log (2083 bytes)
11202024-10-09T23:22:14.093Zuploaded: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_crucible_730f9ba3-b591-4d14-871d-a36777ecd376/root/var/svc/log/oxide-zone-network-setup:default.log
11212024-10-09T23:22:14.093Zuploading: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_crucible_pantry_88ebb407-307a-4426-b2f7-023ffd035250/root/var/svc/log/oxide-crucible-pantry:default.log (780 bytes)
11222024-10-09T23:22:15.188Zuploaded: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_crucible_pantry_88ebb407-307a-4426-b2f7-023ffd035250/root/var/svc/log/oxide-crucible-pantry:default.log
11232024-10-09T23:22:15.213Zuploading: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_crucible_pantry_88ebb407-307a-4426-b2f7-023ffd035250/root/var/svc/log/oxide-zone-network-setup:default.log (2136 bytes)
11242024-10-09T23:22:16.285Zuploaded: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_crucible_pantry_88ebb407-307a-4426-b2f7-023ffd035250/root/var/svc/log/oxide-zone-network-setup:default.log
11252024-10-09T23:22:16.308Zuploading: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_nexus_efde9fa7-2c58-47bd-9c83-7e8e861f5541/root/var/svc/log/oxide-nexus:default.log (35988765 bytes)
11262024-10-09T23:22:17.802Zupload warning: file "/pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_nexus_efde9fa7-2c58-47bd-9c83-7e8e861f5541/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 35988765 -> 36366533
11272024-10-09T23:22:18.874Zuploaded: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_nexus_efde9fa7-2c58-47bd-9c83-7e8e861f5541/root/var/svc/log/oxide-nexus:default.log
11282024-10-09T23:22:18.874Zuploading: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_nexus_efde9fa7-2c58-47bd-9c83-7e8e861f5541/root/var/svc/log/oxide-opte-interface-setup:default.log (1135 bytes)
11292024-10-09T23:22:19.969Zuploaded: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_nexus_efde9fa7-2c58-47bd-9c83-7e8e861f5541/root/var/svc/log/oxide-opte-interface-setup:default.log
11302024-10-09T23:22:19.969Zuploading: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_nexus_efde9fa7-2c58-47bd-9c83-7e8e861f5541/root/var/svc/log/oxide-zone-network-setup:default.log (2056 bytes)
11312024-10-09T23:22:21.064Zuploaded: /pool/ext/2927d630-c36c-46a6-86b2-155254207fe2/crypt/zone/oxz_nexus_efde9fa7-2c58-47bd-9c83-7e8e861f5541/root/var/svc/log/oxide-zone-network-setup:default.log
11322024-10-09T23:22:21.088Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_cockroachdb_91e7504b-6c93-4ea7-a003-f0fa91df59ba/root/var/svc/log/oxide-cockroach-admin:default.log (4704 bytes)
11332024-10-09T23:22:22.158Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_cockroachdb_91e7504b-6c93-4ea7-a003-f0fa91df59ba/root/var/svc/log/oxide-cockroach-admin:default.log
11342024-10-09T23:22:22.159Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_cockroachdb_91e7504b-6c93-4ea7-a003-f0fa91df59ba/root/var/svc/log/oxide-cockroachdb:default.log (4315 bytes)
11352024-10-09T23:22:23.253Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_cockroachdb_91e7504b-6c93-4ea7-a003-f0fa91df59ba/root/var/svc/log/oxide-cockroachdb:default.log
11362024-10-09T23:22:23.254Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_cockroachdb_91e7504b-6c93-4ea7-a003-f0fa91df59ba/root/var/svc/log/oxide-zone-network-setup:default.log (2101 bytes)
11372024-10-09T23:22:24.349Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_cockroachdb_91e7504b-6c93-4ea7-a003-f0fa91df59ba/root/var/svc/log/oxide-zone-network-setup:default.log
11382024-10-09T23:22:24.374Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_crucible_1d5363df-a291-4bb5-94a2-3e748d94e7ab/root/var/svc/log/oxide-crucible-agent:default.log (8485 bytes)
11392024-10-09T23:22:25.467Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_crucible_1d5363df-a291-4bb5-94a2-3e748d94e7ab/root/var/svc/log/oxide-crucible-agent:default.log
11402024-10-09T23:22:25.490Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_crucible_1d5363df-a291-4bb5-94a2-3e748d94e7ab/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11412024-10-09T23:22:26.562Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_crucible_1d5363df-a291-4bb5-94a2-3e748d94e7ab/root/var/svc/log/oxide-crucible-downstairs:default.log
11422024-10-09T23:22:26.562Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_crucible_1d5363df-a291-4bb5-94a2-3e748d94e7ab/root/var/svc/log/oxide-crucible-downstairs:downstairs-27b13720-49b4-450a-b7d4-61c09edc5c8a.log (3019 bytes)
11432024-10-09T23:22:27.657Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_crucible_1d5363df-a291-4bb5-94a2-3e748d94e7ab/root/var/svc/log/oxide-crucible-downstairs:downstairs-27b13720-49b4-450a-b7d4-61c09edc5c8a.log
11442024-10-09T23:22:27.657Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_crucible_1d5363df-a291-4bb5-94a2-3e748d94e7ab/root/var/svc/log/oxide-zone-network-setup:default.log (2082 bytes)
11452024-10-09T23:22:28.755Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_crucible_1d5363df-a291-4bb5-94a2-3e748d94e7ab/root/var/svc/log/oxide-zone-network-setup:default.log
11462024-10-09T23:22:28.755Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_external_dns_b2ea7862-47bb-42f1-b96b-92ee1e8de512/root/var/svc/log/oxide-external_dns:default.log (19351 bytes)
11472024-10-09T23:22:28.847Zupload warning: file "/pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_external_dns_b2ea7862-47bb-42f1-b96b-92ee1e8de512/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 19351 -> 26904
11482024-10-09T23:22:29.919Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_external_dns_b2ea7862-47bb-42f1-b96b-92ee1e8de512/root/var/svc/log/oxide-external_dns:default.log
11492024-10-09T23:22:29.919Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_external_dns_b2ea7862-47bb-42f1-b96b-92ee1e8de512/root/var/svc/log/oxide-opte-interface-setup:default.log (1156 bytes)
11502024-10-09T23:22:31.014Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_external_dns_b2ea7862-47bb-42f1-b96b-92ee1e8de512/root/var/svc/log/oxide-opte-interface-setup:default.log
11512024-10-09T23:22:31.014Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_external_dns_b2ea7862-47bb-42f1-b96b-92ee1e8de512/root/var/svc/log/oxide-zone-network-setup:default.log (2113 bytes)
11522024-10-09T23:22:32.110Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_external_dns_b2ea7862-47bb-42f1-b96b-92ee1e8de512/root/var/svc/log/oxide-zone-network-setup:default.log
11532024-10-09T23:22:32.110Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_internal_dns_9d6d56e0-7c0f-4073-8759-1fa3c136a560/root/var/svc/log/oxide-internal_dns:default.log (408939 bytes)
11542024-10-09T23:22:32.342Zupload warning: file "/pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_internal_dns_9d6d56e0-7c0f-4073-8759-1fa3c136a560/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 408939 -> 416453
11552024-10-09T23:22:33.414Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_internal_dns_9d6d56e0-7c0f-4073-8759-1fa3c136a560/root/var/svc/log/oxide-internal_dns:default.log
11562024-10-09T23:22:33.414Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_internal_dns_9d6d56e0-7c0f-4073-8759-1fa3c136a560/root/var/svc/log/oxide-internal_dns:default.log.0 (4247 bytes)
11572024-10-09T23:22:34.510Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_internal_dns_9d6d56e0-7c0f-4073-8759-1fa3c136a560/root/var/svc/log/oxide-internal_dns:default.log.0
11582024-10-09T23:22:34.510Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_internal_dns_9d6d56e0-7c0f-4073-8759-1fa3c136a560/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
11592024-10-09T23:22:35.582Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_internal_dns_9d6d56e0-7c0f-4073-8759-1fa3c136a560/root/var/svc/log/oxide-zone-network-setup:default.log
11602024-10-09T23:22:35.582Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_internal_dns_9d6d56e0-7c0f-4073-8759-1fa3c136a560/root/var/svc/log/oxide-zone-network-setup:default.log.0 (2094 bytes)
11612024-10-09T23:22:36.677Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_internal_dns_9d6d56e0-7c0f-4073-8759-1fa3c136a560/root/var/svc/log/oxide-zone-network-setup:default.log.0
11622024-10-09T23:22:36.677Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_nexus_780e1728-4013-4877-b14f-85b67afdf49d/root/var/svc/log/oxide-nexus:default.log (2105719 bytes)
11632024-10-09T23:22:37.038Zupload warning: file "/pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_nexus_780e1728-4013-4877-b14f-85b67afdf49d/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 2105719 -> 2688942
11642024-10-09T23:22:38.109Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_nexus_780e1728-4013-4877-b14f-85b67afdf49d/root/var/svc/log/oxide-nexus:default.log
11652024-10-09T23:22:38.109Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_nexus_780e1728-4013-4877-b14f-85b67afdf49d/root/var/svc/log/oxide-opte-interface-setup:default.log (1134 bytes)
11662024-10-09T23:22:39.204Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_nexus_780e1728-4013-4877-b14f-85b67afdf49d/root/var/svc/log/oxide-opte-interface-setup:default.log
11672024-10-09T23:22:39.227Zuploading: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_nexus_780e1728-4013-4877-b14f-85b67afdf49d/root/var/svc/log/oxide-zone-network-setup:default.log (2057 bytes)
11682024-10-09T23:22:40.300Zuploaded: /pool/ext/2c619501-cf3c-4716-acb2-69e678bc135e/crypt/zone/oxz_nexus_780e1728-4013-4877-b14f-85b67afdf49d/root/var/svc/log/oxide-zone-network-setup:default.log
11692024-10-09T23:22:40.324Zuploading: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_cockroachdb_f43ac0d3-a955-46a3-b046-78df1f53f818/root/var/svc/log/oxide-cockroach-admin:default.log (4704 bytes)
11702024-10-09T23:22:41.395Zuploaded: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_cockroachdb_f43ac0d3-a955-46a3-b046-78df1f53f818/root/var/svc/log/oxide-cockroach-admin:default.log
11712024-10-09T23:22:41.419Zuploading: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_cockroachdb_f43ac0d3-a955-46a3-b046-78df1f53f818/root/var/svc/log/oxide-cockroachdb:default.log (4316 bytes)
11722024-10-09T23:22:42.491Zuploaded: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_cockroachdb_f43ac0d3-a955-46a3-b046-78df1f53f818/root/var/svc/log/oxide-cockroachdb:default.log
11732024-10-09T23:22:42.514Zuploading: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_cockroachdb_f43ac0d3-a955-46a3-b046-78df1f53f818/root/var/svc/log/oxide-zone-network-setup:default.log (2101 bytes)
11742024-10-09T23:22:43.586Zuploaded: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_cockroachdb_f43ac0d3-a955-46a3-b046-78df1f53f818/root/var/svc/log/oxide-zone-network-setup:default.log
11752024-10-09T23:22:43.586Zuploading: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_crucible_9dc79142-e4e4-4f5e-824a-83e47e72240b/root/var/svc/log/oxide-crucible-agent:default.log (12932 bytes)
11762024-10-09T23:22:44.704Zuploaded: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_crucible_9dc79142-e4e4-4f5e-824a-83e47e72240b/root/var/svc/log/oxide-crucible-agent:default.log
11772024-10-09T23:22:44.704Zuploading: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_crucible_9dc79142-e4e4-4f5e-824a-83e47e72240b/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11782024-10-09T23:22:45.799Zuploaded: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_crucible_9dc79142-e4e4-4f5e-824a-83e47e72240b/root/var/svc/log/oxide-crucible-downstairs:default.log
11792024-10-09T23:22:45.799Zuploading: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_crucible_9dc79142-e4e4-4f5e-824a-83e47e72240b/root/var/svc/log/oxide-crucible-downstairs:downstairs-4158cb89-26dc-4d0c-a14d-66b7269ff7fa.log (5994 bytes)
11802024-10-09T23:22:46.896Zuploaded: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_crucible_9dc79142-e4e4-4f5e-824a-83e47e72240b/root/var/svc/log/oxide-crucible-downstairs:downstairs-4158cb89-26dc-4d0c-a14d-66b7269ff7fa.log
11812024-10-09T23:22:46.896Zuploading: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_crucible_9dc79142-e4e4-4f5e-824a-83e47e72240b/root/var/svc/log/oxide-zone-network-setup:default.log (2083 bytes)
11822024-10-09T23:22:47.991Zuploaded: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_crucible_9dc79142-e4e4-4f5e-824a-83e47e72240b/root/var/svc/log/oxide-zone-network-setup:default.log
11832024-10-09T23:22:48.015Zuploading: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_internal_dns_9aa16901-d5aa-4906-a0da-9afcb83164e5/root/var/svc/log/oxide-internal_dns:default.log (408952 bytes)
11842024-10-09T23:22:48.222Zupload warning: file "/pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_internal_dns_9aa16901-d5aa-4906-a0da-9afcb83164e5/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 408952 -> 416460
11852024-10-09T23:22:49.294Zuploaded: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_internal_dns_9aa16901-d5aa-4906-a0da-9afcb83164e5/root/var/svc/log/oxide-internal_dns:default.log
11862024-10-09T23:22:49.294Zuploading: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_internal_dns_9aa16901-d5aa-4906-a0da-9afcb83164e5/root/var/svc/log/oxide-internal_dns:default.log.0 (4246 bytes)
11872024-10-09T23:22:50.389Zuploaded: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_internal_dns_9aa16901-d5aa-4906-a0da-9afcb83164e5/root/var/svc/log/oxide-internal_dns:default.log.0
11882024-10-09T23:22:50.389Zuploading: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_internal_dns_9aa16901-d5aa-4906-a0da-9afcb83164e5/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
11892024-10-09T23:22:51.461Zuploaded: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_internal_dns_9aa16901-d5aa-4906-a0da-9afcb83164e5/root/var/svc/log/oxide-zone-network-setup:default.log
11902024-10-09T23:22:51.484Zuploading: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_internal_dns_9aa16901-d5aa-4906-a0da-9afcb83164e5/root/var/svc/log/oxide-zone-network-setup:default.log.0 (2094 bytes)
11912024-10-09T23:22:52.556Zuploaded: /pool/ext/363ba9dc-50b0-41b3-bad2-63af668f1bfb/crypt/zone/oxz_internal_dns_9aa16901-d5aa-4906-a0da-9afcb83164e5/root/var/svc/log/oxide-zone-network-setup:default.log.0
11922024-10-09T23:22:52.556Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_clickhouse_6c473b70-0ba6-4006-b920-2aef978e5a6d/root/var/svc/log/oxide-clickhouse-admin:default.log (876 bytes)
11932024-10-09T23:22:53.653Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_clickhouse_6c473b70-0ba6-4006-b920-2aef978e5a6d/root/var/svc/log/oxide-clickhouse-admin:default.log
11942024-10-09T23:22:53.653Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_clickhouse_6c473b70-0ba6-4006-b920-2aef978e5a6d/root/var/svc/log/oxide-clickhouse:default.log (2306421 bytes)
11952024-10-09T23:22:54.016Zupload warning: file "/pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_clickhouse_6c473b70-0ba6-4006-b920-2aef978e5a6d/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 2306421 -> 3008175
11962024-10-09T23:22:55.087Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_clickhouse_6c473b70-0ba6-4006-b920-2aef978e5a6d/root/var/svc/log/oxide-clickhouse:default.log
11972024-10-09T23:22:55.087Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_clickhouse_6c473b70-0ba6-4006-b920-2aef978e5a6d/root/var/svc/log/oxide-zone-network-setup:default.log (2097 bytes)
11982024-10-09T23:22:56.183Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_clickhouse_6c473b70-0ba6-4006-b920-2aef978e5a6d/root/var/svc/log/oxide-zone-network-setup:default.log
11992024-10-09T23:22:56.183Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_cockroachdb_2f373715-5d4a-4352-ae4b-4e6b8538d865/root/var/svc/log/oxide-cockroach-admin:default.log (4704 bytes)
12002024-10-09T23:22:57.279Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_cockroachdb_2f373715-5d4a-4352-ae4b-4e6b8538d865/root/var/svc/log/oxide-cockroach-admin:default.log
12012024-10-09T23:22:57.279Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_cockroachdb_2f373715-5d4a-4352-ae4b-4e6b8538d865/root/var/svc/log/oxide-cockroachdb:default.log (4387 bytes)
12022024-10-09T23:22:58.375Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_cockroachdb_2f373715-5d4a-4352-ae4b-4e6b8538d865/root/var/svc/log/oxide-cockroachdb:default.log
12032024-10-09T23:22:58.375Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_cockroachdb_2f373715-5d4a-4352-ae4b-4e6b8538d865/root/var/svc/log/oxide-zone-network-setup:default.log (2101 bytes)
12042024-10-09T23:22:59.470Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_cockroachdb_2f373715-5d4a-4352-ae4b-4e6b8538d865/root/var/svc/log/oxide-zone-network-setup:default.log
12052024-10-09T23:22:59.470Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_crucible_01abb86c-7548-4e4e-a856-4e78190a68c2/root/var/svc/log/oxide-crucible-agent:default.log (1554 bytes)
12062024-10-09T23:23:00.566Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_crucible_01abb86c-7548-4e4e-a856-4e78190a68c2/root/var/svc/log/oxide-crucible-agent:default.log
12072024-10-09T23:23:00.566Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_crucible_01abb86c-7548-4e4e-a856-4e78190a68c2/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12082024-10-09T23:23:01.663Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_crucible_01abb86c-7548-4e4e-a856-4e78190a68c2/root/var/svc/log/oxide-crucible-downstairs:default.log
12092024-10-09T23:23:01.663Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_crucible_01abb86c-7548-4e4e-a856-4e78190a68c2/root/var/svc/log/oxide-zone-network-setup:default.log (2082 bytes)
12102024-10-09T23:23:02.758Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_crucible_01abb86c-7548-4e4e-a856-4e78190a68c2/root/var/svc/log/oxide-zone-network-setup:default.log
12112024-10-09T23:23:02.758Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_external_dns_81bb312e-67a2-42b5-a372-8822e031590a/root/var/svc/log/oxide-external_dns:default.log (19347 bytes)
12122024-10-09T23:23:02.851Zupload warning: file "/pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_external_dns_81bb312e-67a2-42b5-a372-8822e031590a/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 19347 -> 26904
12132024-10-09T23:23:03.922Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_external_dns_81bb312e-67a2-42b5-a372-8822e031590a/root/var/svc/log/oxide-external_dns:default.log
12142024-10-09T23:23:03.946Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_external_dns_81bb312e-67a2-42b5-a372-8822e031590a/root/var/svc/log/oxide-opte-interface-setup:default.log (1156 bytes)
12152024-10-09T23:23:05.019Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_external_dns_81bb312e-67a2-42b5-a372-8822e031590a/root/var/svc/log/oxide-opte-interface-setup:default.log
12162024-10-09T23:23:05.043Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_external_dns_81bb312e-67a2-42b5-a372-8822e031590a/root/var/svc/log/oxide-zone-network-setup:default.log (2113 bytes)
12172024-10-09T23:23:06.114Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_external_dns_81bb312e-67a2-42b5-a372-8822e031590a/root/var/svc/log/oxide-zone-network-setup:default.log
12182024-10-09T23:23:06.114Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_internal_dns_257ff83f-2fc4-4972-9236-ce4ec13342d2/root/var/svc/log/oxide-internal_dns:default.log (408968 bytes)
12192024-10-09T23:23:06.344Zupload warning: file "/pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_internal_dns_257ff83f-2fc4-4972-9236-ce4ec13342d2/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 408968 -> 416476
12202024-10-09T23:23:07.415Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_internal_dns_257ff83f-2fc4-4972-9236-ce4ec13342d2/root/var/svc/log/oxide-internal_dns:default.log
12212024-10-09T23:23:07.415Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_internal_dns_257ff83f-2fc4-4972-9236-ce4ec13342d2/root/var/svc/log/oxide-internal_dns:default.log.0 (4244 bytes)
12222024-10-09T23:23:08.510Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_internal_dns_257ff83f-2fc4-4972-9236-ce4ec13342d2/root/var/svc/log/oxide-internal_dns:default.log.0
12232024-10-09T23:23:08.510Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_internal_dns_257ff83f-2fc4-4972-9236-ce4ec13342d2/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
12242024-10-09T23:23:09.582Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_internal_dns_257ff83f-2fc4-4972-9236-ce4ec13342d2/root/var/svc/log/oxide-zone-network-setup:default.log
12252024-10-09T23:23:09.582Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_internal_dns_257ff83f-2fc4-4972-9236-ce4ec13342d2/root/var/svc/log/oxide-zone-network-setup:default.log.0 (2094 bytes)
12262024-10-09T23:23:10.676Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_internal_dns_257ff83f-2fc4-4972-9236-ce4ec13342d2/root/var/svc/log/oxide-zone-network-setup:default.log.0
12272024-10-09T23:23:10.676Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_nexus_11e60681-3fe1-4c95-b87c-299ad6620a05/root/var/svc/log/oxide-nexus:default.log (4604318 bytes)
12282024-10-09T23:23:11.128Zupload warning: file "/pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_nexus_11e60681-3fe1-4c95-b87c-299ad6620a05/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 4604318 -> 5424092
12292024-10-09T23:23:12.199Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_nexus_11e60681-3fe1-4c95-b87c-299ad6620a05/root/var/svc/log/oxide-nexus:default.log
12302024-10-09T23:23:12.199Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_nexus_11e60681-3fe1-4c95-b87c-299ad6620a05/root/var/svc/log/oxide-opte-interface-setup:default.log (1133 bytes)
12312024-10-09T23:23:13.295Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_nexus_11e60681-3fe1-4c95-b87c-299ad6620a05/root/var/svc/log/oxide-opte-interface-setup:default.log
12322024-10-09T23:23:13.295Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_nexus_11e60681-3fe1-4c95-b87c-299ad6620a05/root/var/svc/log/oxide-zone-network-setup:default.log (2055 bytes)
12332024-10-09T23:23:14.392Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_nexus_11e60681-3fe1-4c95-b87c-299ad6620a05/root/var/svc/log/oxide-zone-network-setup:default.log
12342024-10-09T23:23:14.415Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_oximeter_6e9f3473-a977-4665-9200-6b8f70af97ab/root/var/svc/log/oxide-oximeter:default.log (1286542 bytes)
12352024-10-09T23:23:14.718Zupload warning: file "/pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_oximeter_6e9f3473-a977-4665-9200-6b8f70af97ab/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 1286542 -> 1791088
12362024-10-09T23:23:15.790Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_oximeter_6e9f3473-a977-4665-9200-6b8f70af97ab/root/var/svc/log/oxide-oximeter:default.log
12372024-10-09T23:23:15.814Zuploading: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_oximeter_6e9f3473-a977-4665-9200-6b8f70af97ab/root/var/svc/log/oxide-zone-network-setup:default.log (2081 bytes)
12382024-10-09T23:23:16.885Zuploaded: /pool/ext/5e768dfd-a37c-4c4d-a2ad-d71bbce45468/crypt/zone/oxz_oximeter_6e9f3473-a977-4665-9200-6b8f70af97ab/root/var/svc/log/oxide-zone-network-setup:default.log
12392024-10-09T23:23:16.909Zuploading: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_cockroachdb_b8fccc6d-c98d-4d77-8b18-9ed5c07abe26/root/var/svc/log/oxide-cockroach-admin:default.log (4704 bytes)
12402024-10-09T23:23:17.980Zuploaded: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_cockroachdb_b8fccc6d-c98d-4d77-8b18-9ed5c07abe26/root/var/svc/log/oxide-cockroach-admin:default.log
12412024-10-09T23:23:17.980Zuploading: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_cockroachdb_b8fccc6d-c98d-4d77-8b18-9ed5c07abe26/root/var/svc/log/oxide-cockroachdb:default.log (4316 bytes)
12422024-10-09T23:23:19.076Zuploaded: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_cockroachdb_b8fccc6d-c98d-4d77-8b18-9ed5c07abe26/root/var/svc/log/oxide-cockroachdb:default.log
12432024-10-09T23:23:19.076Zuploading: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_cockroachdb_b8fccc6d-c98d-4d77-8b18-9ed5c07abe26/root/var/svc/log/oxide-zone-network-setup:default.log (2099 bytes)
12442024-10-09T23:23:20.171Zuploaded: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_cockroachdb_b8fccc6d-c98d-4d77-8b18-9ed5c07abe26/root/var/svc/log/oxide-zone-network-setup:default.log
12452024-10-09T23:23:20.171Zuploading: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_crucible_ae7593a4-269b-4cc0-8f04-c4fac2da5ac9/root/var/svc/log/oxide-crucible-agent:default.log (28095 bytes)
12462024-10-09T23:23:20.748ZOct 9 23:23:20 buskin xde: WARNING: failed to generate descriptor for stateful action: ICMP:172.30.3.5:0:192.48.105.15:0 Unexpected { msg: "Expected ICMP Echo Request, found: destination unreachable" }
12472024-10-09T23:23:21.312Zuploaded: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_crucible_ae7593a4-269b-4cc0-8f04-c4fac2da5ac9/root/var/svc/log/oxide-crucible-agent:default.log
12482024-10-09T23:23:21.312Zuploading: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_crucible_ae7593a4-269b-4cc0-8f04-c4fac2da5ac9/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12492024-10-09T23:23:22.407Zuploaded: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_crucible_ae7593a4-269b-4cc0-8f04-c4fac2da5ac9/root/var/svc/log/oxide-crucible-downstairs:default.log
12502024-10-09T23:23:22.407Zuploading: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_crucible_ae7593a4-269b-4cc0-8f04-c4fac2da5ac9/root/var/svc/log/oxide-crucible-downstairs:downstairs-7e0c37d7-d403-48a0-b9d7-f51750510261.log (3125 bytes)
12512024-10-09T23:23:23.502Zuploaded: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_crucible_ae7593a4-269b-4cc0-8f04-c4fac2da5ac9/root/var/svc/log/oxide-crucible-downstairs:downstairs-7e0c37d7-d403-48a0-b9d7-f51750510261.log
12522024-10-09T23:23:23.526Zuploading: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_crucible_ae7593a4-269b-4cc0-8f04-c4fac2da5ac9/root/var/svc/log/oxide-crucible-downstairs:downstairs-b18aba8c-46de-4a43-a1d2-ffe2b26ecd55.log (6076 bytes)
12532024-10-09T23:23:24.598Zuploaded: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_crucible_ae7593a4-269b-4cc0-8f04-c4fac2da5ac9/root/var/svc/log/oxide-crucible-downstairs:downstairs-b18aba8c-46de-4a43-a1d2-ffe2b26ecd55.log
12542024-10-09T23:23:24.598Zuploading: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_crucible_ae7593a4-269b-4cc0-8f04-c4fac2da5ac9/root/var/svc/log/oxide-crucible-downstairs:snapshot-b18aba8c-46de-4a43-a1d2-ffe2b26ecd55-c171cea4-eebc-44fb-83e6-14ac51e1d1ff.log (5681 bytes)
12552024-10-09T23:23:25.693Zuploaded: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_crucible_ae7593a4-269b-4cc0-8f04-c4fac2da5ac9/root/var/svc/log/oxide-crucible-downstairs:snapshot-b18aba8c-46de-4a43-a1d2-ffe2b26ecd55-c171cea4-eebc-44fb-83e6-14ac51e1d1ff.log
12562024-10-09T23:23:25.694Zuploading: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_crucible_ae7593a4-269b-4cc0-8f04-c4fac2da5ac9/root/var/svc/log/oxide-zone-network-setup:default.log (2082 bytes)
12572024-10-09T23:23:26.789Zuploaded: /pool/ext/9ebd9b70-0c29-4ec7-962a-d04103f163ac/crypt/zone/oxz_crucible_ae7593a4-269b-4cc0-8f04-c4fac2da5ac9/root/var/svc/log/oxide-zone-network-setup:default.log
12582024-10-09T23:23:26.789Zuploading: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_eebb19fb-8a76-4d3f-b307-c87182050435/root/var/svc/log/oxide-crucible-agent:default.log (30119 bytes)
12592024-10-09T23:23:27.951Zuploaded: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_eebb19fb-8a76-4d3f-b307-c87182050435/root/var/svc/log/oxide-crucible-agent:default.log
12602024-10-09T23:23:27.951Zuploading: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_eebb19fb-8a76-4d3f-b307-c87182050435/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12612024-10-09T23:23:29.046Zuploaded: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_eebb19fb-8a76-4d3f-b307-c87182050435/root/var/svc/log/oxide-crucible-downstairs:default.log
12622024-10-09T23:23:29.079Zuploading: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_eebb19fb-8a76-4d3f-b307-c87182050435/root/var/svc/log/oxide-crucible-downstairs:downstairs-253677c7-c7ba-4291-8561-86da44175c35.log (3021 bytes)
12632024-10-09T23:23:30.151Zuploaded: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_eebb19fb-8a76-4d3f-b307-c87182050435/root/var/svc/log/oxide-crucible-downstairs:downstairs-253677c7-c7ba-4291-8561-86da44175c35.log
12642024-10-09T23:23:30.151Zuploading: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_eebb19fb-8a76-4d3f-b307-c87182050435/root/var/svc/log/oxide-crucible-downstairs:downstairs-3f60daa1-c8d6-4956-b0b4-06cab44c9985.log (5996 bytes)
12652024-10-09T23:23:31.247Zuploaded: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_eebb19fb-8a76-4d3f-b307-c87182050435/root/var/svc/log/oxide-crucible-downstairs:downstairs-3f60daa1-c8d6-4956-b0b4-06cab44c9985.log
12662024-10-09T23:23:31.270Zuploading: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_eebb19fb-8a76-4d3f-b307-c87182050435/root/var/svc/log/oxide-crucible-downstairs:downstairs-844f4702-477b-4c70-a0d1-de79a389484d.log (3127 bytes)
12672024-10-09T23:23:32.343Zuploaded: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_eebb19fb-8a76-4d3f-b307-c87182050435/root/var/svc/log/oxide-crucible-downstairs:downstairs-844f4702-477b-4c70-a0d1-de79a389484d.log
12682024-10-09T23:23:32.343Zuploading: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_eebb19fb-8a76-4d3f-b307-c87182050435/root/var/svc/log/oxide-zone-network-setup:default.log (2082 bytes)
12692024-10-09T23:23:33.438Zuploaded: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_eebb19fb-8a76-4d3f-b307-c87182050435/root/var/svc/log/oxide-zone-network-setup:default.log
12702024-10-09T23:23:33.461Zuploading: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_pantry_87d0e86f-6a94-401b-960c-1ad12cd0f995/root/var/svc/log/oxide-crucible-pantry:default.log (783 bytes)
12712024-10-09T23:23:34.533Zuploaded: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_pantry_87d0e86f-6a94-401b-960c-1ad12cd0f995/root/var/svc/log/oxide-crucible-pantry:default.log
12722024-10-09T23:23:34.533Zuploading: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_pantry_87d0e86f-6a94-401b-960c-1ad12cd0f995/root/var/svc/log/oxide-zone-network-setup:default.log (2139 bytes)
12732024-10-09T23:23:35.628Zuploaded: /pool/ext/c38891dd-3d1d-440e-9c76-9b395f0c96d7/crypt/zone/oxz_crucible_pantry_87d0e86f-6a94-401b-960c-1ad12cd0f995/root/var/svc/log/oxide-zone-network-setup:default.log
12742024-10-09T23:23:35.628Zuploading: /pool/ext/ef6af0ca-2803-42da-991f-fb732a7b3eda/crypt/zone/oxz_crucible_074083cb-db23-46c8-8ec1-8074ad72183f/root/var/svc/log/oxide-crucible-agent:default.log (36486 bytes)
12752024-10-09T23:23:36.790Zuploaded: /pool/ext/ef6af0ca-2803-42da-991f-fb732a7b3eda/crypt/zone/oxz_crucible_074083cb-db23-46c8-8ec1-8074ad72183f/root/var/svc/log/oxide-crucible-agent:default.log
12762024-10-09T23:23:36.790Zuploading: /pool/ext/ef6af0ca-2803-42da-991f-fb732a7b3eda/crypt/zone/oxz_crucible_074083cb-db23-46c8-8ec1-8074ad72183f/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12772024-10-09T23:23:37.886Zuploaded: /pool/ext/ef6af0ca-2803-42da-991f-fb732a7b3eda/crypt/zone/oxz_crucible_074083cb-db23-46c8-8ec1-8074ad72183f/root/var/svc/log/oxide-crucible-downstairs:default.log
12782024-10-09T23:23:37.886Zuploading: /pool/ext/ef6af0ca-2803-42da-991f-fb732a7b3eda/crypt/zone/oxz_crucible_074083cb-db23-46c8-8ec1-8074ad72183f/root/var/svc/log/oxide-crucible-downstairs:downstairs-c1171a08-abc1-45bf-b887-f2fc70943458.log (6077 bytes)
12792024-10-09T23:23:38.981Zuploaded: /pool/ext/ef6af0ca-2803-42da-991f-fb732a7b3eda/crypt/zone/oxz_crucible_074083cb-db23-46c8-8ec1-8074ad72183f/root/var/svc/log/oxide-crucible-downstairs:downstairs-c1171a08-abc1-45bf-b887-f2fc70943458.log
12802024-10-09T23:23:38.981Zuploading: /pool/ext/ef6af0ca-2803-42da-991f-fb732a7b3eda/crypt/zone/oxz_crucible_074083cb-db23-46c8-8ec1-8074ad72183f/root/var/svc/log/oxide-crucible-downstairs:downstairs-db8e06fe-289c-4e42-955e-f220a16e6cc9.log (5991 bytes)
12812024-10-09T23:23:40.076Zuploaded: /pool/ext/ef6af0ca-2803-42da-991f-fb732a7b3eda/crypt/zone/oxz_crucible_074083cb-db23-46c8-8ec1-8074ad72183f/root/var/svc/log/oxide-crucible-downstairs:downstairs-db8e06fe-289c-4e42-955e-f220a16e6cc9.log
12822024-10-09T23:23:40.076Zuploading: /pool/ext/ef6af0ca-2803-42da-991f-fb732a7b3eda/crypt/zone/oxz_crucible_074083cb-db23-46c8-8ec1-8074ad72183f/root/var/svc/log/oxide-crucible-downstairs:snapshot-c1171a08-abc1-45bf-b887-f2fc70943458-c171cea4-eebc-44fb-83e6-14ac51e1d1ff.log (5684 bytes)
12832024-10-09T23:23:41.171Zuploaded: /pool/ext/ef6af0ca-2803-42da-991f-fb732a7b3eda/crypt/zone/oxz_crucible_074083cb-db23-46c8-8ec1-8074ad72183f/root/var/svc/log/oxide-crucible-downstairs:snapshot-c1171a08-abc1-45bf-b887-f2fc70943458-c171cea4-eebc-44fb-83e6-14ac51e1d1ff.log
12842024-10-09T23:23:41.171Zuploading: /pool/ext/ef6af0ca-2803-42da-991f-fb732a7b3eda/crypt/zone/oxz_crucible_074083cb-db23-46c8-8ec1-8074ad72183f/root/var/svc/log/oxide-zone-network-setup:default.log (2081 bytes)
12852024-10-09T23:23:42.266Zuploaded: /pool/ext/ef6af0ca-2803-42da-991f-fb732a7b3eda/crypt/zone/oxz_crucible_074083cb-db23-46c8-8ec1-8074ad72183f/root/var/svc/log/oxide-zone-network-setup:default.log
12862024-10-09T23:23:42.266Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/log/chrony/measurements.log (103983 bytes)
12872024-10-09T23:23:42.472Zupload warning: file "/pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/log/chrony/measurements.log" changed size mid upload: 103983 -> 159742
12882024-10-09T23:23:43.543Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/log/chrony/measurements.log
12892024-10-09T23:23:43.543Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/log/chrony/statistics.log (76518 bytes)
12902024-10-09T23:23:43.703Zupload warning: file "/pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/log/chrony/statistics.log" changed size mid upload: 76518 -> 116064
12912024-10-09T23:23:44.775Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/log/chrony/statistics.log
12922024-10-09T23:23:44.775Zuploading: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/log/chrony/tracking.log (14521 bytes)
12932024-10-09T23:23:44.868Zupload warning: file "/pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/log/chrony/tracking.log" changed size mid upload: 14521 -> 20857
12942024-10-09T23:23:45.940Zuploaded: /pool/ext/2681938d-81e9-4d61-a343-32804acd18fc/crypt/zone/oxz_ntp_57919165-a8d1-491f-9bec-c44ee4ba4bd3/root/var/log/chrony/tracking.log