01J99YFM9YG3GD0RP9D2DM5TES: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log*",
        "%/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.33",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01J99YGJCNPATHKBR3XK0RGNHJ

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12024-10-03T20:14:53.647Zjob dependencies complete; ready to run (waiting for 31 m 48 s)
22024-10-03T20:24:15.507Zjob assigned to worker 01J9A0TGVYJ7DYEKCVYF744TNW (queued for 9 m 21 s)
32024-10-03T20:24:15.675Zhttp://172.20.0.5:9969/ipxe/buskin... ok
42024-10-03T20:24:15.728Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/kernel/amd64/unix... ok
52024-10-03T20:24:15.762Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive... ok
62024-10-03T20:24:15.817Zhttp://172.20.0.5:9969/os/buskin/platform/i86pc/amd64/boot_archive.hash... ok
72024-10-03T20:24:15.851Zhttp://172.20.0.5:9969/postboot/buskin... ok
82024-10-03T20:24:15.885ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
92024-10-03T20:24:15.918Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d5
102024-10-03T20:24:15.951ZOxide Helios Version helios-2.0.22827 64-bit
112024-10-03T20:24:15.985ZConfiguring devices.
122024-10-03T20:24:16.029Z##################################################
132024-10-03T20:24:16.063Z#### ############# ##
142024-10-03T20:24:16.096Z### ### ############ ##
152024-10-03T20:24:16.129Z## ### # ## ### ## Oxide Computer Company ##
162024-10-03T20:24:16.163Z## ## ## ### # ### ##
172024-10-03T20:24:16.196Z## # ### #### #### This Station Under ##
182024-10-03T20:24:16.230Z### ### #### # ### Computer Control ##
192024-10-03T20:24:16.461Z#### #### ### ## ##
202024-10-03T20:24:16.619Z##################################################
212024-10-03T20:24:16.686ZHostname: buskin
222024-10-03T20:24:16.719ZLoading smf(7) service descriptions:
232024-10-03T20:24:17.854Zdownloading input: /input/package/work/package.tar.gz
242024-10-03T20:29:00.466Zdownloaded input: /input/package/work/package.tar.gz
 
252024-10-03T20:29:00.466Zstarting task 0: "setup"
262024-10-03T20:29:00.491Z++ uname -s
272024-10-03T20:29:00.491Z+ kern=SunOS
282024-10-03T20:29:00.491Z+ case "$kern" in
292024-10-03T20:29:00.491Z+ groupadd -g 12345 build
302024-10-03T20:29:00.491Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
312024-10-03T20:29:02.414Z+ zfs create -o mountpoint=/work rpool/work
322024-10-03T20:29:02.439Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
332024-10-03T20:29:02.463Z+ home_fs=zfs
342024-10-03T20:29:02.463Z+ [[ zfs == autofs ]]
352024-10-03T20:29:02.463Z+ mkdir -p /home/build
362024-10-03T20:29:02.463Z+ chown build:build /home/build /work
372024-10-03T20:29:04.447Z+ chmod 0700 /home/build /work
382024-10-03T20:29:04.471Zprocess exited: duration 4055 ms, exit code 0
 
392024-10-03T20:29:04.519Zstarting task 1: "authentication"
402024-10-03T20:29:04.563Zprocess exited: duration 43 ms, exit code 0
 
412024-10-03T20:29:04.610Zstarting task 2: "build"
422024-10-03T20:29:04.633Z+ trap _exit_trap EXIT
432024-10-03T20:29:04.633Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
442024-10-03T20:29:04.730Zsecpolicy_fs_linkdir: 0x55 = 0x55
452024-10-03T20:29:04.730Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
462024-10-03T20:29:04.730Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
472024-10-03T20:29:04.730Zsecpolicy_fs_linkdir+3: 0 = 0xe5
482024-10-03T20:29:04.731Zsecpolicy_fs_linkdir+4: 0 = 0xb8
492024-10-03T20:29:04.731Zsecpolicy_fs_linkdir+5: 0 = 0x1
502024-10-03T20:29:04.731Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
512024-10-03T20:29:04.731Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
522024-10-03T20:29:04.754Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
532024-10-03T20:29:04.754Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
542024-10-03T20:29:04.754Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
552024-10-03T20:29:04.754Z+ [[ -d /opt/oxide ]]
562024-10-03T20:29:04.754Z+ pfexec mv /opt/oxide /opt/oxide-underneath
572024-10-03T20:29:04.754Z+ pfexec mkdir /opt/oxide
582024-10-03T20:29:04.754Z+ pfexec mount -F tmpfs -O swap /opt/oxide
592024-10-03T20:29:04.754Z+ [[ -d /opt/oxide-underneath ]]
602024-10-03T20:29:04.754Z+ cd /opt/oxide-underneath
612024-10-03T20:29:04.754Z+ pfexec tar ceEp@/f - .
622024-10-03T20:29:04.754Z+ cd /opt/oxide
632024-10-03T20:29:04.754Z+ pfexec tar xveEp@/f -
642024-10-03T20:29:04.861Zx ., 0 bytes, 0 tape blocks
652024-10-03T20:29:04.861Zx ./opte, 0 bytes, 0 tape blocks
662024-10-03T20:29:04.861Zx ./opte/bin, 0 bytes, 0 tape blocks
672024-10-03T20:29:04.862Zx ./opte/bin/opteadm, 52693888 bytes, 102918 tape blocks
682024-10-03T20:29:04.884Z+ pfexec rm -rf /opt/oxide-underneath
692024-10-03T20:29:04.884Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
702024-10-03T20:29:04.907Z+ pfexec mkdir /opt/oxide/work
712024-10-03T20:29:04.907Z+ pfexec chown build:build /opt/oxide/work
722024-10-03T20:29:04.907Z+ cd /opt/oxide/work
732024-10-03T20:29:04.907Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
742024-10-03T20:29:17.249ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
752024-10-03T20:29:17.250Zx out/target/test, 107 bytes, 1 tape blocks
762024-10-03T20:29:17.250Zx out/npuzone/npuzone, 23699288 bytes, 46288 tape blocks
772024-10-03T20:29:17.250Zx package-manifest.toml, 32084 bytes, 63 tape blocks
782024-10-03T20:29:17.250Zx smf/sled-agent/non-gimlet/config.toml, 3569 bytes, 7 tape blocks
792024-10-03T20:29:17.250Zx target/release/omicron-package, 33502272 bytes, 65435 tape blocks
802024-10-03T20:29:17.250Zx target/release/xtask, 12344728 bytes, 24111 tape blocks
812024-10-03T20:29:17.250Zx target/debug/bootstrap, 202109464 bytes, 394746 tape blocks
822024-10-03T20:29:17.250Zx tests/bootstrap-ed9ec115c9a0b0d5, 24349840 bytes, 47559 tape blocks
832024-10-03T20:29:17.250Zx tests/commtest-3e99aed6362922f5, 24582256 bytes, 48013 tape blocks
842024-10-03T20:29:17.250Zx tests/dhcp_server-a33e1c62b9efac3e, 24349856 bytes, 47559 tape blocks
852024-10-03T20:29:17.250Zx tests/end_to_end_tests-46e57f5259837a5f, 220011952 bytes, 429711 tape blocks
862024-10-03T20:29:17.250Zx out/clickhouse.tar.gz, 239314297 bytes, 467411 tape blocks
872024-10-03T20:29:17.250Zx out/clickhouse_keeper.tar.gz, 239286549 bytes, 467357 tape blocks
882024-10-03T20:29:17.250Zx out/clickhouse_server.tar.gz, 239286859 bytes, 467358 tape blocks
892024-10-03T20:29:17.250Zx out/cockroachdb.tar.gz, 135283429 bytes, 264226 tape blocks
902024-10-03T20:29:17.250Zx out/crucible-dtrace.tar, 83456 bytes, 163 tape blocks
912024-10-03T20:29:17.250Zx out/crucible-pantry-zone.tar.gz, 27572452 bytes, 53853 tape blocks
922024-10-03T20:29:17.250Zx out/crucible-zone.tar.gz, 35953237 bytes, 70222 tape blocks
932024-10-03T20:29:17.250Zx out/external-dns.tar.gz, 25776575 bytes, 50345 tape blocks
942024-10-03T20:29:17.250Zx out/internal-dns.tar.gz, 25776167 bytes, 50345 tape blocks
952024-10-03T20:29:17.250Zx out/mg-ddm-gz.tar, 63651840 bytes, 124320 tape blocks
962024-10-03T20:29:17.250Zx out/nexus.tar.gz, 97340911 bytes, 190119 tape blocks
972024-10-03T20:29:17.250Zx out/ntp.tar.gz, 9283067 bytes, 18131 tape blocks
982024-10-03T20:29:17.250Zx out/omicron-gateway-softnpu.tar.gz, 21514116 bytes, 42020 tape blocks
992024-10-03T20:29:17.250Zx out/omicron-sled-agent.tar, 104531456 bytes, 204163 tape blocks
1002024-10-03T20:29:17.250Zx out/overlay.tar.gz, 2622 bytes, 6 tape blocks
1012024-10-03T20:29:17.250Zx out/oximeter.tar.gz, 44679390 bytes, 87265 tape blocks
1022024-10-03T20:29:17.250Zx out/oxlog.tar, 3109376 bytes, 6073 tape blocks
1032024-10-03T20:29:17.250Zx out/probe.tar.gz, 2829470 bytes, 5527 tape blocks
1042024-10-03T20:29:17.250Zx out/propolis-server.tar.gz, 68085710 bytes, 132980 tape blocks
1052024-10-03T20:29:17.250Zx out/pumpkind-gz.tar, 27304960 bytes, 53330 tape blocks
1062024-10-03T20:29:17.250Zx out/switch-softnpu.tar.gz, 226139233 bytes, 441679 tape blocks
1072024-10-03T20:29:17.274Z
1082024-10-03T20:29:17.274Zreal 12.329112424
1092024-10-03T20:29:17.274Zuser 10.917162478
1102024-10-03T20:29:17.274Zsys 2.405102922
1112024-10-03T20:29:17.274Ztrap 0.000489971
1122024-10-03T20:29:17.274Ztflt 0.000000000
1132024-10-03T20:29:17.274Zdflt 0.000000000
1142024-10-03T20:29:17.274Zkflt 0.000000000
1152024-10-03T20:29:17.274Zlock 0.000000000
1162024-10-03T20:29:17.274Zslp 11.024714387
1172024-10-03T20:29:17.274Zlat 0.308613982
1182024-10-03T20:29:17.274Zstop 0.000134662
1192024-10-03T20:29:17.274Z+ bmat address ls
1202024-10-03T20:29:17.274ZNAME CIDR FIRST LAST COUNT
1212024-10-03T20:29:17.274Zextra 10.151.1.0/24 10.151.1.164 10.151.1.227 64
1222024-10-03T20:29:17.274Z++ bmat address ls -f extra -Ho first,last,count
1232024-10-03T20:29:17.274Z+ set -- 10.151.1.164 10.151.1.227 64
1242024-10-03T20:29:17.274Z+ EXTRA_IP_START=10.151.1.164
1252024-10-03T20:29:17.274Z+ EXTRA_IP_END=10.151.1.227
1262024-10-03T20:29:17.274Z+ EXTRA_IP_COUNT=64
1272024-10-03T20:29:17.274Z+ (( EXTRA_IP_COUNT >= 32 ))
1282024-10-03T20:29:17.274Z+ EXTRA_IP_BASE=10.151.1
1292024-10-03T20:29:17.274Z+ EXTRA_IP_FOCTET=164
1302024-10-03T20:29:17.274Z+ EXTRA_IP_LOCTET=227
1312024-10-03T20:29:17.274Z+ SERVICE_IP_POOL_START=10.151.1.164
1322024-10-03T20:29:17.274Z+ SERVICE_IP_POOL_END=10.151.1.173
1332024-10-03T20:29:17.274Z+ DNS_IP1=10.151.1.164
1342024-10-03T20:29:17.274Z+ DNS_IP2=10.151.1.165
1352024-10-03T20:29:17.274Z+ UPLINK_IP=10.151.1.174
1362024-10-03T20:29:17.274Z+ IPPOOL_START=10.151.1.175
1372024-10-03T20:29:17.274Z+ IPPOOL_END=10.151.1.227
1382024-10-03T20:29:17.274Z++ ipadm show-addr -po type,addr
1392024-10-03T20:29:17.274Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
1402024-10-03T20:29:17.275Z+ GATEWAY_IP=10.151.1.100
1412024-10-03T20:29:17.275Z+ [[ -n 10.151.1.100 ]]
1422024-10-03T20:29:17.275Z+ ping -s 10.151.1.100 56 1
1432024-10-03T20:29:17.275ZPING 10.151.1.100: 56 data bytes
1442024-10-03T20:29:17.275Z64 bytes from 10.151.1.100: icmp_seq=0. time=0.011 ms
1452024-10-03T20:29:17.275Z
1462024-10-03T20:29:17.275Z----10.151.1.100 PING Statistics----
1472024-10-03T20:29:17.275Z1 packets transmitted, 1 packets received, 0% packet loss
1482024-10-03T20:29:17.275Zround-trip (ms) min/avg/max/stddev = 0.011/0.011/0.011/-nan
1492024-10-03T20:29:17.275Z++ arp -an
1502024-10-03T20:29:17.275Z++ awk -vgw=10.151.1.100 '$2 == gw {print $NF}'
1512024-10-03T20:29:17.298Z+ GATEWAY_MAC=18:c0:4d:0d:9f:b2
1522024-10-03T20:29:17.298Z+ routeadm -e ipv4-forwarding -u
1532024-10-03T20:29:17.385Z+ PXA_START=10.151.1.164
1542024-10-03T20:29:17.385Z+ PXA_END=10.151.1.227
1552024-10-03T20:29:17.408Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
1562024-10-03T20:29:17.563Z+ ptime -m pfexec ./target/release/xtask virtual-hardware --vdev-dir /scratch create --gateway-ip 10.151.1.100 --gateway-mac 18:c0:4d:0d:9f:b2 --pxa-start 10.151.1.164 --pxa-end 10.151.1.227
1572024-10-03T20:29:17.587ZCouldn't find Cargo.toml, using /opt/oxide/work as workspace root
1582024-10-03T20:29:17.611Zcreating virtual hardware
1592024-10-03T20:29:17.611Zcreating /scratch/m2_0.vdev
1602024-10-03T20:29:17.611Zcreating /scratch/m2_1.vdev
1612024-10-03T20:29:17.611Zcreating /scratch/u2_0.vdev
1622024-10-03T20:29:17.611Zcreating /scratch/u2_1.vdev
1632024-10-03T20:29:17.611Zcreating /scratch/u2_2.vdev
1642024-10-03T20:29:17.611Zcreating /scratch/u2_3.vdev
1652024-10-03T20:29:17.611Zcreating /scratch/u2_4.vdev
1662024-10-03T20:29:17.611Zcreating /scratch/u2_5.vdev
1672024-10-03T20:29:17.611Zcreating /scratch/u2_6.vdev
1682024-10-03T20:29:17.611Zcreating /scratch/u2_7.vdev
1692024-10-03T20:29:17.611Zcreating /scratch/u2_8.vdev
1702024-10-03T20:29:17.657ZSimnet net0/sc0_0 exists
1712024-10-03T20:29:17.687ZSimnet net1/sc1_0 exists
1722024-10-03T20:29:17.710ZVnic sc0_1 exists
1732024-10-03T20:29:30.089ZUsing 10.151.1.100 as gateway ip
1742024-10-03T20:29:30.089Zusing 18:C0:4D:0D:9F:B2 as gateway mac
1752024-10-03T20:29:30.089Zconfiguring SoftNPU ARP entry
1762024-10-03T20:29:30.118Zconfiguring SoftNPU proxy ARP
1772024-10-03T20:29:30.173ZSoftNPU state:
1782024-10-03T20:29:30.173Z local v6:
1792024-10-03T20:29:30.173Z local v4:
1802024-10-03T20:29:30.173Z router v6:
1812024-10-03T20:29:30.173Z router v4_idx:
1822024-10-03T20:29:30.173Z router v4_routes:
1832024-10-03T20:29:30.173Z resolver v4:
1842024-10-03T20:29:30.173Z 10.151.1.100 -> 18:c0:4d:0d:9f:b2
1852024-10-03T20:29:30.173Z resolver v6:
1862024-10-03T20:29:30.173Z nat v4:
1872024-10-03T20:29:30.173Z nat_v6:
1882024-10-03T20:29:30.173Z port_mac:
1892024-10-03T20:29:30.173Z proxy arp:
1902024-10-03T20:29:30.173Z 10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
1912024-10-03T20:29:30.173Zcreated virtual hardware
1922024-10-03T20:29:30.197Z
1932024-10-03T20:29:30.197Zreal 12.644907154
1942024-10-03T20:29:30.197Zuser 0.667641721
1952024-10-03T20:29:30.197Zsys 3.297678442
1962024-10-03T20:29:30.197Ztrap 0.000082729
1972024-10-03T20:29:30.197Ztflt 0.000347481
1982024-10-03T20:29:30.197Zdflt 0.000367170
1992024-10-03T20:29:30.197Zkflt 0.000006690
2002024-10-03T20:29:30.197Zlock 9:45.699109953
2012024-10-03T20:29:30.197Zslp 38.131989709
2022024-10-03T20:29:30.197Zlat 0.077603229
2032024-10-03T20:29:30.197Zstop 0.359044465
2042024-10-03T20:29:30.197Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml pkg/config.toml
2052024-10-03T20:29:30.221Z+ sed -E -i~ 's/(m2|u2)(.*\.vdev)/\/scratch\/\1\2/g' pkg/config.toml
2062024-10-03T20:29:30.221Z+ diff -u pkg/config.toml~ pkg/config.toml
2072024-10-03T20:29:30.221Z--- pkg/config.toml~ Mon Jul 24 01:21:28 2006
2082024-10-03T20:29:30.221Z+++ pkg/config.toml Thu Oct 3 20:29:30 2024
2092024-10-03T20:29:30.221Z@@ -29,18 +29,18 @@
2102024-10-03T20:29:30.221Z # These paths have the prefix of either "u2" or "m2", followed by an underscore,
2112024-10-03T20:29:30.221Z # followed by a string that is embedded into their fake serial values.
2122024-10-03T20:29:30.221Z vdevs = [
2132024-10-03T20:29:30.221Z- "m2_0.vdev",
2142024-10-03T20:29:30.221Z- "m2_1.vdev",
2152024-10-03T20:29:30.221Z+ "/scratch/m2_0.vdev",
2162024-10-03T20:29:30.221Z+ "/scratch/m2_1.vdev",
2172024-10-03T20:29:30.221Z
2182024-10-03T20:29:30.221Z- "u2_0.vdev",
2192024-10-03T20:29:30.221Z- "u2_1.vdev",
2202024-10-03T20:29:30.221Z- "u2_2.vdev",
2212024-10-03T20:29:30.221Z- "u2_3.vdev",
2222024-10-03T20:29:30.221Z- "u2_4.vdev",
2232024-10-03T20:29:30.221Z- "u2_5.vdev",
2242024-10-03T20:29:30.221Z- "u2_6.vdev",
2252024-10-03T20:29:30.221Z- "u2_7.vdev",
2262024-10-03T20:29:30.221Z- "u2_8.vdev",
2272024-10-03T20:29:30.221Z+ "/scratch/u2_0.vdev",
2282024-10-03T20:29:30.221Z+ "/scratch/u2_1.vdev",
2292024-10-03T20:29:30.221Z+ "/scratch/u2_2.vdev",
2302024-10-03T20:29:30.221Z+ "/scratch/u2_3.vdev",
2312024-10-03T20:29:30.221Z+ "/scratch/u2_4.vdev",
2322024-10-03T20:29:30.221Z+ "/scratch/u2_5.vdev",
2332024-10-03T20:29:30.221Z+ "/scratch/u2_6.vdev",
2342024-10-03T20:29:30.221Z+ "/scratch/u2_7.vdev",
2352024-10-03T20:29:30.221Z+ "/scratch/u2_8.vdev",
2362024-10-03T20:29:30.221Z ]
2372024-10-03T20:29:30.221Z
2382024-10-03T20:29:30.222Z # Percentage of usable physical DRAM to use for the VMM reservoir, which
2392024-10-03T20:29:30.222Z+ true
2402024-10-03T20:29:30.222Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
2412024-10-03T20:29:30.222Z+ SILO_NAME=recovery
2422024-10-03T20:29:30.222Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
2432024-10-03T20:29:30.222Z+ EXTERNAL_DNS_DOMAIN=oxide.test
2442024-10-03T20:29:30.222Z+ sed -i~ '
2452024-10-03T20:29:30.222Z /^external_dns_ips/c\
2462024-10-03T20:29:30.222Zexternal_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
2472024-10-03T20:29:30.222Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
2482024-10-03T20:29:30.222Z /^first/c\
2492024-10-03T20:29:30.222Zfirst = "10.151.1.164"
2502024-10-03T20:29:30.222Z /^last/c\
2512024-10-03T20:29:30.222Zlast = "10.151.1.173"
2522024-10-03T20:29:30.222Z }
2532024-10-03T20:29:30.222Z /^infra_ip_first/c\
2542024-10-03T20:29:30.222Zinfra_ip_first = "10.151.1.174"
2552024-10-03T20:29:30.222Z /^infra_ip_last/c\
2562024-10-03T20:29:30.222Zinfra_ip_last = "10.151.1.174"
2572024-10-03T20:29:30.222Z /^\[\[rack_network_config.ports/,/^$/ {
2582024-10-03T20:29:30.222Z /^routes/c\
2592024-10-03T20:29:30.222Zroutes = \[{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}\]
2602024-10-03T20:29:30.222Z /^addresses/c\
2612024-10-03T20:29:30.222Zaddresses = \[{address = "10.151.1.174/24"} \]
2622024-10-03T20:29:30.222Z }
2632024-10-03T20:29:30.222Z' pkg/config-rss.toml
2642024-10-03T20:29:30.246Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
2652024-10-03T20:29:30.246Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
2662024-10-03T20:29:30.246Z+++ pkg/config-rss.toml Thu Oct 3 20:29:30 2024
2672024-10-03T20:29:30.246Z@@ -33,7 +33,7 @@
2682024-10-03T20:29:30.246Z # the DNS domain delegated to the rack by the customer. Each of these addresses
2692024-10-03T20:29:30.246Z # must be contained in one of the "internal services" IP Pool ranges listed
2702024-10-03T20:29:30.246Z # below.
2712024-10-03T20:29:30.246Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
2722024-10-03T20:29:30.246Z+external_dns_ips = [ "10.151.1.164", "10.151.1.165" ]
2732024-10-03T20:29:30.246Z
2742024-10-03T20:29:30.246Z # Initial TLS certificates for the external API
2752024-10-03T20:29:30.246Z #
2762024-10-03T20:29:30.246Z@@ -69,8 +69,8 @@
2772024-10-03T20:29:30.246Z #
2782024-10-03T20:29:30.246Z # For more on this and what to put here, see docs/how-to-run.adoc.
2792024-10-03T20:29:30.246Z [[internal_services_ip_pool_ranges]]
2802024-10-03T20:29:30.246Z-first = "192.168.1.20"
2812024-10-03T20:29:30.246Z-last = "192.168.1.29"
2822024-10-03T20:29:30.246Z+first = "10.151.1.164"
2832024-10-03T20:29:30.246Z+last = "10.151.1.173"
2842024-10-03T20:29:30.246Z
2852024-10-03T20:29:30.246Z # TODO - this configuration is subject to change going forward. Ultimately these
2862024-10-03T20:29:30.247Z # parameters should be provided to the control plane via wicket, but we need to
2872024-10-03T20:29:30.247Z@@ -91,8 +91,8 @@
2882024-10-03T20:29:30.247Z # A range of IP addresses used by Boundary Services on the external network. In
2892024-10-03T20:29:30.247Z # a real system, these would be addresses of the uplink ports on the Sidecar.
2902024-10-03T20:29:30.247Z # With softnpu, only one address is used.
2912024-10-03T20:29:30.247Z-infra_ip_first = "192.168.1.30"
2922024-10-03T20:29:30.247Z-infra_ip_last = "192.168.1.30"
2932024-10-03T20:29:30.247Z+infra_ip_first = "10.151.1.174"
2942024-10-03T20:29:30.247Z+infra_ip_last = "10.151.1.174"
2952024-10-03T20:29:30.247Z
2962024-10-03T20:29:30.247Z # Configurations for BGP routers to run on the scrimlets.
2972024-10-03T20:29:30.247Z bgp = []
2982024-10-03T20:29:30.247Z@@ -100,9 +100,9 @@
2992024-10-03T20:29:30.247Z # You can configure multiple uplinks by repeating the following stanza
3002024-10-03T20:29:30.247Z [[rack_network_config.ports]]
3012024-10-03T20:29:30.247Z # Routes associated with this port.
3022024-10-03T20:29:30.247Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
3032024-10-03T20:29:30.247Z+routes = [{nexthop = "10.151.1.100", destination = "0.0.0.0/0"}]
3042024-10-03T20:29:30.247Z # Addresses associated with this port.
3052024-10-03T20:29:30.247Z-addresses = [{address = "192.168.1.30/24"}]
3062024-10-03T20:29:30.247Z+addresses = [{address = "10.151.1.174/24"} ]
3072024-10-03T20:29:30.247Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
3082024-10-03T20:29:30.247Z port = "qsfp0"
3092024-10-03T20:29:30.247Z # The speed of this port.
3102024-10-03T20:29:30.247Z+ true
3112024-10-03T20:29:30.247Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml pkg/config.toml
3122024-10-03T20:29:30.247Za pkg/config-rss.toml 8K
3132024-10-03T20:29:30.247Za pkg/config.toml 4K
3142024-10-03T20:29:30.247Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~ pkg/config.toml pkg/config.toml~
3152024-10-03T20:29:30.247Z+ TLS_NAME=recovery.sys.oxide.test
3162024-10-03T20:29:30.247Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
3172024-10-03T20:29:31.655Z.+.......+..+...+.+...........+.+..+.+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+...........+...+.+...+.....+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.....+......+........+.........+...+....+...+..+......+............+...+............+.+..+......................+......+......+.....+.+...+.....+.........+......+....+..................+..+...+.........+.............+........+.......+..+.........+.........+...+......................+..+......+.......+......+..............+...+.......+...+...+....................+.+........+............+...+..........+...........+......+.+..+............+......+..................+.+.....+.........+.+.....+.+...+.....+......+...+......+....+...............+..+...+................+............+...+..............+.......+......+..+......+.+........+.............+.....+............+.......+..................+...+.......................+......+...............+.+............+..+......+...............+.........+...+.......+...+...+..+.......+.........+...............+..+...+...............+.+......+............+......+.........+..+....+..+....+......+.....+..................+............+...+.......+..+............+......+.......+.....+...............+................+..+....+........+.+...........+.+........+.......+..+...+..........+.....+............+..........+..+......+...+....+........+....+......+..+.......+...........+.........+.+.....+..........+...+.....+............+.......+...........+....+........+....+.....+.+.................+.......+...+........+...+.+..............+....+..+.+.................................+..+...+.+.....+...............+....+..+.........+.+........+....+..+.+........+......+....+......+.........+..+....+...........+...+......+...+.............+.........+...........+....+...+.................+.....................+..........+...+.....+....+..+....+..............+..........+..+.+.....+.......+..+.............+........+.........................+..+.............+..................+..+............................+............+...+......+.....+........................+.+...............+........+...+....+........+..........+....................+.+..+....+.....+....+......+........+.+.....+....+..................+...+..............+.......+.....+.+.................+................+.....+.........+...............+.......+........+...+...+....+...+.....+...+..........+.........+...+...+.........+.....+...+.............+......+...+...........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3182024-10-03T20:29:32.076Z....+....+......+.......................+.........+.+.........+.....+.+.....+.+......+.....+.......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+..........+...........+....+..............+.......+........+.+..+...+.+...+..+.........+......+.+........+....+...+........+.+...+...............+...+.....+.......+........+...+.+........+............+.+..+.+......+......+..+......+......+...+............+.+.....+...+....+..+....+......+...+......+......+......+..+...+...+....+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+........+..........+...+.........+.....+...+.......+.....+..................+......+.......+..............+.+...+......+...........+.......+...+...+..+...+........................+.+..+.........+...+...+.........+....+..+...+....+.....+.......+.....+....+..+...+....+...........+.......+........+......+.+.....+.........+..........+..............+...............................+....................+....+......+.....+....+...........+......+..........+.....+...+....+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3192024-10-03T20:29:32.115Z-----
3202024-10-03T20:29:32.116Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
3212024-10-03T20:29:32.116Za pkg/initial-tls-cert.pem 2K
3222024-10-03T20:29:32.116Za pkg/initial-tls-key.pem 4K
3232024-10-03T20:29:32.116Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
3242024-10-03T20:29:32.116Z+ rmdir pkg
3252024-10-03T20:29:32.116Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
3262024-10-03T20:29:32.116Z+ pfexec mkdir -p /usr/oxide
3272024-10-03T20:29:32.116Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0
3282024-10-03T20:29:32.782Z+ pfexec chmod +x /usr/oxide/oxide
3292024-10-03T20:29:32.805Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
3302024-10-03T20:29:51.069Z+ OMICRON_NO_UNINSTALL=1
3312024-10-03T20:29:51.069Z+ ptime -m pfexec ./target/release/omicron-package -t test install
3322024-10-03T20:29:51.093ZLogging to: /opt/oxide/work/out/LOG
3332024-10-03T20:29:51.674Z
3342024-10-03T20:29:51.698Zreal 0.600250392
3352024-10-03T20:29:51.698Zuser 0.133476148
3362024-10-03T20:29:51.698Zsys 3.145545971
3372024-10-03T20:29:51.699Ztrap 0.000034838
3382024-10-03T20:29:51.699Ztflt 0.000004989
3392024-10-03T20:29:51.699Zdflt 0.000631024
3402024-10-03T20:29:51.699Zkflt 0.000000000
3412024-10-03T20:29:51.699Zlock 53.956518743
3422024-10-03T20:29:51.699Zslp 0.787793979
3432024-10-03T20:29:51.699Zlat 0.002961976
3442024-10-03T20:29:51.699Zstop 0.198947612
3452024-10-03T20:29:51.699Z+ retry=0
3462024-10-03T20:29:51.699Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3472024-10-03T20:29:51.699Z+ [[ 0 -gt 30 ]]
3482024-10-03T20:29:51.699Z+ sleep 1
3492024-10-03T20:29:52.690Z+ retry=1
3502024-10-03T20:29:52.690Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3512024-10-03T20:29:52.716Z+ [[ 1 -gt 30 ]]
3522024-10-03T20:29:52.716Z+ sleep 1
3532024-10-03T20:29:53.704Z+ retry=2
3542024-10-03T20:29:53.704Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3552024-10-03T20:29:53.727Z+ [[ 2 -gt 30 ]]
3562024-10-03T20:29:53.727Z+ sleep 1
3572024-10-03T20:29:54.718Z+ retry=3
3582024-10-03T20:29:54.718Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3592024-10-03T20:30:14.675Z+ [[ 3 -gt 30 ]]
3602024-10-03T20:30:14.676Z+ sleep 1
3612024-10-03T20:30:15.679Z+ retry=4
3622024-10-03T20:30:15.679Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3632024-10-03T20:30:15.703Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
3642024-10-03T20:30:15.727Zlocal v6:
3652024-10-03T20:30:15.727Zfd2f:6b72:1710:42ad::1
3662024-10-03T20:30:15.727Zfe80::5465:d0ff:fe61:c71b
3672024-10-03T20:30:15.727Zlocal v4:
3682024-10-03T20:30:15.727Z10.151.1.174
3692024-10-03T20:30:15.727Zrouter v6:
3702024-10-03T20:30:15.727Zrouter v4_idx:
3712024-10-03T20:30:15.727Zrouter v4_routes:
3722024-10-03T20:30:15.727Zresolver v4:
3732024-10-03T20:30:15.727Z10.151.1.100 -> 18:c0:4d:0d:9f:b2
3742024-10-03T20:30:15.727Zresolver v6:
3752024-10-03T20:30:15.727Zfe80::8062:7aff:fe9a:bfc8 -> 82:62:7a:9a:bf:c8
3762024-10-03T20:30:15.727Zfe80::5465:d0ff:fe61:c71b -> 56:65:d0:61:c7:1b
3772024-10-03T20:30:15.727Znat v4:
3782024-10-03T20:30:15.727Znat_v6:
3792024-10-03T20:30:15.728Zport_mac:
3802024-10-03T20:30:15.728Z1000: a8:40:25:49:47:08
3812024-10-03T20:30:15.728Z1: a8:40:25:49:47:09
3822024-10-03T20:30:15.728Z2: a8:40:25:49:47:29
3832024-10-03T20:30:15.728Zproxy arp:
3842024-10-03T20:30:15.728Z10.151.1.164/10.151.1.227: a8:e1:de:01:70:1d
3852024-10-03T20:30:15.728Z+ retry=0
3862024-10-03T20:30:15.728Z+++ zoneadm list -n
3872024-10-03T20:30:15.752Z+++ grep oxz_ntp
3882024-10-03T20:30:15.752Z++ pfexec svcs -z -Hostate oxide/ntp
3892024-10-03T20:30:15.752Zsvcs: zone '-Hostate': entity not found
3902024-10-03T20:30:15.752Z++ true
3912024-10-03T20:30:15.752Z+ [[ '' != online ]]
3922024-10-03T20:30:15.752Z+ [[ 0 -gt 60 ]]
3932024-10-03T20:30:15.752Z+ sleep 1
3942024-10-03T20:30:16.744Z+ retry=1
3952024-10-03T20:30:16.769Z+++ zoneadm list -n
3962024-10-03T20:30:16.769Z+++ grep oxz_ntp
3972024-10-03T20:30:16.769Z++ pfexec svcs -z -Hostate oxide/ntp
3982024-10-03T20:30:16.769Zsvcs: zone '-Hostate': entity not found
3992024-10-03T20:30:16.769Z++ true
4002024-10-03T20:30:16.769Z+ [[ '' != online ]]
4012024-10-03T20:30:16.769Z+ [[ 1 -gt 60 ]]
4022024-10-03T20:30:16.769Z+ sleep 1
4032024-10-03T20:30:17.766Z+ retry=2
4042024-10-03T20:30:17.790Z+++ zoneadm list -n
4052024-10-03T20:30:17.790Z+++ grep oxz_ntp
4062024-10-03T20:30:17.790Z++ pfexec svcs -z -Hostate oxide/ntp
4072024-10-03T20:30:17.790Zsvcs: zone '-Hostate': entity not found
4082024-10-03T20:30:17.790Z++ true
4092024-10-03T20:30:17.790Z+ [[ '' != online ]]
4102024-10-03T20:30:17.790Z+ [[ 2 -gt 60 ]]
4112024-10-03T20:30:17.790Z+ sleep 1
4122024-10-03T20:30:18.793Z+ retry=3
4132024-10-03T20:30:18.817Z+++ zoneadm list -n
4142024-10-03T20:30:18.817Z+++ grep oxz_ntp
4152024-10-03T20:30:18.817Z++ pfexec svcs -z oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d -Hostate oxide/ntp
4162024-10-03T20:30:18.841Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
4172024-10-03T20:30:18.841Z++ true
4182024-10-03T20:30:18.841Z+ [[ '' != online ]]
4192024-10-03T20:30:18.841Z+ [[ 3 -gt 60 ]]
4202024-10-03T20:30:18.841Z+ sleep 1
4212024-10-03T20:30:19.838Z+ retry=4
4222024-10-03T20:30:19.861Z+++ zoneadm list -n
4232024-10-03T20:30:19.861Z+++ grep oxz_ntp
4242024-10-03T20:30:19.861Z++ pfexec svcs -z oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d -Hostate oxide/ntp
4252024-10-03T20:30:19.884Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
4262024-10-03T20:30:19.884Z++ true
4272024-10-03T20:30:19.884Z+ [[ '' != online ]]
4282024-10-03T20:30:19.884Z+ [[ 4 -gt 60 ]]
4292024-10-03T20:30:19.884Z+ sleep 1
4302024-10-03T20:30:20.869Z+ retry=5
4312024-10-03T20:30:20.892Z+++ zoneadm list -n
4322024-10-03T20:30:20.892Z+++ grep oxz_ntp
4332024-10-03T20:30:20.892Z++ pfexec svcs -z oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d -Hostate oxide/ntp
4342024-10-03T20:30:20.915Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
4352024-10-03T20:30:20.915Z++ true
4362024-10-03T20:30:20.915Z+ [[ '' != online ]]
4372024-10-03T20:30:20.915Z+ [[ 5 -gt 60 ]]
4382024-10-03T20:30:20.915Z+ sleep 1
4392024-10-03T20:30:21.899Z+ retry=6
4402024-10-03T20:30:21.922Z+++ zoneadm list -n
4412024-10-03T20:30:21.922Z+++ grep oxz_ntp
4422024-10-03T20:30:21.922Z++ pfexec svcs -z oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d -Hostate oxide/ntp
4432024-10-03T20:30:21.945Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
4442024-10-03T20:30:21.945Z++ true
4452024-10-03T20:30:21.945Z+ [[ '' != online ]]
4462024-10-03T20:30:21.945Z+ [[ 6 -gt 60 ]]
4472024-10-03T20:30:21.945Z+ sleep 1
4482024-10-03T20:30:22.929Z+ retry=7
4492024-10-03T20:30:22.952Z+++ zoneadm list -n
4502024-10-03T20:30:22.952Z+++ grep oxz_ntp
4512024-10-03T20:30:22.952Z++ pfexec svcs -z oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d -Hostate oxide/ntp
4522024-10-03T20:30:22.952Z+ [[ offline != online ]]
4532024-10-03T20:30:22.952Z+ [[ 7 -gt 60 ]]
4542024-10-03T20:30:22.952Z+ sleep 1
4552024-10-03T20:30:23.951Z+ retry=8
4562024-10-03T20:30:23.975Z+++ zoneadm list -n
4572024-10-03T20:30:23.975Z+++ grep oxz_ntp
4582024-10-03T20:30:23.975Z++ pfexec svcs -z oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d -Hostate oxide/ntp
4592024-10-03T20:30:23.975Z+ [[ offline != online ]]
4602024-10-03T20:30:23.975Z+ [[ 8 -gt 60 ]]
4612024-10-03T20:30:23.975Z+ sleep 1
4622024-10-03T20:30:24.971Z+ retry=9
4632024-10-03T20:30:24.995Z+++ zoneadm list -n
4642024-10-03T20:30:24.995Z+++ grep oxz_ntp
4652024-10-03T20:30:24.995Z++ pfexec svcs -z oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d -Hostate oxide/ntp
4662024-10-03T20:30:24.995Z+ [[ offline != online ]]
4672024-10-03T20:30:24.995Z+ [[ 9 -gt 60 ]]
4682024-10-03T20:30:24.995Z+ sleep 1
4692024-10-03T20:30:25.993Z+ retry=10
4702024-10-03T20:30:26.016Z+++ zoneadm list -n
4712024-10-03T20:30:26.017Z+++ grep oxz_ntp
4722024-10-03T20:30:26.017Z++ pfexec svcs -z oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d -Hostate oxide/ntp
4732024-10-03T20:30:26.017Z+ [[ offline != online ]]
4742024-10-03T20:30:26.017Z+ [[ 10 -gt 60 ]]
4752024-10-03T20:30:26.017Z+ sleep 1
4762024-10-03T20:30:27.015Z+ retry=11
4772024-10-03T20:30:27.039Z+++ zoneadm list -n
4782024-10-03T20:30:27.039Z+++ grep oxz_ntp
4792024-10-03T20:30:27.039Z++ pfexec svcs -z oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d -Hostate oxide/ntp
4802024-10-03T20:30:27.039Z+ [[ offline != online ]]
4812024-10-03T20:30:27.039Z+ [[ 11 -gt 60 ]]
4822024-10-03T20:30:27.039Z+ sleep 1
4832024-10-03T20:30:28.039Z+ retry=12
4842024-10-03T20:30:28.062Z+++ zoneadm list -n
4852024-10-03T20:30:28.062Z+++ grep oxz_ntp
4862024-10-03T20:30:28.062Z++ pfexec svcs -z oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d -Hostate oxide/ntp
4872024-10-03T20:30:28.062Z+ [[ offline != online ]]
4882024-10-03T20:30:28.062Z+ [[ 12 -gt 60 ]]
4892024-10-03T20:30:28.062Z+ sleep 1
 
4902024-10-03T20:30:28.090Zbuskin console login: Oct 3 20:30:27 buskin ip: ip: joining multicasts failed (4) on opte0 - will use link layer broadcasts for multicast
 
4912024-10-03T20:30:29.062Z+ retry=13
4922024-10-03T20:30:29.085Z+++ zoneadm list -n
4932024-10-03T20:30:29.085Z+++ grep oxz_ntp
4942024-10-03T20:30:29.085Z++ pfexec svcs -z oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d -Hostate oxide/ntp
4952024-10-03T20:30:29.085Z+ [[ online != online ]]
4962024-10-03T20:30:29.085Z+ echo 'Waited for chrony: 13s'
4972024-10-03T20:30:29.085ZWaited for chrony: 13s
4982024-10-03T20:30:29.085Z+ retry=0
4992024-10-03T20:30:29.085Z+ zoneadm list
5002024-10-03T20:30:29.108Z+ grep nexus
5012024-10-03T20:30:29.108Z+ [[ 0 -gt 300 ]]
5022024-10-03T20:30:29.108Z+ sleep 1
5032024-10-03T20:30:30.093Z+ retry=1
5042024-10-03T20:30:30.119Z+ zoneadm list
5052024-10-03T20:30:30.119Z+ grep nexus
5062024-10-03T20:30:30.119Z+ [[ 1 -gt 300 ]]
5072024-10-03T20:30:30.119Z+ sleep 1
5082024-10-03T20:30:31.107Z+ retry=2
5092024-10-03T20:30:31.131Z+ zoneadm list
5102024-10-03T20:30:31.131Z+ grep nexus
5112024-10-03T20:30:31.131Z+ [[ 2 -gt 300 ]]
5122024-10-03T20:30:31.131Z+ sleep 1
5132024-10-03T20:30:32.122Z+ retry=3
5142024-10-03T20:30:32.145Z+ zoneadm list
5152024-10-03T20:30:32.145Z+ grep nexus
5162024-10-03T20:30:32.145Z+ [[ 3 -gt 300 ]]
5172024-10-03T20:30:32.145Z+ sleep 1
5182024-10-03T20:30:33.136Z+ retry=4
5192024-10-03T20:30:33.160Z+ zoneadm list
5202024-10-03T20:30:33.160Z+ grep nexus
5212024-10-03T20:30:33.160Z+ [[ 4 -gt 300 ]]
5222024-10-03T20:30:33.160Z+ sleep 1
5232024-10-03T20:30:34.151Z+ retry=5
5242024-10-03T20:30:34.174Z+ zoneadm list
5252024-10-03T20:30:34.174Z+ grep nexus
5262024-10-03T20:30:34.174Z+ [[ 5 -gt 300 ]]
5272024-10-03T20:30:34.174Z+ sleep 1
5282024-10-03T20:30:35.163Z+ retry=6
5292024-10-03T20:30:35.186Z+ zoneadm list
5302024-10-03T20:30:35.186Z+ grep nexus
5312024-10-03T20:30:35.186Z+ [[ 6 -gt 300 ]]
5322024-10-03T20:30:35.186Z+ sleep 1
5332024-10-03T20:30:36.180Z+ retry=7
5342024-10-03T20:30:36.205Z+ grep nexus
5352024-10-03T20:30:36.205Z+ zoneadm list
5362024-10-03T20:30:36.205Z+ [[ 7 -gt 300 ]]
5372024-10-03T20:30:36.205Z+ sleep 1
5382024-10-03T20:30:37.252Z+ retry=8
5392024-10-03T20:30:37.289Z+ + grep nexus
5402024-10-03T20:30:37.289Zzoneadm list
5412024-10-03T20:30:37.323Z+ [[ 8 -gt 300 ]]
5422024-10-03T20:30:37.323Z+ sleep 1
5432024-10-03T20:30:38.330Z+ retry=9
5442024-10-03T20:30:38.353Z+ zoneadm list
5452024-10-03T20:30:38.353Z+ grep nexus
5462024-10-03T20:30:38.353Z+ [[ 9 -gt 300 ]]
5472024-10-03T20:30:38.353Z+ sleep 1
5482024-10-03T20:30:39.341Z+ retry=10
5492024-10-03T20:30:39.364Z+ zoneadm list
5502024-10-03T20:30:39.364Z+ grep nexus
5512024-10-03T20:30:39.364Z+ [[ 10 -gt 300 ]]
5522024-10-03T20:30:39.365Z+ sleep 1
5532024-10-03T20:30:40.354Z+ retry=11
5542024-10-03T20:30:40.378Z+ zoneadm list
5552024-10-03T20:30:40.378Z+ grep nexus
5562024-10-03T20:30:40.378Z+ [[ 11 -gt 300 ]]
5572024-10-03T20:30:40.378Z+ sleep 1
5582024-10-03T20:30:41.366Z+ retry=12
5592024-10-03T20:30:41.389Z+ zoneadm list
5602024-10-03T20:30:41.389Z+ grep nexus
5612024-10-03T20:30:41.389Z+ [[ 12 -gt 300 ]]
5622024-10-03T20:30:41.390Z+ sleep 1
5632024-10-03T20:30:42.377Z+ retry=13
5642024-10-03T20:30:42.400Z+ zoneadm list
5652024-10-03T20:30:42.400Z+ grep nexus
5662024-10-03T20:30:42.400Z+ [[ 13 -gt 300 ]]
5672024-10-03T20:30:42.400Z+ sleep 1
5682024-10-03T20:30:43.389Z+ retry=14
5692024-10-03T20:30:43.412Z+ zoneadm list
5702024-10-03T20:30:43.412Z+ grep nexus
5712024-10-03T20:30:43.412Z+ [[ 14 -gt 300 ]]
5722024-10-03T20:30:43.412Z+ sleep 1
5732024-10-03T20:30:44.402Z+ retry=15
5742024-10-03T20:30:44.425Z+ zoneadm list
5752024-10-03T20:30:44.425Z+ grep nexus
5762024-10-03T20:30:44.425Z+ [[ 15 -gt 300 ]]
5772024-10-03T20:30:44.425Z+ sleep 1
5782024-10-03T20:30:45.446Z+ retry=16
5792024-10-03T20:30:45.470Z+ zoneadm list
5802024-10-03T20:30:45.470Z+ grep nexus
5812024-10-03T20:30:45.470Z+ [[ 16 -gt 300 ]]
5822024-10-03T20:30:45.470Z+ sleep 1
5832024-10-03T20:30:46.471Z+ retry=17
5842024-10-03T20:30:46.495Z+ zoneadm list
5852024-10-03T20:30:46.495Z+ grep nexus
5862024-10-03T20:30:46.495Z+ [[ 17 -gt 300 ]]
5872024-10-03T20:30:46.495Z+ sleep 1
5882024-10-03T20:30:47.485Z+ retry=18
5892024-10-03T20:30:47.508Z+ zoneadm list
5902024-10-03T20:30:47.508Z+ grep nexus
5912024-10-03T20:30:47.508Z+ [[ 18 -gt 300 ]]
5922024-10-03T20:30:47.508Z+ sleep 1
5932024-10-03T20:30:48.500Z+ retry=19
5942024-10-03T20:30:48.524Z+ zoneadm list
5952024-10-03T20:30:48.524Z+ grep nexus
5962024-10-03T20:30:48.524Z+ [[ 19 -gt 300 ]]
5972024-10-03T20:30:48.524Z+ sleep 1
5982024-10-03T20:30:49.513Z+ retry=20
5992024-10-03T20:30:49.536Z+ zoneadm list
6002024-10-03T20:30:49.536Z+ grep nexus
6012024-10-03T20:30:49.536Z+ [[ 20 -gt 300 ]]
6022024-10-03T20:30:49.536Z+ sleep 1
6032024-10-03T20:30:50.526Z+ retry=21
6042024-10-03T20:30:50.549Z+ zoneadm list
6052024-10-03T20:30:50.549Z+ grep nexus
6062024-10-03T20:30:50.549Z+ [[ 21 -gt 300 ]]
6072024-10-03T20:30:50.549Z+ sleep 1
6082024-10-03T20:30:51.537Z+ retry=22
6092024-10-03T20:30:51.560Z+ zoneadm list
6102024-10-03T20:30:51.560Z+ grep nexus
6112024-10-03T20:30:51.560Z+ [[ 22 -gt 300 ]]
6122024-10-03T20:30:51.560Z+ sleep 1
6132024-10-03T20:30:52.549Z+ retry=23
6142024-10-03T20:30:52.572Z+ zoneadm list
6152024-10-03T20:30:52.572Z+ grep nexus
6162024-10-03T20:30:52.572Z+ [[ 23 -gt 300 ]]
6172024-10-03T20:30:52.572Z+ sleep 1
6182024-10-03T20:30:53.562Z+ retry=24
6192024-10-03T20:30:53.585Z+ zoneadm list
6202024-10-03T20:30:53.585Z+ grep nexus
6212024-10-03T20:30:53.585Z+ [[ 24 -gt 300 ]]
6222024-10-03T20:30:53.585Z+ sleep 1
6232024-10-03T20:30:54.574Z+ retry=25
6242024-10-03T20:30:54.598Z+ zoneadm list
6252024-10-03T20:30:54.598Z+ grep nexus
6262024-10-03T20:30:54.598Z+ [[ 25 -gt 300 ]]
6272024-10-03T20:30:54.598Z+ sleep 1
6282024-10-03T20:30:55.588Z+ retry=26
6292024-10-03T20:30:55.611Z+ zoneadm list
6302024-10-03T20:30:55.611Z+ grep nexus
6312024-10-03T20:30:55.611Z+ [[ 26 -gt 300 ]]
6322024-10-03T20:30:55.611Z+ sleep 1
6332024-10-03T20:30:56.724Z+ retry=27
6342024-10-03T20:30:56.799Z+ zoneadm list
6352024-10-03T20:30:56.824Z+ grep nexus
6362024-10-03T20:30:56.824Z+ [[ 27 -gt 300 ]]
6372024-10-03T20:30:56.824Z+ sleep 1
6382024-10-03T20:31:02.063Z+ retry=28
6392024-10-03T20:31:02.291Z+ zoneadm list
6402024-10-03T20:31:02.333Z+ grep nexus
6412024-10-03T20:31:02.414Z+ [[ 28 -gt 300 ]]
6422024-10-03T20:31:02.414Z+ sleep 1
 
6432024-10-03T20:31:02.733ZOct 3 20:31:02 buskin xde: WARNING: failed to generate descriptor for stateful action: ICMP:172.30.3.5:0:168.61.215.74:0 Unexpected { msg: "Expected ICMP Echo Request, found: destination unreachable" }
 
6442024-10-03T20:31:03.420Z+ retry=29
6452024-10-03T20:31:03.443Z+ zoneadm list
6462024-10-03T20:31:03.443Z+ grep nexus
6472024-10-03T20:31:03.443Z+ [[ 29 -gt 300 ]]
6482024-10-03T20:31:03.443Z+ sleep 1
6492024-10-03T20:31:04.436Z+ retry=30
6502024-10-03T20:31:04.459Z+ zoneadm list
6512024-10-03T20:31:04.459Z+ grep nexus
6522024-10-03T20:31:04.459Z+ [[ 30 -gt 300 ]]
6532024-10-03T20:31:04.459Z+ sleep 1
6542024-10-03T20:31:05.449Z+ retry=31
6552024-10-03T20:31:05.472Z+ zoneadm list
6562024-10-03T20:31:05.472Z+ grep nexus
6572024-10-03T20:31:05.472Z+ [[ 31 -gt 300 ]]
6582024-10-03T20:31:05.472Z+ sleep 1
6592024-10-03T20:31:06.472Z+ retry=32
6602024-10-03T20:31:06.501Z+ zoneadm list
6612024-10-03T20:31:06.501Z+ grep nexus
6622024-10-03T20:31:06.501Z+ [[ 32 -gt 300 ]]
6632024-10-03T20:31:06.501Z+ sleep 1
6642024-10-03T20:31:07.481Z+ retry=33
6652024-10-03T20:31:07.505Z+ zoneadm list
6662024-10-03T20:31:07.505Z+ grep nexus
6672024-10-03T20:31:07.505Z+ [[ 33 -gt 300 ]]
6682024-10-03T20:31:07.505Z+ sleep 1
6692024-10-03T20:31:08.493Z+ retry=34
6702024-10-03T20:31:08.517Z+ zoneadm list
6712024-10-03T20:31:08.517Z+ grep nexus
6722024-10-03T20:31:08.517Z+ [[ 34 -gt 300 ]]
6732024-10-03T20:31:08.517Z+ sleep 1
6742024-10-03T20:31:09.506Z+ retry=35
6752024-10-03T20:31:09.530Z+ zoneadm list
6762024-10-03T20:31:09.530Z+ grep nexus
6772024-10-03T20:31:09.530Z+ [[ 35 -gt 300 ]]
6782024-10-03T20:31:09.530Z+ sleep 1
6792024-10-03T20:31:10.517Z+ retry=36
6802024-10-03T20:31:10.540Z+ zoneadm list
6812024-10-03T20:31:10.540Z+ grep nexus
6822024-10-03T20:31:10.540Z+ [[ 36 -gt 300 ]]
6832024-10-03T20:31:10.540Z+ sleep 1
6842024-10-03T20:31:11.529Z+ retry=37
6852024-10-03T20:31:11.552Z+ zoneadm list
6862024-10-03T20:31:11.552Z+ grep nexus
6872024-10-03T20:31:11.552Z+ [[ 37 -gt 300 ]]
6882024-10-03T20:31:11.552Z+ sleep 1
6892024-10-03T20:31:12.540Z+ retry=38
6902024-10-03T20:31:12.564Z+ zoneadm list
6912024-10-03T20:31:12.564Z+ grep nexus
6922024-10-03T20:31:12.564Z+ [[ 38 -gt 300 ]]
6932024-10-03T20:31:12.564Z+ sleep 1
6942024-10-03T20:31:13.552Z+ retry=39
6952024-10-03T20:31:13.575Z+ zoneadm list
6962024-10-03T20:31:13.575Z+ grep nexus
6972024-10-03T20:31:13.575Z+ [[ 39 -gt 300 ]]
6982024-10-03T20:31:13.575Z+ sleep 1
6992024-10-03T20:31:14.563Z+ retry=40
7002024-10-03T20:31:14.586Z+ zoneadm list
7012024-10-03T20:31:14.587Z+ grep nexus
7022024-10-03T20:31:14.587Z+ [[ 40 -gt 300 ]]
7032024-10-03T20:31:14.587Z+ sleep 1
7042024-10-03T20:31:15.575Z+ retry=41
7052024-10-03T20:31:15.598Z+ zoneadm list
7062024-10-03T20:31:15.598Z+ grep nexus
7072024-10-03T20:31:15.598Z+ [[ 41 -gt 300 ]]
7082024-10-03T20:31:15.598Z+ sleep 1
7092024-10-03T20:31:16.587Z+ retry=42
7102024-10-03T20:31:16.610Z+ zoneadm list
7112024-10-03T20:31:16.610Z+ grep nexus
7122024-10-03T20:31:16.610Z+ [[ 42 -gt 300 ]]
7132024-10-03T20:31:16.610Z+ sleep 1
7142024-10-03T20:31:17.597Z+ retry=43
7152024-10-03T20:31:17.621Z+ zoneadm list
7162024-10-03T20:31:17.621Z+ grep nexus
7172024-10-03T20:31:17.621Z+ [[ 43 -gt 300 ]]
7182024-10-03T20:31:17.621Z+ sleep 1
7192024-10-03T20:31:18.613Z+ retry=44
7202024-10-03T20:31:18.637Z+ zoneadm list
7212024-10-03T20:31:18.637Z+ grep nexus
7222024-10-03T20:31:18.637Z+ [[ 44 -gt 300 ]]
7232024-10-03T20:31:18.637Z+ sleep 1
7242024-10-03T20:31:19.630Z+ retry=45
7252024-10-03T20:31:19.653Z+ zoneadm list
7262024-10-03T20:31:19.653Z+ grep nexus
7272024-10-03T20:31:19.677Z+ [[ 45 -gt 300 ]]
7282024-10-03T20:31:19.677Z+ sleep 1
7292024-10-03T20:31:20.716Z+ retry=46
7302024-10-03T20:31:20.738Z+ zoneadm list
7312024-10-03T20:31:20.738Z+ grep nexus
7322024-10-03T20:31:20.738Z+ [[ 46 -gt 300 ]]
7332024-10-03T20:31:20.738Z+ sleep 1
7342024-10-03T20:31:21.732Z+ retry=47
7352024-10-03T20:31:21.755Z+ zoneadm list
7362024-10-03T20:31:21.755Z+ grep nexus
7372024-10-03T20:31:21.755Z+ [[ 47 -gt 300 ]]
7382024-10-03T20:31:21.755Z+ sleep 1
7392024-10-03T20:31:22.749Z+ retry=48
7402024-10-03T20:31:22.772Z+ zoneadm list
7412024-10-03T20:31:22.772Z+ grep nexus
7422024-10-03T20:31:22.795Z+ [[ 48 -gt 300 ]]
7432024-10-03T20:31:22.795Z+ sleep 1
7442024-10-03T20:31:23.819Z+ retry=49
7452024-10-03T20:31:23.819Z+ zoneadm list
7462024-10-03T20:31:23.819Z+ grep nexus
7472024-10-03T20:31:23.819Z+ [[ 49 -gt 300 ]]
7482024-10-03T20:31:23.819Z+ sleep 1
7492024-10-03T20:31:24.842Z+ retry=50
7502024-10-03T20:31:24.867Z+ zoneadm list
7512024-10-03T20:31:24.867Z+ grep nexus
7522024-10-03T20:31:24.867Z+ [[ 50 -gt 300 ]]
7532024-10-03T20:31:24.867Z+ sleep 1
7542024-10-03T20:31:25.862Z+ retry=51
7552024-10-03T20:31:25.886Z+ zoneadm list
7562024-10-03T20:31:25.886Z+ grep nexus
7572024-10-03T20:31:25.886Z+ [[ 51 -gt 300 ]]
7582024-10-03T20:31:25.886Z+ sleep 1
7592024-10-03T20:31:26.884Z+ retry=52
7602024-10-03T20:31:26.910Z+ zoneadm list
7612024-10-03T20:31:26.910Z+ grep nexus
7622024-10-03T20:31:26.910Z+ [[ 52 -gt 300 ]]
7632024-10-03T20:31:26.910Z+ sleep 1
7642024-10-03T20:31:27.904Z+ retry=53
7652024-10-03T20:31:27.929Z+ zoneadm list
7662024-10-03T20:31:27.929Z+ grep nexus
7672024-10-03T20:31:27.929Z+ [[ 53 -gt 300 ]]
7682024-10-03T20:31:27.929Z+ sleep 1
7692024-10-03T20:31:28.925Z+ retry=54
7702024-10-03T20:31:28.949Z+ zoneadm list
7712024-10-03T20:31:28.949Z+ grep nexus
7722024-10-03T20:31:28.950Z+ [[ 54 -gt 300 ]]
7732024-10-03T20:31:28.950Z+ sleep 1
7742024-10-03T20:31:29.941Z+ retry=55
7752024-10-03T20:31:29.967Z+ zoneadm list
7762024-10-03T20:31:29.967Z+ grep nexus
7772024-10-03T20:31:29.967Z+ [[ 55 -gt 300 ]]
7782024-10-03T20:31:29.967Z+ sleep 1
7792024-10-03T20:31:30.960Z+ retry=56
7802024-10-03T20:31:30.984Z+ zoneadm list
7812024-10-03T20:31:30.984Z+ grep nexus
7822024-10-03T20:31:30.984Z+ [[ 56 -gt 300 ]]
7832024-10-03T20:31:30.984Z+ sleep 1
7842024-10-03T20:31:31.975Z+ retry=57
7852024-10-03T20:31:31.999Z+ zoneadm list
7862024-10-03T20:31:31.999Z+ grep nexus
7872024-10-03T20:31:31.999Z+ [[ 57 -gt 300 ]]
7882024-10-03T20:31:31.999Z+ sleep 1
7892024-10-03T20:31:32.991Z+ retry=58
7902024-10-03T20:31:33.016Z+ zoneadm list
7912024-10-03T20:31:33.016Z+ grep nexus
7922024-10-03T20:31:33.016Z+ [[ 58 -gt 300 ]]
7932024-10-03T20:31:33.016Z+ sleep 1
7942024-10-03T20:31:34.009Z+ retry=59
7952024-10-03T20:31:34.042Z+ zoneadm list
7962024-10-03T20:31:34.042Z+ grep nexus
7972024-10-03T20:31:34.042Z+ [[ 59 -gt 300 ]]
7982024-10-03T20:31:34.042Z+ sleep 1
7992024-10-03T20:31:35.023Z+ retry=60
8002024-10-03T20:31:35.048Z+ zoneadm list
8012024-10-03T20:31:35.048Z+ grep nexus
8022024-10-03T20:31:35.048Z+ [[ 60 -gt 300 ]]
8032024-10-03T20:31:35.048Z+ sleep 1
8042024-10-03T20:31:36.044Z+ retry=61
8052024-10-03T20:31:36.068Z+ zoneadm list
8062024-10-03T20:31:36.068Z+ grep nexus
8072024-10-03T20:31:36.068Z+ [[ 61 -gt 300 ]]
8082024-10-03T20:31:36.068Z+ sleep 1
8092024-10-03T20:31:37.068Z+ retry=62
8102024-10-03T20:31:37.092Z+ zoneadm list
8112024-10-03T20:31:37.092Z+ grep nexus
8122024-10-03T20:31:37.092Z+ [[ 62 -gt 300 ]]
8132024-10-03T20:31:37.092Z+ sleep 1
8142024-10-03T20:31:38.091Z+ retry=63
8152024-10-03T20:31:38.116Z+ zoneadm list
8162024-10-03T20:31:38.116Z+ grep nexus
8172024-10-03T20:31:38.116Zoxz_nexus_4955bbb7-4e36-47ac-bedf-e040f180a990
8182024-10-03T20:31:38.211Z+ echo 'Waited for nexus: 63s'
8192024-10-03T20:31:38.211ZWaited for nexus: 63s
8202024-10-03T20:31:38.211Z+ export RUST_BACKTRACE=1
8212024-10-03T20:31:38.235Z+ RUST_BACKTRACE=1
8222024-10-03T20:31:38.235Z+ export E2E_TLS_CERT IPPOOL_START IPPOOL_END
8232024-10-03T20:31:38.260Z++ ./target/debug/bootstrap
8242024-10-03T20:31:40.570ZUsing external DNS server at 10.151.1.164:53
8252024-10-03T20:31:40.657Z2024-10-03 20:31:40.622430518 UTC: attempting to log into API
8262024-10-03T20:31:55.705Z2024-10-03 20:31:55.689987419 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local)
8272024-10-03T20:31:56.704Z2024-10-03 20:31:56.691244031 UTC: attempting to log into API
 
8282024-10-03T20:31:58.519ZOct 3 20:31:57 buskin ip: ip: joining multicasts failed (4) on opte4 - will use link layer broadcasts for multicast
8292024-10-03T20:31:59.098ZOct 3 20:31:58 buskin ip: ip: joining multicasts failed (4) on opte1 - will use link layer broadcasts for multicast
8302024-10-03T20:31:59.403ZOct 3 20:31:58 buskin xde: WARNING: failed to generate descriptor for stateful action: ICMP:172.30.3.5:0:168.61.215.74:0 Unexpected { msg: "Expected ICMP Echo Request, found: destination unreachable" }
8312024-10-03T20:31:59.432ZOct 3 20:31:58 buskin ip: ip: joining multicasts failed (4) on opte5 - will use link layer broadcasts for multicast
8322024-10-03T20:32:00.015ZOct 3 20:31:59 buskin ip: ip: joining multicasts failed (4) on opte2 - will use link layer broadcasts for multicast
8332024-10-03T20:32:00.045ZOct 3 20:31:59 buskin ip: ip: joining multicasts failed (4) on opte3 - will use link layer broadcasts for multicast
 
8342024-10-03T20:32:11.738Z2024-10-03 20:32:11.724518829 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local)
8352024-10-03T20:32:12.738Z2024-10-03 20:32:12.724757866 UTC: attempting to log into API
8362024-10-03T20:32:27.771Z2024-10-03 20:32:27.757871078 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local)
8372024-10-03T20:32:28.773Z2024-10-03 20:32:28.759087650 UTC: attempting to log into API
8382024-10-03T20:32:38.825Z2024-10-03 20:32:38.811908706 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local)
8392024-10-03T20:32:39.827Z2024-10-03 20:32:39.813189119 UTC: attempting to log into API
8402024-10-03T20:32:40.646Z2024-10-03 20:32:40.632378579 UTC: login succeeded
8412024-10-03T20:32:40.678Zwaiting for nexus to come up...
8422024-10-03T20:32:40.744Zcreating IP pool... 10.151.1.175 - 10.151.1.227
8432024-10-03T20:32:40.974Zsetting up quotas...
8442024-10-03T20:32:41.026Zensuring datasets are ready...
8452024-10-03T20:32:43.580Zlet's roll.
8462024-10-03T20:32:43.604Z+ eval 'OXIDE_HOST=https://recovery.sys.oxide.test
8472024-10-03T20:32:43.604ZOXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
8482024-10-03T20:32:43.604ZOXIDE_TOKEN=oxide-token-2b92d8def2f3e26fd8e2f73df3fed381d436499b'
8492024-10-03T20:32:43.604Z++ OXIDE_HOST=https://recovery.sys.oxide.test
8502024-10-03T20:32:43.604Z++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.1.166
8512024-10-03T20:32:43.604Z++ OXIDE_TOKEN=oxide-token-2b92d8def2f3e26fd8e2f73df3fed381d436499b
8522024-10-03T20:32:43.604Z+ export OXIDE_HOST OXIDE_TOKEN
8532024-10-03T20:32:43.604Z+ retry=0
8542024-10-03T20:32:43.604Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8552024-10-03T20:32:43.604Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8562024-10-03T20:32:43.604Z+ [[ 0 -gt 60 ]]
8572024-10-03T20:32:43.604Z+ sleep 1
8582024-10-03T20:32:44.607Z+ retry=1
8592024-10-03T20:32:44.607Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8602024-10-03T20:32:44.631Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8612024-10-03T20:32:44.631Z+ [[ 1 -gt 60 ]]
8622024-10-03T20:32:44.631Z+ sleep 1
8632024-10-03T20:32:45.623Z+ retry=2
8642024-10-03T20:32:45.623Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8652024-10-03T20:32:45.647Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8662024-10-03T20:32:45.647Z+ [[ 2 -gt 60 ]]
8672024-10-03T20:32:45.647Z+ sleep 1
8682024-10-03T20:32:46.638Z+ retry=3
8692024-10-03T20:32:46.639Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8702024-10-03T20:32:46.663Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8712024-10-03T20:32:46.663Z+ [[ 3 -gt 60 ]]
8722024-10-03T20:32:46.663Z+ sleep 1
8732024-10-03T20:32:47.655Z+ retry=4
8742024-10-03T20:32:47.655Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8752024-10-03T20:32:47.679Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8762024-10-03T20:32:47.679Z+ [[ 4 -gt 60 ]]
8772024-10-03T20:32:47.679Z+ sleep 1
8782024-10-03T20:32:48.669Z+ retry=5
8792024-10-03T20:32:48.669Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8802024-10-03T20:32:48.694Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8812024-10-03T20:32:48.694Z+ [[ 5 -gt 60 ]]
8822024-10-03T20:32:48.694Z+ sleep 1
8832024-10-03T20:32:49.686Z+ retry=6
8842024-10-03T20:32:49.686Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8852024-10-03T20:32:49.711Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8862024-10-03T20:32:49.711Z+ [[ 6 -gt 60 ]]
8872024-10-03T20:32:49.711Z+ sleep 1
8882024-10-03T20:32:50.701Z+ retry=7
8892024-10-03T20:32:50.725Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8902024-10-03T20:32:50.725Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8912024-10-03T20:32:50.725Z+ [[ 7 -gt 60 ]]
8922024-10-03T20:32:50.726Z+ sleep 1
8932024-10-03T20:32:51.716Z+ retry=8
8942024-10-03T20:32:51.716Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8952024-10-03T20:32:51.741Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8962024-10-03T20:32:51.741Z+ [[ 8 -gt 60 ]]
8972024-10-03T20:32:51.741Z+ sleep 1
8982024-10-03T20:32:52.731Z+ retry=9
8992024-10-03T20:32:52.731Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9002024-10-03T20:32:52.755Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9012024-10-03T20:32:52.755Z+ [[ 9 -gt 60 ]]
9022024-10-03T20:32:52.755Z+ sleep 1
9032024-10-03T20:32:53.747Z+ retry=10
9042024-10-03T20:32:53.748Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9052024-10-03T20:32:53.772Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9062024-10-03T20:32:53.772Z+ [[ 10 -gt 60 ]]
9072024-10-03T20:32:53.772Z+ sleep 1
9082024-10-03T20:32:54.761Z+ retry=11
9092024-10-03T20:32:54.761Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9102024-10-03T20:32:54.785Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9112024-10-03T20:32:54.785Z+ [[ 11 -gt 60 ]]
9122024-10-03T20:32:54.785Z+ sleep 1
9132024-10-03T20:32:55.777Z+ retry=12
9142024-10-03T20:32:55.778Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9152024-10-03T20:32:55.802Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9162024-10-03T20:32:55.802Z+ [[ 12 -gt 60 ]]
9172024-10-03T20:32:55.802Z+ sleep 1
9182024-10-03T20:32:56.794Z+ retry=13
9192024-10-03T20:32:56.794Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9202024-10-03T20:32:56.818Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9212024-10-03T20:32:56.818Z+ [[ 13 -gt 60 ]]
9222024-10-03T20:32:56.818Z+ sleep 1
9232024-10-03T20:32:57.835Z+ retry=14
9242024-10-03T20:32:57.835Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9252024-10-03T20:32:57.835Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9262024-10-03T20:32:57.835Z+ [[ 14 -gt 60 ]]
9272024-10-03T20:32:57.835Z+ sleep 1
9282024-10-03T20:32:58.825Z+ retry=15
9292024-10-03T20:32:58.825Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9302024-10-03T20:32:58.849Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9312024-10-03T20:32:58.849Z+ [[ 15 -gt 60 ]]
9322024-10-03T20:32:58.849Z+ sleep 1
9332024-10-03T20:32:59.841Z+ retry=16
9342024-10-03T20:32:59.841Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9352024-10-03T20:32:59.864Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9362024-10-03T20:32:59.864Z+ [[ 16 -gt 60 ]]
9372024-10-03T20:32:59.864Z+ sleep 1
9382024-10-03T20:33:00.857Z+ retry=17
9392024-10-03T20:33:00.857Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9402024-10-03T20:33:00.881Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9412024-10-03T20:33:00.881Z+ [[ 17 -gt 60 ]]
9422024-10-03T20:33:00.881Z+ sleep 1
9432024-10-03T20:33:01.873Z+ retry=18
9442024-10-03T20:33:01.873Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9452024-10-03T20:33:01.897Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9462024-10-03T20:33:01.897Z+ [[ 18 -gt 60 ]]
9472024-10-03T20:33:01.897Z+ sleep 1
9482024-10-03T20:33:02.888Z+ retry=19
9492024-10-03T20:33:02.888Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9502024-10-03T20:33:02.912Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9512024-10-03T20:33:02.912Z+ [[ 19 -gt 60 ]]
9522024-10-03T20:33:02.912Z+ sleep 1
9532024-10-03T20:33:03.908Z+ retry=20
9542024-10-03T20:33:03.908Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9552024-10-03T20:33:03.932Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9562024-10-03T20:33:03.932Z+ [[ 20 -gt 60 ]]
9572024-10-03T20:33:03.932Z+ sleep 1
9582024-10-03T20:33:04.918Z+ retry=21
9592024-10-03T20:33:04.918Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9602024-10-03T20:33:04.942Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9612024-10-03T20:33:04.942Z+ [[ 21 -gt 60 ]]
9622024-10-03T20:33:04.943Z+ sleep 1
9632024-10-03T20:33:05.933Z+ retry=22
9642024-10-03T20:33:05.933Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9652024-10-03T20:33:05.957Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9662024-10-03T20:33:05.957Z+ [[ 22 -gt 60 ]]
9672024-10-03T20:33:05.957Z+ sleep 1
9682024-10-03T20:33:06.950Z+ retry=23
9692024-10-03T20:33:06.950Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9702024-10-03T20:33:07.058Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images'
9712024-10-03T20:33:07.855Z{"status":"ok"}success
9722024-10-03T20:33:07.855ZProject {
9732024-10-03T20:33:07.855Z description: "some images",
9742024-10-03T20:33:07.855Z id: 975e3d5e-8911-47af-9f7a-94030499ce57,
9752024-10-03T20:33:07.856Z name: Name(
9762024-10-03T20:33:07.856Z "images",
9772024-10-03T20:33:07.856Z ),
9782024-10-03T20:33:07.856Z time_created: 2024-10-03T20:33:07.243779Z,
9792024-10-03T20:33:07.856Z time_modified: 2024-10-03T20:33:07.243779Z,
9802024-10-03T20:33:07.856Z}
9812024-10-03T20:33:07.880Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11
9822024-10-03T20:36:14.693Zdone!
9832024-10-03T20:36:14.787Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.1.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11
9842024-10-03T20:36:14.910Zsuccess
9852024-10-03T20:36:14.910ZImage {
9862024-10-03T20:36:14.910Z block_size: ByteCount(
9872024-10-03T20:36:14.910Z 512,
9882024-10-03T20:36:14.910Z ),
9892024-10-03T20:36:14.910Z description: "debian 11 original base image",
9902024-10-03T20:36:14.910Z digest: None,
9912024-10-03T20:36:14.910Z id: 7f8bade8-ab27-4280-9b35-c21b3ed38e47,
9922024-10-03T20:36:14.910Z name: Name(
9932024-10-03T20:36:14.910Z "debian11",
9942024-10-03T20:36:14.910Z ),
9952024-10-03T20:36:14.935Z os: "debian",
9962024-10-03T20:36:14.935Z project_id: None,
9972024-10-03T20:36:14.935Z size: ByteCount(
9982024-10-03T20:36:14.935Z 2147483648,
9992024-10-03T20:36:14.935Z ),
10002024-10-03T20:36:14.935Z time_created: 2024-10-03T20:36:14.618733Z,
10012024-10-03T20:36:14.935Z time_modified: 2024-10-03T20:36:14.884324Z,
10022024-10-03T20:36:14.935Z url: None,
10032024-10-03T20:36:14.935Z version: "11",
10042024-10-03T20:36:14.935Z}
10052024-10-03T20:36:14.935Z+ for test_bin in tests/*
10062024-10-03T20:36:14.935Z+ ./tests/bootstrap-ed9ec115c9a0b0d5
10072024-10-03T20:36:14.959Z
10082024-10-03T20:36:14.959Zrunning 0 tests
10092024-10-03T20:36:14.959Z
10102024-10-03T20:36:14.959Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
10112024-10-03T20:36:14.959Z
10122024-10-03T20:36:14.959Z+ for test_bin in tests/*
10132024-10-03T20:36:14.959Z+ ./tests/commtest-3e99aed6362922f5
10142024-10-03T20:36:14.959Z
10152024-10-03T20:36:14.959Zrunning 0 tests
10162024-10-03T20:36:14.959Z
10172024-10-03T20:36:14.959Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
10182024-10-03T20:36:14.959Z
10192024-10-03T20:36:14.959Z+ for test_bin in tests/*
10202024-10-03T20:36:14.959Z+ ./tests/dhcp_server-a33e1c62b9efac3e
10212024-10-03T20:36:14.983Z
10222024-10-03T20:36:14.983Zrunning 0 tests
10232024-10-03T20:36:14.983Z
10242024-10-03T20:36:14.983Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
10252024-10-03T20:36:14.984Z
10262024-10-03T20:36:14.984Z+ for test_bin in tests/*
10272024-10-03T20:36:14.984Z+ ./tests/end_to_end_tests-46e57f5259837a5f
10282024-10-03T20:36:15.071Z
10292024-10-03T20:36:15.071Zrunning 2 tests
10302024-10-03T20:36:15.315Ztest no_spoof::no_spoof_here ... ok
10312024-10-03T20:36:42.767Ztest instance_launch::instance_launch ... ok
10322024-10-03T20:36:42.767Z
10332024-10-03T20:36:42.791Ztest result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 27.70s
10342024-10-03T20:36:42.791Z
10352024-10-03T20:36:42.791Z+ _exit_trap
10362024-10-03T20:36:42.791Z+ local status=0
10372024-10-03T20:36:42.791Z+ set +o errexit
10382024-10-03T20:36:42.791Z+ pfexec svcadm -Z disable -s cron
10392024-10-03T20:36:44.461Z+ [[ 0 -eq 0 ]]
10402024-10-03T20:36:44.461Z+ exit 0
10412024-10-03T20:36:44.484Zprocess exited: duration 459850 ms, exit code 0
 
10422024-10-03T20:36:44.562Zfound 108 output files
10432024-10-03T20:36:44.562Zuploading: /var/svc/log/oxide-sled-agent:default.log (865013 bytes)
10442024-10-03T20:36:45.892Zuploaded: /var/svc/log/oxide-sled-agent:default.log
10452024-10-03T20:36:45.892Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log (3485555 bytes)
10462024-10-03T20:36:46.277Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log" changed size mid upload: 3485555 -> 3501250
10472024-10-03T20:36:47.347Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log
10482024-10-03T20:36:47.347Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log (35102 bytes)
10492024-10-03T20:36:48.506Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log
10502024-10-03T20:36:48.506Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log (127332 bytes)
10512024-10-03T20:36:48.713Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log" changed size mid upload: 127332 -> 128896
10522024-10-03T20:36:49.790Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log
10532024-10-03T20:36:49.790Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log (166221 bytes)
10542024-10-03T20:36:49.997Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log" changed size mid upload: 166221 -> 168567
10552024-10-03T20:36:51.066Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log
10562024-10-03T20:36:51.091Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log (1088988 bytes)
10572024-10-03T20:36:51.344Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log" changed size mid upload: 1088988 -> 1105986
10582024-10-03T20:36:52.418Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log
10592024-10-03T20:36:52.442Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log (2574338 bytes)
10602024-10-03T20:36:52.789Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log" changed size mid upload: 2574338 -> 2634574
10612024-10-03T20:36:53.860Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log
10622024-10-03T20:36:53.860Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log (3339 bytes)
10632024-10-03T20:36:54.953Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log
10642024-10-03T20:36:54.981Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log (195864 bytes)
10652024-10-03T20:36:55.151Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log" changed size mid upload: 195864 -> 201042
10662024-10-03T20:36:56.219Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log
10672024-10-03T20:36:56.219Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log (5515 bytes)
10682024-10-03T20:36:57.310Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log
10692024-10-03T20:36:57.310Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log (3814 bytes)
10702024-10-03T20:36:58.404Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log
10712024-10-03T20:36:58.404Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-zone-network-setup:default.log (8451 bytes)
10722024-10-03T20:36:59.518Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-zone-network-setup:default.log
10732024-10-03T20:36:59.518Zuploading: /pool/ext/1220f876-5a4e-47b8-b4aa-f7dee07ba8ea/crypt/zone/oxz_crucible_37e7ecdf-b9c1-40af-a2c7-f8870b3ffc4e/root/var/svc/log/oxide-crucible-agent:default.log (8485 bytes)
10742024-10-03T20:37:00.633Zuploaded: /pool/ext/1220f876-5a4e-47b8-b4aa-f7dee07ba8ea/crypt/zone/oxz_crucible_37e7ecdf-b9c1-40af-a2c7-f8870b3ffc4e/root/var/svc/log/oxide-crucible-agent:default.log
10752024-10-03T20:37:00.633Zuploading: /pool/ext/1220f876-5a4e-47b8-b4aa-f7dee07ba8ea/crypt/zone/oxz_crucible_37e7ecdf-b9c1-40af-a2c7-f8870b3ffc4e/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10762024-10-03T20:37:01.727Zuploaded: /pool/ext/1220f876-5a4e-47b8-b4aa-f7dee07ba8ea/crypt/zone/oxz_crucible_37e7ecdf-b9c1-40af-a2c7-f8870b3ffc4e/root/var/svc/log/oxide-crucible-downstairs:default.log
10772024-10-03T20:37:01.727Zuploading: /pool/ext/1220f876-5a4e-47b8-b4aa-f7dee07ba8ea/crypt/zone/oxz_crucible_37e7ecdf-b9c1-40af-a2c7-f8870b3ffc4e/root/var/svc/log/oxide-crucible-downstairs:downstairs-4ef04814-c6b5-45dc-84f6-9f15aaeffdbb.log (3020 bytes)
10782024-10-03T20:37:02.859Zuploaded: /pool/ext/1220f876-5a4e-47b8-b4aa-f7dee07ba8ea/crypt/zone/oxz_crucible_37e7ecdf-b9c1-40af-a2c7-f8870b3ffc4e/root/var/svc/log/oxide-crucible-downstairs:downstairs-4ef04814-c6b5-45dc-84f6-9f15aaeffdbb.log
10792024-10-03T20:37:02.859Zuploading: /pool/ext/1220f876-5a4e-47b8-b4aa-f7dee07ba8ea/crypt/zone/oxz_crucible_37e7ecdf-b9c1-40af-a2c7-f8870b3ffc4e/root/var/svc/log/oxide-zone-network-setup:default.log (2082 bytes)
10802024-10-03T20:37:03.915Zuploaded: /pool/ext/1220f876-5a4e-47b8-b4aa-f7dee07ba8ea/crypt/zone/oxz_crucible_37e7ecdf-b9c1-40af-a2c7-f8870b3ffc4e/root/var/svc/log/oxide-zone-network-setup:default.log
10812024-10-03T20:37:03.915Zuploading: /pool/ext/1220f876-5a4e-47b8-b4aa-f7dee07ba8ea/crypt/zone/oxz_crucible_pantry_13b171e8-3788-4f60-b40e-56a319a99abe/root/var/svc/log/oxide-crucible-pantry:default.log (783 bytes)
10822024-10-03T20:37:05.007Zuploaded: /pool/ext/1220f876-5a4e-47b8-b4aa-f7dee07ba8ea/crypt/zone/oxz_crucible_pantry_13b171e8-3788-4f60-b40e-56a319a99abe/root/var/svc/log/oxide-crucible-pantry:default.log
10832024-10-03T20:37:05.031Zuploading: /pool/ext/1220f876-5a4e-47b8-b4aa-f7dee07ba8ea/crypt/zone/oxz_crucible_pantry_13b171e8-3788-4f60-b40e-56a319a99abe/root/var/svc/log/oxide-zone-network-setup:default.log (2139 bytes)
10842024-10-03T20:37:06.102Zuploaded: /pool/ext/1220f876-5a4e-47b8-b4aa-f7dee07ba8ea/crypt/zone/oxz_crucible_pantry_13b171e8-3788-4f60-b40e-56a319a99abe/root/var/svc/log/oxide-zone-network-setup:default.log
10852024-10-03T20:37:06.102Zuploading: /pool/ext/12b34c54-e8f4-4b50-a4d7-839f5c9ddbe9/crypt/zone/oxz_crucible_05b7fd1f-5795-4866-bfcc-8dc7293400ff/root/var/svc/log/oxide-crucible-agent:default.log (18569 bytes)
10862024-10-03T20:37:07.236Zuploaded: /pool/ext/12b34c54-e8f4-4b50-a4d7-839f5c9ddbe9/crypt/zone/oxz_crucible_05b7fd1f-5795-4866-bfcc-8dc7293400ff/root/var/svc/log/oxide-crucible-agent:default.log
10872024-10-03T20:37:07.236Zuploading: /pool/ext/12b34c54-e8f4-4b50-a4d7-839f5c9ddbe9/crypt/zone/oxz_crucible_05b7fd1f-5795-4866-bfcc-8dc7293400ff/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10882024-10-03T20:37:08.327Zuploaded: /pool/ext/12b34c54-e8f4-4b50-a4d7-839f5c9ddbe9/crypt/zone/oxz_crucible_05b7fd1f-5795-4866-bfcc-8dc7293400ff/root/var/svc/log/oxide-crucible-downstairs:default.log
10892024-10-03T20:37:08.327Zuploading: /pool/ext/12b34c54-e8f4-4b50-a4d7-839f5c9ddbe9/crypt/zone/oxz_crucible_05b7fd1f-5795-4866-bfcc-8dc7293400ff/root/var/svc/log/oxide-crucible-downstairs:downstairs-84b11064-a042-425c-a1a3-f7fba2d9a45c.log (3018 bytes)
10902024-10-03T20:37:09.546Zuploaded: /pool/ext/12b34c54-e8f4-4b50-a4d7-839f5c9ddbe9/crypt/zone/oxz_crucible_05b7fd1f-5795-4866-bfcc-8dc7293400ff/root/var/svc/log/oxide-crucible-downstairs:downstairs-84b11064-a042-425c-a1a3-f7fba2d9a45c.log
10912024-10-03T20:37:09.576Zuploading: /pool/ext/12b34c54-e8f4-4b50-a4d7-839f5c9ddbe9/crypt/zone/oxz_crucible_05b7fd1f-5795-4866-bfcc-8dc7293400ff/root/var/svc/log/oxide-crucible-downstairs:downstairs-ceca784c-cd6e-4f55-b669-eb1bc308b4c3.log (3125 bytes)
10922024-10-03T20:37:10.637Zuploaded: /pool/ext/12b34c54-e8f4-4b50-a4d7-839f5c9ddbe9/crypt/zone/oxz_crucible_05b7fd1f-5795-4866-bfcc-8dc7293400ff/root/var/svc/log/oxide-crucible-downstairs:downstairs-ceca784c-cd6e-4f55-b669-eb1bc308b4c3.log
10932024-10-03T20:37:10.637Zuploading: /pool/ext/12b34c54-e8f4-4b50-a4d7-839f5c9ddbe9/crypt/zone/oxz_crucible_05b7fd1f-5795-4866-bfcc-8dc7293400ff/root/var/svc/log/oxide-zone-network-setup:default.log (2082 bytes)
10942024-10-03T20:37:11.749Zuploaded: /pool/ext/12b34c54-e8f4-4b50-a4d7-839f5c9ddbe9/crypt/zone/oxz_crucible_05b7fd1f-5795-4866-bfcc-8dc7293400ff/root/var/svc/log/oxide-zone-network-setup:default.log
10952024-10-03T20:37:11.787Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_cockroachdb_86b36d5c-394e-499a-8e63-0d1e9b7e54a6/root/var/svc/log/oxide-cockroach-admin:default.log (4697 bytes)
10962024-10-03T20:37:12.861Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_cockroachdb_86b36d5c-394e-499a-8e63-0d1e9b7e54a6/root/var/svc/log/oxide-cockroach-admin:default.log
10972024-10-03T20:37:12.861Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_cockroachdb_86b36d5c-394e-499a-8e63-0d1e9b7e54a6/root/var/svc/log/oxide-cockroachdb:default.log (4314 bytes)
10982024-10-03T20:37:13.953Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_cockroachdb_86b36d5c-394e-499a-8e63-0d1e9b7e54a6/root/var/svc/log/oxide-cockroachdb:default.log
10992024-10-03T20:37:13.953Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_cockroachdb_86b36d5c-394e-499a-8e63-0d1e9b7e54a6/root/var/svc/log/oxide-zone-network-setup:default.log (2093 bytes)
11002024-10-03T20:37:15.047Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_cockroachdb_86b36d5c-394e-499a-8e63-0d1e9b7e54a6/root/var/svc/log/oxide-zone-network-setup:default.log
11012024-10-03T20:37:15.047Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_02b46aad-d593-4b9f-86d0-f084daa4a4f0/root/var/svc/log/oxide-crucible-agent:default.log (36036 bytes)
11022024-10-03T20:37:16.204Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_02b46aad-d593-4b9f-86d0-f084daa4a4f0/root/var/svc/log/oxide-crucible-agent:default.log
11032024-10-03T20:37:16.204Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_02b46aad-d593-4b9f-86d0-f084daa4a4f0/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11042024-10-03T20:37:17.296Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_02b46aad-d593-4b9f-86d0-f084daa4a4f0/root/var/svc/log/oxide-crucible-downstairs:default.log
11052024-10-03T20:37:17.321Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_02b46aad-d593-4b9f-86d0-f084daa4a4f0/root/var/svc/log/oxide-crucible-downstairs:downstairs-025e3246-a1bf-4cfd-91d6-a68f4ef7cb2c.log (5995 bytes)
11062024-10-03T20:37:18.389Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_02b46aad-d593-4b9f-86d0-f084daa4a4f0/root/var/svc/log/oxide-crucible-downstairs:downstairs-025e3246-a1bf-4cfd-91d6-a68f4ef7cb2c.log
11072024-10-03T20:37:18.389Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_02b46aad-d593-4b9f-86d0-f084daa4a4f0/root/var/svc/log/oxide-crucible-downstairs:downstairs-f2591d0a-310c-4551-b6dd-7ca41cdb0016.log (6077 bytes)
11082024-10-03T20:37:19.480Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_02b46aad-d593-4b9f-86d0-f084daa4a4f0/root/var/svc/log/oxide-crucible-downstairs:downstairs-f2591d0a-310c-4551-b6dd-7ca41cdb0016.log
11092024-10-03T20:37:19.480Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_02b46aad-d593-4b9f-86d0-f084daa4a4f0/root/var/svc/log/oxide-crucible-downstairs:snapshot-f2591d0a-310c-4551-b6dd-7ca41cdb0016-bec3d930-a591-451d-82e5-be08acca0f21.log (5684 bytes)
11102024-10-03T20:37:20.571Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_02b46aad-d593-4b9f-86d0-f084daa4a4f0/root/var/svc/log/oxide-crucible-downstairs:snapshot-f2591d0a-310c-4551-b6dd-7ca41cdb0016-bec3d930-a591-451d-82e5-be08acca0f21.log
11112024-10-03T20:37:20.571Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_02b46aad-d593-4b9f-86d0-f084daa4a4f0/root/var/svc/log/oxide-zone-network-setup:default.log (2083 bytes)
11122024-10-03T20:37:21.663Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_02b46aad-d593-4b9f-86d0-f084daa4a4f0/root/var/svc/log/oxide-zone-network-setup:default.log
11132024-10-03T20:37:21.663Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_pantry_05729706-2bf6-4811-8495-09746f087f40/root/var/svc/log/oxide-crucible-pantry:default.log (764193 bytes)
11142024-10-03T20:37:22.980Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_pantry_05729706-2bf6-4811-8495-09746f087f40/root/var/svc/log/oxide-crucible-pantry:default.log
11152024-10-03T20:37:22.980Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_pantry_05729706-2bf6-4811-8495-09746f087f40/root/var/svc/log/oxide-zone-network-setup:default.log (2137 bytes)
11162024-10-03T20:37:24.079Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_pantry_05729706-2bf6-4811-8495-09746f087f40/root/var/svc/log/oxide-zone-network-setup:default.log
11172024-10-03T20:37:24.079Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_pantry_7f69f9c4-db80-49b1-832b-30a7e126f86a/root/var/svc/log/oxide-crucible-pantry:default.log (783 bytes)
11182024-10-03T20:37:25.175Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_pantry_7f69f9c4-db80-49b1-832b-30a7e126f86a/root/var/svc/log/oxide-crucible-pantry:default.log
11192024-10-03T20:37:25.175Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_pantry_7f69f9c4-db80-49b1-832b-30a7e126f86a/root/var/svc/log/oxide-zone-network-setup:default.log (2137 bytes)
11202024-10-03T20:37:26.267Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_crucible_pantry_7f69f9c4-db80-49b1-832b-30a7e126f86a/root/var/svc/log/oxide-zone-network-setup:default.log
11212024-10-03T20:37:26.267Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_external_dns_f893e829-73ec-4e54-9fff-bede8510178f/root/var/svc/log/oxide-external_dns:default.log (19349 bytes)
11222024-10-03T20:37:26.359Zupload warning: file "/pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_external_dns_f893e829-73ec-4e54-9fff-bede8510178f/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 19349 -> 23122
11232024-10-03T20:37:27.428Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_external_dns_f893e829-73ec-4e54-9fff-bede8510178f/root/var/svc/log/oxide-external_dns:default.log
11242024-10-03T20:37:27.428Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_external_dns_f893e829-73ec-4e54-9fff-bede8510178f/root/var/svc/log/oxide-opte-interface-setup:default.log (1156 bytes)
11252024-10-03T20:37:28.521Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_external_dns_f893e829-73ec-4e54-9fff-bede8510178f/root/var/svc/log/oxide-opte-interface-setup:default.log
11262024-10-03T20:37:28.521Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_external_dns_f893e829-73ec-4e54-9fff-bede8510178f/root/var/svc/log/oxide-zone-network-setup:default.log (2112 bytes)
11272024-10-03T20:37:29.624Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_external_dns_f893e829-73ec-4e54-9fff-bede8510178f/root/var/svc/log/oxide-zone-network-setup:default.log
11282024-10-03T20:37:29.624Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_internal_dns_8d353f1a-40a8-4a8e-b2ea-a12cc5a12546/root/var/svc/log/oxide-internal_dns:default.log (399335 bytes)
11292024-10-03T20:37:29.849Zupload warning: file "/pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_internal_dns_8d353f1a-40a8-4a8e-b2ea-a12cc5a12546/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 399335 -> 403087
11302024-10-03T20:37:30.919Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_internal_dns_8d353f1a-40a8-4a8e-b2ea-a12cc5a12546/root/var/svc/log/oxide-internal_dns:default.log
11312024-10-03T20:37:30.919Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_internal_dns_8d353f1a-40a8-4a8e-b2ea-a12cc5a12546/root/var/svc/log/oxide-zone-network-setup:default.log (2094 bytes)
11322024-10-03T20:37:32.011Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_internal_dns_8d353f1a-40a8-4a8e-b2ea-a12cc5a12546/root/var/svc/log/oxide-zone-network-setup:default.log
11332024-10-03T20:37:32.011Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_nexus_b403454b-e367-4049-9c03-93623637be08/root/var/svc/log/oxide-nexus:default.log (2032412 bytes)
11342024-10-03T20:37:32.343Zupload warning: file "/pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_nexus_b403454b-e367-4049-9c03-93623637be08/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 2032412 -> 2302185
11352024-10-03T20:37:33.410Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_nexus_b403454b-e367-4049-9c03-93623637be08/root/var/svc/log/oxide-nexus:default.log
11362024-10-03T20:37:33.410Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_nexus_b403454b-e367-4049-9c03-93623637be08/root/var/svc/log/oxide-opte-interface-setup:default.log (1135 bytes)
11372024-10-03T20:37:34.503Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_nexus_b403454b-e367-4049-9c03-93623637be08/root/var/svc/log/oxide-opte-interface-setup:default.log
11382024-10-03T20:37:34.503Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_nexus_b403454b-e367-4049-9c03-93623637be08/root/var/svc/log/oxide-zone-network-setup:default.log (2055 bytes)
11392024-10-03T20:37:35.597Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_nexus_b403454b-e367-4049-9c03-93623637be08/root/var/svc/log/oxide-zone-network-setup:default.log
11402024-10-03T20:37:35.597Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/svc/log/oxide-chrony-setup:default.log (14496 bytes)
11412024-10-03T20:37:37.016Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/svc/log/oxide-chrony-setup:default.log
11422024-10-03T20:37:37.016Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/svc/log/oxide-ntp:default.log (459 bytes)
11432024-10-03T20:37:38.315Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/svc/log/oxide-ntp:default.log
11442024-10-03T20:37:38.315Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/svc/log/oxide-opte-interface-setup:default.log (1126 bytes)
11452024-10-03T20:37:39.407Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/svc/log/oxide-opte-interface-setup:default.log
11462024-10-03T20:37:39.433Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/svc/log/oxide-zone-network-setup:default.log (2031 bytes)
11472024-10-03T20:37:40.503Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/svc/log/oxide-zone-network-setup:default.log
11482024-10-03T20:37:40.527Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_clickhouse_bdbc39f6-cd8c-49c5-8e14-2507e1c1c964/root/var/svc/log/oxide-clickhouse-admin:default.log (875 bytes)
11492024-10-03T20:37:41.901Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_clickhouse_bdbc39f6-cd8c-49c5-8e14-2507e1c1c964/root/var/svc/log/oxide-clickhouse-admin:default.log
11502024-10-03T20:37:41.901Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_clickhouse_bdbc39f6-cd8c-49c5-8e14-2507e1c1c964/root/var/svc/log/oxide-clickhouse:default.log (2311012 bytes)
11512024-10-03T20:37:42.283Zupload warning: file "/pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_clickhouse_bdbc39f6-cd8c-49c5-8e14-2507e1c1c964/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 2311012 -> 2751836
11522024-10-03T20:37:43.351Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_clickhouse_bdbc39f6-cd8c-49c5-8e14-2507e1c1c964/root/var/svc/log/oxide-clickhouse:default.log
11532024-10-03T20:37:43.351Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_clickhouse_bdbc39f6-cd8c-49c5-8e14-2507e1c1c964/root/var/svc/log/oxide-zone-network-setup:default.log (2094 bytes)
11542024-10-03T20:37:44.444Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_clickhouse_bdbc39f6-cd8c-49c5-8e14-2507e1c1c964/root/var/svc/log/oxide-zone-network-setup:default.log
11552024-10-03T20:37:44.444Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_cockroachdb_02c08223-9982-4ae3-a080-c6e28803b6ed/root/var/svc/log/oxide-cockroach-admin:default.log (4692 bytes)
11562024-10-03T20:37:45.535Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_cockroachdb_02c08223-9982-4ae3-a080-c6e28803b6ed/root/var/svc/log/oxide-cockroach-admin:default.log
11572024-10-03T20:37:45.535Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_cockroachdb_02c08223-9982-4ae3-a080-c6e28803b6ed/root/var/svc/log/oxide-cockroachdb:default.log (4289 bytes)
11582024-10-03T20:37:46.627Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_cockroachdb_02c08223-9982-4ae3-a080-c6e28803b6ed/root/var/svc/log/oxide-cockroachdb:default.log
11592024-10-03T20:37:46.651Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_cockroachdb_02c08223-9982-4ae3-a080-c6e28803b6ed/root/var/svc/log/oxide-zone-network-setup:default.log (2094 bytes)
11602024-10-03T20:37:47.719Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_cockroachdb_02c08223-9982-4ae3-a080-c6e28803b6ed/root/var/svc/log/oxide-zone-network-setup:default.log
11612024-10-03T20:37:47.719Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_crucible_566afbcc-e288-42ff-812e-f452b079397a/root/var/svc/log/oxide-crucible-agent:default.log (1556 bytes)
11622024-10-03T20:37:48.810Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_crucible_566afbcc-e288-42ff-812e-f452b079397a/root/var/svc/log/oxide-crucible-agent:default.log
11632024-10-03T20:37:48.810Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_crucible_566afbcc-e288-42ff-812e-f452b079397a/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11642024-10-03T20:37:49.905Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_crucible_566afbcc-e288-42ff-812e-f452b079397a/root/var/svc/log/oxide-crucible-downstairs:default.log
11652024-10-03T20:37:49.905Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_crucible_566afbcc-e288-42ff-812e-f452b079397a/root/var/svc/log/oxide-zone-network-setup:default.log (2083 bytes)
11662024-10-03T20:37:50.997Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_crucible_566afbcc-e288-42ff-812e-f452b079397a/root/var/svc/log/oxide-zone-network-setup:default.log
11672024-10-03T20:37:50.998Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_external_dns_e16c7455-6a1b-459a-bda2-63357961e027/root/var/svc/log/oxide-external_dns:default.log (23451 bytes)
11682024-10-03T20:37:51.087Zupload warning: file "/pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_external_dns_e16c7455-6a1b-459a-bda2-63357961e027/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 23451 -> 27229
11692024-10-03T20:37:52.159Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_external_dns_e16c7455-6a1b-459a-bda2-63357961e027/root/var/svc/log/oxide-external_dns:default.log
11702024-10-03T20:37:52.159Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_external_dns_e16c7455-6a1b-459a-bda2-63357961e027/root/var/svc/log/oxide-opte-interface-setup:default.log (1156 bytes)
11712024-10-03T20:37:53.251Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_external_dns_e16c7455-6a1b-459a-bda2-63357961e027/root/var/svc/log/oxide-opte-interface-setup:default.log
11722024-10-03T20:37:53.251Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_external_dns_e16c7455-6a1b-459a-bda2-63357961e027/root/var/svc/log/oxide-zone-network-setup:default.log (2111 bytes)
11732024-10-03T20:37:54.347Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_external_dns_e16c7455-6a1b-459a-bda2-63357961e027/root/var/svc/log/oxide-zone-network-setup:default.log
11742024-10-03T20:37:54.347Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_internal_dns_3e467632-ea6c-446e-934a-4e4b2c63a244/root/var/svc/log/oxide-internal_dns:default.log (399339 bytes)
11752024-10-03T20:37:54.572Zupload warning: file "/pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_internal_dns_3e467632-ea6c-446e-934a-4e4b2c63a244/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 399339 -> 403093
11762024-10-03T20:37:55.641Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_internal_dns_3e467632-ea6c-446e-934a-4e4b2c63a244/root/var/svc/log/oxide-internal_dns:default.log
11772024-10-03T20:37:55.641Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_internal_dns_3e467632-ea6c-446e-934a-4e4b2c63a244/root/var/svc/log/oxide-zone-network-setup:default.log (2094 bytes)
11782024-10-03T20:37:56.793Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_internal_dns_3e467632-ea6c-446e-934a-4e4b2c63a244/root/var/svc/log/oxide-zone-network-setup:default.log
11792024-10-03T20:37:56.870Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_nexus_edfa7356-d8fe-4100-b01b-f03be311c5d2/root/var/svc/log/oxide-nexus:default.log (35782228 bytes)
11802024-10-03T20:37:58.261Zupload warning: file "/pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_nexus_edfa7356-d8fe-4100-b01b-f03be311c5d2/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 35782228 -> 36170401
11812024-10-03T20:37:59.329Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_nexus_edfa7356-d8fe-4100-b01b-f03be311c5d2/root/var/svc/log/oxide-nexus:default.log
11822024-10-03T20:37:59.329Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_nexus_edfa7356-d8fe-4100-b01b-f03be311c5d2/root/var/svc/log/oxide-opte-interface-setup:default.log (1135 bytes)
11832024-10-03T20:38:00.421Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_nexus_edfa7356-d8fe-4100-b01b-f03be311c5d2/root/var/svc/log/oxide-opte-interface-setup:default.log
11842024-10-03T20:38:00.421Zuploading: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_nexus_edfa7356-d8fe-4100-b01b-f03be311c5d2/root/var/svc/log/oxide-zone-network-setup:default.log (2055 bytes)
11852024-10-03T20:38:01.513Zuploaded: /pool/ext/9be3601f-65b1-4f43-9377-aa5513d9b31a/crypt/zone/oxz_nexus_edfa7356-d8fe-4100-b01b-f03be311c5d2/root/var/svc/log/oxide-zone-network-setup:default.log
11862024-10-03T20:38:01.513Zuploading: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_cockroachdb_a5cdf7ab-d77a-4af7-b844-163e49b18349/root/var/svc/log/oxide-cockroach-admin:default.log (4696 bytes)
11872024-10-03T20:38:02.606Zuploaded: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_cockroachdb_a5cdf7ab-d77a-4af7-b844-163e49b18349/root/var/svc/log/oxide-cockroach-admin:default.log
11882024-10-03T20:38:02.606Zuploading: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_cockroachdb_a5cdf7ab-d77a-4af7-b844-163e49b18349/root/var/svc/log/oxide-cockroachdb:default.log (4315 bytes)
11892024-10-03T20:38:03.702Zuploaded: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_cockroachdb_a5cdf7ab-d77a-4af7-b844-163e49b18349/root/var/svc/log/oxide-cockroachdb:default.log
11902024-10-03T20:38:03.702Zuploading: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_cockroachdb_a5cdf7ab-d77a-4af7-b844-163e49b18349/root/var/svc/log/oxide-zone-network-setup:default.log (2090 bytes)
11912024-10-03T20:38:04.800Zuploaded: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_cockroachdb_a5cdf7ab-d77a-4af7-b844-163e49b18349/root/var/svc/log/oxide-zone-network-setup:default.log
11922024-10-03T20:38:04.800Zuploading: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_crucible_1efebcfb-f493-45ea-8271-aeca5471fc7f/root/var/svc/log/oxide-crucible-agent:default.log (36019 bytes)
11932024-10-03T20:38:05.971Zuploaded: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_crucible_1efebcfb-f493-45ea-8271-aeca5471fc7f/root/var/svc/log/oxide-crucible-agent:default.log
11942024-10-03T20:38:05.971Zuploading: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_crucible_1efebcfb-f493-45ea-8271-aeca5471fc7f/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11952024-10-03T20:38:07.067Zuploaded: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_crucible_1efebcfb-f493-45ea-8271-aeca5471fc7f/root/var/svc/log/oxide-crucible-downstairs:default.log
11962024-10-03T20:38:07.067Zuploading: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_crucible_1efebcfb-f493-45ea-8271-aeca5471fc7f/root/var/svc/log/oxide-crucible-downstairs:downstairs-4bd29bfa-66c2-4612-9f43-8c789d2ff822.log (6076 bytes)
11972024-10-03T20:38:08.193Zuploaded: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_crucible_1efebcfb-f493-45ea-8271-aeca5471fc7f/root/var/svc/log/oxide-crucible-downstairs:downstairs-4bd29bfa-66c2-4612-9f43-8c789d2ff822.log
11982024-10-03T20:38:08.194Zuploading: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_crucible_1efebcfb-f493-45ea-8271-aeca5471fc7f/root/var/svc/log/oxide-crucible-downstairs:downstairs-a1ba0e7d-2f0c-47b9-8e92-061fa96207d7.log (5993 bytes)
11992024-10-03T20:38:09.342Zuploaded: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_crucible_1efebcfb-f493-45ea-8271-aeca5471fc7f/root/var/svc/log/oxide-crucible-downstairs:downstairs-a1ba0e7d-2f0c-47b9-8e92-061fa96207d7.log
12002024-10-03T20:38:09.342Zuploading: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_crucible_1efebcfb-f493-45ea-8271-aeca5471fc7f/root/var/svc/log/oxide-crucible-downstairs:snapshot-4bd29bfa-66c2-4612-9f43-8c789d2ff822-bec3d930-a591-451d-82e5-be08acca0f21.log (5686 bytes)
12012024-10-03T20:38:10.458Zuploaded: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_crucible_1efebcfb-f493-45ea-8271-aeca5471fc7f/root/var/svc/log/oxide-crucible-downstairs:snapshot-4bd29bfa-66c2-4612-9f43-8c789d2ff822-bec3d930-a591-451d-82e5-be08acca0f21.log
12022024-10-03T20:38:10.458Zuploading: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_crucible_1efebcfb-f493-45ea-8271-aeca5471fc7f/root/var/svc/log/oxide-zone-network-setup:default.log (2082 bytes)
12032024-10-03T20:38:11.671Zuploaded: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_crucible_1efebcfb-f493-45ea-8271-aeca5471fc7f/root/var/svc/log/oxide-zone-network-setup:default.log
12042024-10-03T20:38:11.671Zuploading: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_oximeter_b2738475-b101-4721-bd4b-1d02e9fffd16/root/var/svc/log/oxide-oximeter:default.log (1221960 bytes)
12052024-10-03T20:38:12.154Zupload warning: file "/pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_oximeter_b2738475-b101-4721-bd4b-1d02e9fffd16/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 1221960 -> 1607108
12062024-10-03T20:38:13.329Zuploaded: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_oximeter_b2738475-b101-4721-bd4b-1d02e9fffd16/root/var/svc/log/oxide-oximeter:default.log
12072024-10-03T20:38:13.329Zuploading: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_oximeter_b2738475-b101-4721-bd4b-1d02e9fffd16/root/var/svc/log/oxide-zone-network-setup:default.log (2081 bytes)
12082024-10-03T20:38:14.497Zuploaded: /pool/ext/bb86cec0-ba38-4d2f-b865-3bb4c225bc60/crypt/zone/oxz_oximeter_b2738475-b101-4721-bd4b-1d02e9fffd16/root/var/svc/log/oxide-zone-network-setup:default.log
12092024-10-03T20:38:14.497Zuploading: /pool/ext/bbefb67e-cb66-4fc4-8b99-53014030c8bf/crypt/zone/oxz_cockroachdb_2898cc25-5215-4939-869d-96d5a9fa5daa/root/var/svc/log/oxide-cockroach-admin:default.log (4696 bytes)
12102024-10-03T20:38:15.610Zuploaded: /pool/ext/bbefb67e-cb66-4fc4-8b99-53014030c8bf/crypt/zone/oxz_cockroachdb_2898cc25-5215-4939-869d-96d5a9fa5daa/root/var/svc/log/oxide-cockroach-admin:default.log
12112024-10-03T20:38:15.610Zuploading: /pool/ext/bbefb67e-cb66-4fc4-8b99-53014030c8bf/crypt/zone/oxz_cockroachdb_2898cc25-5215-4939-869d-96d5a9fa5daa/root/var/svc/log/oxide-cockroachdb:default.log (4315 bytes)
12122024-10-03T20:38:16.707Zuploaded: /pool/ext/bbefb67e-cb66-4fc4-8b99-53014030c8bf/crypt/zone/oxz_cockroachdb_2898cc25-5215-4939-869d-96d5a9fa5daa/root/var/svc/log/oxide-cockroachdb:default.log
12132024-10-03T20:38:16.707Zuploading: /pool/ext/bbefb67e-cb66-4fc4-8b99-53014030c8bf/crypt/zone/oxz_cockroachdb_2898cc25-5215-4939-869d-96d5a9fa5daa/root/var/svc/log/oxide-zone-network-setup:default.log (2094 bytes)
12142024-10-03T20:38:17.803Zuploaded: /pool/ext/bbefb67e-cb66-4fc4-8b99-53014030c8bf/crypt/zone/oxz_cockroachdb_2898cc25-5215-4939-869d-96d5a9fa5daa/root/var/svc/log/oxide-zone-network-setup:default.log
12152024-10-03T20:38:17.803Zuploading: /pool/ext/bbefb67e-cb66-4fc4-8b99-53014030c8bf/crypt/zone/oxz_crucible_b3c85f58-2961-4708-ae11-eb8f3ccb12fd/root/var/svc/log/oxide-crucible-agent:default.log (1555 bytes)
12162024-10-03T20:38:18.934Zuploaded: /pool/ext/bbefb67e-cb66-4fc4-8b99-53014030c8bf/crypt/zone/oxz_crucible_b3c85f58-2961-4708-ae11-eb8f3ccb12fd/root/var/svc/log/oxide-crucible-agent:default.log
12172024-10-03T20:38:18.934Zuploading: /pool/ext/bbefb67e-cb66-4fc4-8b99-53014030c8bf/crypt/zone/oxz_crucible_b3c85f58-2961-4708-ae11-eb8f3ccb12fd/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12182024-10-03T20:38:20.054Zuploaded: /pool/ext/bbefb67e-cb66-4fc4-8b99-53014030c8bf/crypt/zone/oxz_crucible_b3c85f58-2961-4708-ae11-eb8f3ccb12fd/root/var/svc/log/oxide-crucible-downstairs:default.log
12192024-10-03T20:38:20.054Zuploading: /pool/ext/bbefb67e-cb66-4fc4-8b99-53014030c8bf/crypt/zone/oxz_crucible_b3c85f58-2961-4708-ae11-eb8f3ccb12fd/root/var/svc/log/oxide-zone-network-setup:default.log (2082 bytes)
12202024-10-03T20:38:21.146Zuploaded: /pool/ext/bbefb67e-cb66-4fc4-8b99-53014030c8bf/crypt/zone/oxz_crucible_b3c85f58-2961-4708-ae11-eb8f3ccb12fd/root/var/svc/log/oxide-zone-network-setup:default.log
12212024-10-03T20:38:21.170Zuploading: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_cockroachdb_04066342-4569-4979-8492-9802d7f37f30/root/var/svc/log/oxide-cockroach-admin:default.log (4698 bytes)
12222024-10-03T20:38:22.238Zuploaded: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_cockroachdb_04066342-4569-4979-8492-9802d7f37f30/root/var/svc/log/oxide-cockroach-admin:default.log
12232024-10-03T20:38:22.238Zuploading: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_cockroachdb_04066342-4569-4979-8492-9802d7f37f30/root/var/svc/log/oxide-cockroachdb:default.log (4313 bytes)
12242024-10-03T20:38:23.340Zuploaded: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_cockroachdb_04066342-4569-4979-8492-9802d7f37f30/root/var/svc/log/oxide-cockroachdb:default.log
12252024-10-03T20:38:23.340Zuploading: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_cockroachdb_04066342-4569-4979-8492-9802d7f37f30/root/var/svc/log/oxide-zone-network-setup:default.log (2094 bytes)
12262024-10-03T20:38:24.432Zuploaded: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_cockroachdb_04066342-4569-4979-8492-9802d7f37f30/root/var/svc/log/oxide-zone-network-setup:default.log
12272024-10-03T20:38:24.432Zuploading: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-crucible-agent:default.log (38783 bytes)
12282024-10-03T20:38:25.589Zuploaded: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-crucible-agent:default.log
12292024-10-03T20:38:25.589Zuploading: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12302024-10-03T20:38:26.680Zuploaded: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-crucible-downstairs:default.log
12312024-10-03T20:38:26.680Zuploading: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-crucible-downstairs:downstairs-54249848-dd06-4e42-a336-c8e751c02810.log (5994 bytes)
12322024-10-03T20:38:27.772Zuploaded: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-crucible-downstairs:downstairs-54249848-dd06-4e42-a336-c8e751c02810.log
12332024-10-03T20:38:27.772Zuploading: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-crucible-downstairs:downstairs-65e69107-8969-458f-82a7-9eb00e23f97f.log (3126 bytes)
12342024-10-03T20:38:28.863Zuploaded: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-crucible-downstairs:downstairs-65e69107-8969-458f-82a7-9eb00e23f97f.log
12352024-10-03T20:38:28.863Zuploading: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-crucible-downstairs:downstairs-fb92a9b9-79ff-4c09-9ec6-3dc468cb1969.log (6076 bytes)
12362024-10-03T20:38:29.955Zuploaded: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-crucible-downstairs:downstairs-fb92a9b9-79ff-4c09-9ec6-3dc468cb1969.log
12372024-10-03T20:38:29.955Zuploading: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-crucible-downstairs:snapshot-fb92a9b9-79ff-4c09-9ec6-3dc468cb1969-bec3d930-a591-451d-82e5-be08acca0f21.log (5687 bytes)
12382024-10-03T20:38:31.048Zuploaded: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-crucible-downstairs:snapshot-fb92a9b9-79ff-4c09-9ec6-3dc468cb1969-bec3d930-a591-451d-82e5-be08acca0f21.log
12392024-10-03T20:38:31.048Zuploading: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-zone-network-setup:default.log (2081 bytes)
12402024-10-03T20:38:32.140Zuploaded: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_crucible_435514a1-7f04-4079-96c5-f901c0639856/root/var/svc/log/oxide-zone-network-setup:default.log
12412024-10-03T20:38:32.140Zuploading: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_internal_dns_30aa01c9-780d-4456-824b-3ef76ad5ee62/root/var/svc/log/oxide-internal_dns:default.log (399334 bytes)
12422024-10-03T20:38:32.364Zupload warning: file "/pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_internal_dns_30aa01c9-780d-4456-824b-3ef76ad5ee62/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 399334 -> 406844
12432024-10-03T20:38:33.432Zuploaded: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_internal_dns_30aa01c9-780d-4456-824b-3ef76ad5ee62/root/var/svc/log/oxide-internal_dns:default.log
12442024-10-03T20:38:33.433Zuploading: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_internal_dns_30aa01c9-780d-4456-824b-3ef76ad5ee62/root/var/svc/log/oxide-zone-network-setup:default.log (2094 bytes)
12452024-10-03T20:38:34.528Zuploaded: /pool/ext/cced9d79-ed80-4c89-994e-8eaec18a3ffe/crypt/zone/oxz_internal_dns_30aa01c9-780d-4456-824b-3ef76ad5ee62/root/var/svc/log/oxide-zone-network-setup:default.log
12462024-10-03T20:38:34.557Zuploading: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_crucible_4eaf4311-cec9-49e9-8b5f-3fd064339493/root/var/svc/log/oxide-crucible-agent:default.log (8912 bytes)
12472024-10-03T20:38:35.641Zuploaded: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_crucible_4eaf4311-cec9-49e9-8b5f-3fd064339493/root/var/svc/log/oxide-crucible-agent:default.log
12482024-10-03T20:38:35.641Zuploading: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_crucible_4eaf4311-cec9-49e9-8b5f-3fd064339493/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12492024-10-03T20:38:36.733Zuploaded: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_crucible_4eaf4311-cec9-49e9-8b5f-3fd064339493/root/var/svc/log/oxide-crucible-downstairs:default.log
12502024-10-03T20:38:36.733Zuploading: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_crucible_4eaf4311-cec9-49e9-8b5f-3fd064339493/root/var/svc/log/oxide-crucible-downstairs:downstairs-15d9bcfc-de83-45a1-b7b5-d79e355d8dd6.log (3020 bytes)
12512024-10-03T20:38:37.835Zuploaded: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_crucible_4eaf4311-cec9-49e9-8b5f-3fd064339493/root/var/svc/log/oxide-crucible-downstairs:downstairs-15d9bcfc-de83-45a1-b7b5-d79e355d8dd6.log
12522024-10-03T20:38:37.835Zuploading: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_crucible_4eaf4311-cec9-49e9-8b5f-3fd064339493/root/var/svc/log/oxide-zone-network-setup:default.log (2082 bytes)
12532024-10-03T20:38:38.921Zuploaded: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_crucible_4eaf4311-cec9-49e9-8b5f-3fd064339493/root/var/svc/log/oxide-zone-network-setup:default.log
12542024-10-03T20:38:38.921Zuploading: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_nexus_4955bbb7-4e36-47ac-bedf-e040f180a990/root/var/svc/log/oxide-nexus:default.log (4117161 bytes)
12552024-10-03T20:38:39.611Zupload warning: file "/pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_nexus_4955bbb7-4e36-47ac-bedf-e040f180a990/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 4117161 -> 5057066
12562024-10-03T20:38:40.680Zuploaded: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_nexus_4955bbb7-4e36-47ac-bedf-e040f180a990/root/var/svc/log/oxide-nexus:default.log
12572024-10-03T20:38:40.705Zuploading: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_nexus_4955bbb7-4e36-47ac-bedf-e040f180a990/root/var/svc/log/oxide-opte-interface-setup:default.log (1134 bytes)
12582024-10-03T20:38:41.774Zuploaded: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_nexus_4955bbb7-4e36-47ac-bedf-e040f180a990/root/var/svc/log/oxide-opte-interface-setup:default.log
12592024-10-03T20:38:41.774Zuploading: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_nexus_4955bbb7-4e36-47ac-bedf-e040f180a990/root/var/svc/log/oxide-zone-network-setup:default.log (2057 bytes)
12602024-10-03T20:38:42.866Zuploaded: /pool/ext/cd2cdd9e-d8f2-4e2c-82d3-c2e2c498dc39/crypt/zone/oxz_nexus_4955bbb7-4e36-47ac-bedf-e040f180a990/root/var/svc/log/oxide-zone-network-setup:default.log
12612024-10-03T20:38:42.866Zuploading: /pool/ext/dab8d29a-db6e-412c-9f30-46af8b4a3a6f/crypt/zone/oxz_crucible_ec9f692f-185b-4065-b0ed-73ee322ed1eb/root/var/svc/log/oxide-crucible-agent:default.log (12074 bytes)
12622024-10-03T20:38:44.054Zuploaded: /pool/ext/dab8d29a-db6e-412c-9f30-46af8b4a3a6f/crypt/zone/oxz_crucible_ec9f692f-185b-4065-b0ed-73ee322ed1eb/root/var/svc/log/oxide-crucible-agent:default.log
12632024-10-03T20:38:44.054Zuploading: /pool/ext/dab8d29a-db6e-412c-9f30-46af8b4a3a6f/crypt/zone/oxz_crucible_ec9f692f-185b-4065-b0ed-73ee322ed1eb/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12642024-10-03T20:38:45.155Zuploaded: /pool/ext/dab8d29a-db6e-412c-9f30-46af8b4a3a6f/crypt/zone/oxz_crucible_ec9f692f-185b-4065-b0ed-73ee322ed1eb/root/var/svc/log/oxide-crucible-downstairs:default.log
12652024-10-03T20:38:45.155Zuploading: /pool/ext/dab8d29a-db6e-412c-9f30-46af8b4a3a6f/crypt/zone/oxz_crucible_ec9f692f-185b-4065-b0ed-73ee322ed1eb/root/var/svc/log/oxide-crucible-downstairs:downstairs-18150462-20c3-44b1-8412-759a8fd0ddfb.log (3127 bytes)
12662024-10-03T20:38:46.246Zuploaded: /pool/ext/dab8d29a-db6e-412c-9f30-46af8b4a3a6f/crypt/zone/oxz_crucible_ec9f692f-185b-4065-b0ed-73ee322ed1eb/root/var/svc/log/oxide-crucible-downstairs:downstairs-18150462-20c3-44b1-8412-759a8fd0ddfb.log
12672024-10-03T20:38:46.246Zuploading: /pool/ext/dab8d29a-db6e-412c-9f30-46af8b4a3a6f/crypt/zone/oxz_crucible_ec9f692f-185b-4065-b0ed-73ee322ed1eb/root/var/svc/log/oxide-zone-network-setup:default.log (2082 bytes)
12682024-10-03T20:38:47.338Zuploaded: /pool/ext/dab8d29a-db6e-412c-9f30-46af8b4a3a6f/crypt/zone/oxz_crucible_ec9f692f-185b-4065-b0ed-73ee322ed1eb/root/var/svc/log/oxide-zone-network-setup:default.log
12692024-10-03T20:38:47.362Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/log/chrony/measurements.log (65897 bytes)
12702024-10-03T20:38:47.516Zupload warning: file "/pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/log/chrony/measurements.log" changed size mid upload: 65897 -> 97818
12712024-10-03T20:38:48.584Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/log/chrony/measurements.log
12722024-10-03T20:38:48.584Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/log/chrony/statistics.log (48789 bytes)
12732024-10-03T20:38:48.720Zupload warning: file "/pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/log/chrony/statistics.log" changed size mid upload: 48789 -> 71019
12742024-10-03T20:38:49.790Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/log/chrony/statistics.log
12752024-10-03T20:38:49.790Zuploading: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/log/chrony/tracking.log (10959 bytes)
12762024-10-03T20:38:49.890Zupload warning: file "/pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/log/chrony/tracking.log" changed size mid upload: 10959 -> 15843
12772024-10-03T20:38:50.958Zuploaded: /pool/ext/7f092e33-1af0-4e4e-87b6-07db26b26d4d/crypt/zone/oxz_ntp_17492d37-9cb6-4c17-a8d5-bb46b5ac1c2d/root/var/log/chrony/tracking.log