01J8FXSTVFEP5F209MXAWQ2DQR: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log*",
        "%/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.33",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01J8FXTXJHFC5VC552HZTFRJMD

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12024-09-23T17:42:50.976Zjob dependencies complete; ready to run (waiting for 31 m 50 s)
22024-09-23T18:13:48.186Zjob assigned to worker 01J8G1CGMD33C2QN9Y3QEYAWKW (queued for 30 m 57 s)
32024-09-23T18:13:48.352Zhttp://172.20.0.5:9969/ipxe/sock... ok
42024-09-23T18:13:48.401Zxcvsh0tfvd xcvsh0tfvd xcvsh0tfvd xcvsh0tfvd xcvsh0tfvd xcvsh0tfvd xcvsh0tfvd xcvsh0tfvd
52024-09-23T18:13:48.435Zhttp://172.20.0.5:9969/ipxe/sock... ok
62024-09-23T18:13:48.468Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/kernel/amd64/unix... ok
72024-09-23T18:13:48.502Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/amd64/boot_archive... ok
82024-09-23T18:13:48.535Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/amd64/boot_archive.hash... ok
92024-09-23T18:13:48.569Zhttp://172.20.0.5:9969/postboot/sock... ok
102024-09-23T18:13:48.602ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
112024-09-23T18:13:48.636Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d5
122024-09-23T18:13:48.669ZOxide Helios Version helios-2.0.22827 64-bit
132024-09-23T18:13:48.703ZConfiguring devices.
142024-09-23T18:13:48.736Z##################################################
152024-09-23T18:13:48.770Z#### ############# ##
162024-09-23T18:13:48.806Z### ### ############ ##
172024-09-23T18:13:48.837Z## ### # ## ### ## Oxide Computer Company ##
182024-09-23T18:13:48.870Z## ## ## ### # ### ##
192024-09-23T18:13:48.903Z## # ### #### #### This Station Under ##
202024-09-23T18:13:48.937Z### ### #### # ### Computer Control ##
212024-09-23T18:13:48.970Z#### #### ### ## ##
222024-09-23T18:13:49.004Z##################################################
232024-09-23T18:13:49.059ZHostname: sock
242024-09-23T18:13:49.092ZLoading smf(7) service descriptions:
252024-09-23T18:13:49.404Zdownloading input: /input/package/work/package.tar.gz
262024-09-23T18:18:22.090Zdownloaded input: /input/package/work/package.tar.gz
 
272024-09-23T18:18:22.090Zstarting task 0: "setup"
282024-09-23T18:18:22.114Z++ uname -s
292024-09-23T18:18:22.114Z+ kern=SunOS
302024-09-23T18:18:22.114Z+ case "$kern" in
312024-09-23T18:18:22.114Z+ groupadd -g 12345 build
322024-09-23T18:18:22.114Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
332024-09-23T18:18:24.105Z+ zfs create -o mountpoint=/work rpool/work
342024-09-23T18:18:24.128Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
352024-09-23T18:18:24.128Z+ home_fs=zfs
362024-09-23T18:18:24.128Z+ [[ zfs == autofs ]]
372024-09-23T18:18:24.128Z+ mkdir -p /home/build
382024-09-23T18:18:24.150Z+ chown build:build /home/build /work
392024-09-23T18:18:26.130Z+ chmod 0700 /home/build /work
402024-09-23T18:18:26.153Zprocess exited: duration 4047 ms, exit code 0
 
412024-09-23T18:18:26.199Zstarting task 1: "authentication"
422024-09-23T18:18:26.242Zprocess exited: duration 42 ms, exit code 0
 
432024-09-23T18:18:26.289Zstarting task 2: "build"
442024-09-23T18:18:26.312Z+ trap _exit_trap EXIT
452024-09-23T18:18:26.312Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
462024-09-23T18:18:26.407Zsecpolicy_fs_linkdir: 0x55 = 0x55
472024-09-23T18:18:26.408Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
482024-09-23T18:18:26.408Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
492024-09-23T18:18:26.408Zsecpolicy_fs_linkdir+3: 0 = 0xe5
502024-09-23T18:18:26.408Zsecpolicy_fs_linkdir+4: 0 = 0xb8
512024-09-23T18:18:26.408Zsecpolicy_fs_linkdir+5: 0 = 0x1
522024-09-23T18:18:26.408Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
532024-09-23T18:18:26.408Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
542024-09-23T18:18:26.408Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
552024-09-23T18:18:26.408Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
562024-09-23T18:18:26.408Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
572024-09-23T18:18:26.432Z+ [[ -d /opt/oxide ]]
582024-09-23T18:18:26.432Z+ pfexec mv /opt/oxide /opt/oxide-underneath
592024-09-23T18:18:26.432Z+ pfexec mkdir /opt/oxide
602024-09-23T18:18:26.432Z+ pfexec mount -F tmpfs -O swap /opt/oxide
612024-09-23T18:18:26.432Z+ [[ -d /opt/oxide-underneath ]]
622024-09-23T18:18:26.432Z+ cd /opt/oxide-underneath
632024-09-23T18:18:26.432Z+ pfexec tar ceEp@/f - .
642024-09-23T18:18:26.432Z+ cd /opt/oxide
652024-09-23T18:18:26.432Z+ pfexec tar xveEp@/f -
662024-09-23T18:18:26.536Zx ., 0 bytes, 0 tape blocks
672024-09-23T18:18:26.536Zx ./opte, 0 bytes, 0 tape blocks
682024-09-23T18:18:26.536Zx ./opte/bin, 0 bytes, 0 tape blocks
692024-09-23T18:18:26.536Zx ./opte/bin/opteadm, 52693888 bytes, 102918 tape blocks
702024-09-23T18:18:26.559Z+ pfexec rm -rf /opt/oxide-underneath
712024-09-23T18:18:26.559Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
722024-09-23T18:18:26.584Z+ pfexec mkdir /opt/oxide/work
732024-09-23T18:18:26.584Z+ pfexec chown build:build /opt/oxide/work
742024-09-23T18:18:26.584Z+ cd /opt/oxide/work
752024-09-23T18:18:26.584Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
762024-09-23T18:18:38.841ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
772024-09-23T18:18:38.842Zx out/target/test, 107 bytes, 1 tape blocks
782024-09-23T18:18:38.842Zx out/npuzone/npuzone, 23699288 bytes, 46288 tape blocks
792024-09-23T18:18:38.842Zx package-manifest.toml, 31809 bytes, 63 tape blocks
802024-09-23T18:18:38.842Zx smf/sled-agent/non-gimlet/config.toml, 3569 bytes, 7 tape blocks
812024-09-23T18:18:38.842Zx target/release/omicron-package, 33311880 bytes, 65063 tape blocks
822024-09-23T18:18:38.842Zx target/release/xtask, 12282776 bytes, 23990 tape blocks
832024-09-23T18:18:38.842Zx target/debug/bootstrap, 198100672 bytes, 386916 tape blocks
842024-09-23T18:18:38.842Zx tests/bootstrap-4bd4b6817bc5b95b, 24548496 bytes, 47947 tape blocks
852024-09-23T18:18:38.842Zx tests/commtest-5ebf0895cc3e3c74, 24816632 bytes, 48470 tape blocks
862024-09-23T18:18:38.842Zx tests/dhcp_server-9a6f8575738de90f, 24548528 bytes, 47947 tape blocks
872024-09-23T18:18:38.842Zx tests/end_to_end_tests-cdbb8d2022e3297e, 216574384 bytes, 422997 tape blocks
882024-09-23T18:18:38.842Zx out/clickhouse.tar.gz, 239327388 bytes, 467437 tape blocks
892024-09-23T18:18:38.842Zx out/clickhouse_keeper.tar.gz, 239297735 bytes, 467379 tape blocks
902024-09-23T18:18:38.842Zx out/clickhouse_server.tar.gz, 239293116 bytes, 467370 tape blocks
912024-09-23T18:18:38.842Zx out/cockroachdb.tar.gz, 135356044 bytes, 264368 tape blocks
922024-09-23T18:18:38.842Zx out/crucible-dtrace.tar, 86016 bytes, 168 tape blocks
932024-09-23T18:18:38.842Zx out/crucible-pantry-zone.tar.gz, 26601909 bytes, 51957 tape blocks
942024-09-23T18:18:38.842Zx out/crucible-zone.tar.gz, 35064810 bytes, 68486 tape blocks
952024-09-23T18:18:38.842Zx out/external-dns.tar.gz, 25728503 bytes, 50251 tape blocks
962024-09-23T18:18:38.842Zx out/internal-dns.tar.gz, 25728731 bytes, 50252 tape blocks
972024-09-23T18:18:38.842Zx out/mg-ddm-gz.tar, 63651840 bytes, 124320 tape blocks
982024-09-23T18:18:38.842Zx out/nexus.tar.gz, 95046047 bytes, 185637 tape blocks
992024-09-23T18:18:38.842Zx out/ntp.tar.gz, 9271899 bytes, 18110 tape blocks
1002024-09-23T18:18:38.842Zx out/omicron-gateway-softnpu.tar.gz, 21399615 bytes, 41797 tape blocks
1012024-09-23T18:18:38.842Zx out/omicron-sled-agent.tar, 104543232 bytes, 204186 tape blocks
1022024-09-23T18:18:38.842Zx out/overlay.tar.gz, 2622 bytes, 6 tape blocks
1032024-09-23T18:18:38.842Zx out/oximeter.tar.gz, 44479189 bytes, 86874 tape blocks
1042024-09-23T18:18:38.842Zx out/oxlog.tar, 3110400 bytes, 6075 tape blocks
1052024-09-23T18:18:38.842Zx out/probe.tar.gz, 2829470 bytes, 5527 tape blocks
1062024-09-23T18:18:38.842Zx out/propolis-server.tar.gz, 67269180 bytes, 131386 tape blocks
1072024-09-23T18:18:38.842Zx out/pumpkind-gz.tar, 27304960 bytes, 53330 tape blocks
1082024-09-23T18:18:38.842Zx out/switch-softnpu.tar.gz, 223526395 bytes, 436575 tape blocks
1092024-09-23T18:18:38.867Z
1102024-09-23T18:18:38.867Zreal 12.246485292
1112024-09-23T18:18:38.867Zuser 10.835003640
1122024-09-23T18:18:38.867Zsys 2.449021422
1132024-09-23T18:18:38.867Ztrap 0.000454149
1142024-09-23T18:18:38.867Ztflt 0.000000000
1152024-09-23T18:18:38.867Zdflt 0.000000000
1162024-09-23T18:18:38.867Zkflt 0.000000000
1172024-09-23T18:18:38.867Zlock 0.000000000
1182024-09-23T18:18:38.867Zslp 10.924256829
1192024-09-23T18:18:38.867Zlat 0.282254217
1202024-09-23T18:18:38.867Zstop 0.000129103
1212024-09-23T18:18:38.867Z+ bmat address ls
1222024-09-23T18:18:38.868ZNAME CIDR FIRST LAST COUNT
1232024-09-23T18:18:38.868Zextra 10.151.2.0/24 10.151.2.164 10.151.2.227 64
1242024-09-23T18:18:38.868Z++ bmat address ls -f extra -Ho first,last,count
1252024-09-23T18:18:38.868Z+ set -- 10.151.2.164 10.151.2.227 64
1262024-09-23T18:18:38.868Z+ EXTRA_IP_START=10.151.2.164
1272024-09-23T18:18:38.868Z+ EXTRA_IP_END=10.151.2.227
1282024-09-23T18:18:38.868Z+ EXTRA_IP_COUNT=64
1292024-09-23T18:18:38.868Z+ (( EXTRA_IP_COUNT >= 32 ))
1302024-09-23T18:18:38.868Z+ EXTRA_IP_BASE=10.151.2
1312024-09-23T18:18:38.868Z+ EXTRA_IP_FOCTET=164
1322024-09-23T18:18:38.868Z+ EXTRA_IP_LOCTET=227
1332024-09-23T18:18:38.868Z+ SERVICE_IP_POOL_START=10.151.2.164
1342024-09-23T18:18:38.868Z+ SERVICE_IP_POOL_END=10.151.2.173
1352024-09-23T18:18:38.868Z+ DNS_IP1=10.151.2.164
1362024-09-23T18:18:38.868Z+ DNS_IP2=10.151.2.165
1372024-09-23T18:18:38.868Z+ UPLINK_IP=10.151.2.174
1382024-09-23T18:18:38.868Z+ IPPOOL_START=10.151.2.175
1392024-09-23T18:18:38.869Z+ IPPOOL_END=10.151.2.227
1402024-09-23T18:18:38.869Z++ ipadm show-addr -po type,addr
1412024-09-23T18:18:38.869Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
1422024-09-23T18:18:38.869Z+ GATEWAY_IP=10.151.2.100
1432024-09-23T18:18:38.869Z+ [[ -n 10.151.2.100 ]]
1442024-09-23T18:18:38.869Z+ ping -s 10.151.2.100 56 1
1452024-09-23T18:18:38.869ZPING 10.151.2.100: 56 data bytes
1462024-09-23T18:18:38.869Z64 bytes from 10.151.2.100: icmp_seq=0. time=0.012 ms
1472024-09-23T18:18:38.869Z
1482024-09-23T18:18:38.869Z----10.151.2.100 PING Statistics----
1492024-09-23T18:18:38.869Z1 packets transmitted, 1 packets received, 0% packet loss
1502024-09-23T18:18:38.869Zround-trip (ms) min/avg/max/stddev = 0.012/0.012/0.012/-nan
1512024-09-23T18:18:38.869Z++ arp -an
1522024-09-23T18:18:38.869Z++ awk -vgw=10.151.2.100 '$2 == gw {print $NF}'
1532024-09-23T18:18:38.869Z+ GATEWAY_MAC=18:c0:4d:0d:a0:2a
1542024-09-23T18:18:38.869Z+ routeadm -e ipv4-forwarding -u
1552024-09-23T18:18:38.976Z+ PXA_START=10.151.2.164
1562024-09-23T18:18:38.976Z+ PXA_END=10.151.2.227
1572024-09-23T18:18:38.999Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
1582024-09-23T18:18:39.110Z+ ptime -m pfexec ./target/release/xtask virtual-hardware --vdev-dir /scratch create --gateway-ip 10.151.2.100 --gateway-mac 18:c0:4d:0d:a0:2a --pxa-start 10.151.2.164 --pxa-end 10.151.2.227
1592024-09-23T18:18:39.134ZCouldn't find Cargo.toml, using /opt/oxide/work as workspace root
1602024-09-23T18:18:39.189Zcreating virtual hardware
1612024-09-23T18:18:39.212Zcreating /scratch/m2_0.vdev
1622024-09-23T18:18:39.212Zcreating /scratch/m2_1.vdev
1632024-09-23T18:18:39.212Zcreating /scratch/u2_0.vdev
1642024-09-23T18:18:39.212Zcreating /scratch/u2_1.vdev
1652024-09-23T18:18:39.212Zcreating /scratch/u2_2.vdev
1662024-09-23T18:18:39.212Zcreating /scratch/u2_3.vdev
1672024-09-23T18:18:39.212Zcreating /scratch/u2_4.vdev
1682024-09-23T18:18:39.212Zcreating /scratch/u2_5.vdev
1692024-09-23T18:18:39.212Zcreating /scratch/u2_6.vdev
1702024-09-23T18:18:39.212Zcreating /scratch/u2_7.vdev
1712024-09-23T18:18:39.212Zcreating /scratch/u2_8.vdev
1722024-09-23T18:18:39.245ZSimnet net0/sc0_0 exists
1732024-09-23T18:18:39.268ZSimnet net1/sc1_0 exists
1742024-09-23T18:18:39.291ZVnic sc0_1 exists
1752024-09-23T18:18:51.767ZUsing 10.151.2.100 as gateway ip
1762024-09-23T18:18:51.768Zusing 18:C0:4D:0D:A0:2A as gateway mac
1772024-09-23T18:18:51.768Zconfiguring SoftNPU ARP entry
1782024-09-23T18:18:51.795Zconfiguring SoftNPU proxy ARP
1792024-09-23T18:18:51.849ZSoftNPU state:
1802024-09-23T18:18:51.849Z local v6:
1812024-09-23T18:18:51.849Z local v4:
1822024-09-23T18:18:51.849Z router v6:
1832024-09-23T18:18:51.849Z router v4_idx:
1842024-09-23T18:18:51.873Z router v4_routes:
1852024-09-23T18:18:51.873Z resolver v4:
1862024-09-23T18:18:51.873Z 10.151.2.100 -> 18:c0:4d:0d:a0:2a
1872024-09-23T18:18:51.873Z resolver v6:
1882024-09-23T18:18:51.873Z nat v4:
1892024-09-23T18:18:51.873Z nat_v6:
1902024-09-23T18:18:51.873Z port_mac:
1912024-09-23T18:18:51.873Z proxy arp:
1922024-09-23T18:18:51.873Z 10.151.2.164/10.151.2.227: a8:e1:de:01:70:1d
1932024-09-23T18:18:51.874Zcreated virtual hardware
1942024-09-23T18:18:51.874Z
1952024-09-23T18:18:51.874Zreal 12.736423115
1962024-09-23T18:18:51.874Zuser 0.670215041
1972024-09-23T18:18:51.874Zsys 3.104404984
1982024-09-23T18:18:51.874Ztrap 0.000071346
1992024-09-23T18:18:51.874Ztflt 0.000187537
2002024-09-23T18:18:51.874Zdflt 0.000408753
2012024-09-23T18:18:51.874Zkflt 0.000000000
2022024-09-23T18:18:51.874Zlock 9:50.085320970
2032024-09-23T18:18:51.874Zslp 38.437371887
2042024-09-23T18:18:51.874Zlat 0.081760197
2052024-09-23T18:18:51.874Zstop 0.357655412
2062024-09-23T18:18:51.874Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml pkg/config.toml
2072024-09-23T18:18:51.897Z+ sed -E -i~ 's/(m2|u2)(.*\.vdev)/\/scratch\/\1\2/g' pkg/config.toml
2082024-09-23T18:18:51.897Z+ diff -u pkg/config.toml~ pkg/config.toml
2092024-09-23T18:18:51.897Z--- pkg/config.toml~ Mon Jul 24 01:21:28 2006
2102024-09-23T18:18:51.898Z+++ pkg/config.toml Mon Sep 23 18:18:51 2024
2112024-09-23T18:18:51.898Z@@ -29,18 +29,18 @@
2122024-09-23T18:18:51.898Z # These paths have the prefix of either "u2" or "m2", followed by an underscore,
2132024-09-23T18:18:51.898Z # followed by a string that is embedded into their fake serial values.
2142024-09-23T18:18:51.898Z vdevs = [
2152024-09-23T18:18:51.898Z- "m2_0.vdev",
2162024-09-23T18:18:51.898Z- "m2_1.vdev",
2172024-09-23T18:18:51.898Z+ "/scratch/m2_0.vdev",
2182024-09-23T18:18:51.898Z+ "/scratch/m2_1.vdev",
2192024-09-23T18:18:51.898Z
2202024-09-23T18:18:51.898Z- "u2_0.vdev",
2212024-09-23T18:18:51.898Z- "u2_1.vdev",
2222024-09-23T18:18:51.898Z- "u2_2.vdev",
2232024-09-23T18:18:51.898Z- "u2_3.vdev",
2242024-09-23T18:18:51.898Z- "u2_4.vdev",
2252024-09-23T18:18:51.898Z- "u2_5.vdev",
2262024-09-23T18:18:51.898Z- "u2_6.vdev",
2272024-09-23T18:18:51.898Z- "u2_7.vdev",
2282024-09-23T18:18:51.898Z- "u2_8.vdev",
2292024-09-23T18:18:51.898Z+ "/scratch/u2_0.vdev",
2302024-09-23T18:18:51.898Z+ "/scratch/u2_1.vdev",
2312024-09-23T18:18:51.898Z+ "/scratch/u2_2.vdev",
2322024-09-23T18:18:51.898Z+ "/scratch/u2_3.vdev",
2332024-09-23T18:18:51.898Z+ "/scratch/u2_4.vdev",
2342024-09-23T18:18:51.898Z+ "/scratch/u2_5.vdev",
2352024-09-23T18:18:51.898Z+ "/scratch/u2_6.vdev",
2362024-09-23T18:18:51.898Z+ "/scratch/u2_7.vdev",
2372024-09-23T18:18:51.898Z+ "/scratch/u2_8.vdev",
2382024-09-23T18:18:51.898Z ]
2392024-09-23T18:18:51.898Z
2402024-09-23T18:18:51.898Z # Percentage of usable physical DRAM to use for the VMM reservoir, which
2412024-09-23T18:18:51.898Z+ true
2422024-09-23T18:18:51.898Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
2432024-09-23T18:18:51.898Z+ SILO_NAME=recovery
2442024-09-23T18:18:51.898Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
2452024-09-23T18:18:51.898Z+ EXTERNAL_DNS_DOMAIN=oxide.test
2462024-09-23T18:18:51.898Z+ sed -i~ '
2472024-09-23T18:18:51.898Z /^external_dns_ips/c\
2482024-09-23T18:18:51.898Zexternal_dns_ips = [ "10.151.2.164", "10.151.2.165" ]
2492024-09-23T18:18:51.898Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
2502024-09-23T18:18:51.899Z /^first/c\
2512024-09-23T18:18:51.899Zfirst = "10.151.2.164"
2522024-09-23T18:18:51.899Z /^last/c\
2532024-09-23T18:18:51.899Zlast = "10.151.2.173"
2542024-09-23T18:18:51.899Z }
2552024-09-23T18:18:51.899Z /^infra_ip_first/c\
2562024-09-23T18:18:51.899Zinfra_ip_first = "10.151.2.174"
2572024-09-23T18:18:51.899Z /^infra_ip_last/c\
2582024-09-23T18:18:51.899Zinfra_ip_last = "10.151.2.174"
2592024-09-23T18:18:51.899Z /^\[\[rack_network_config.ports/,/^$/ {
2602024-09-23T18:18:51.899Z /^routes/c\
2612024-09-23T18:18:51.899Zroutes = \[{nexthop = "10.151.2.100", destination = "0.0.0.0/0"}\]
2622024-09-23T18:18:51.899Z /^addresses/c\
2632024-09-23T18:18:51.899Zaddresses = \[{address = "10.151.2.174/24"} \]
2642024-09-23T18:18:51.899Z }
2652024-09-23T18:18:51.899Z' pkg/config-rss.toml
2662024-09-23T18:18:51.899Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
2672024-09-23T18:18:51.923Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
2682024-09-23T18:18:51.923Z+++ pkg/config-rss.toml Mon Sep 23 18:18:51 2024
2692024-09-23T18:18:51.923Z@@ -33,7 +33,7 @@
2702024-09-23T18:18:51.923Z # the DNS domain delegated to the rack by the customer. Each of these addresses
2712024-09-23T18:18:51.923Z # must be contained in one of the "internal services" IP Pool ranges listed
2722024-09-23T18:18:51.923Z # below.
2732024-09-23T18:18:51.923Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
2742024-09-23T18:18:51.923Z+external_dns_ips = [ "10.151.2.164", "10.151.2.165" ]
2752024-09-23T18:18:51.923Z
2762024-09-23T18:18:51.923Z # Initial TLS certificates for the external API
2772024-09-23T18:18:51.923Z #
2782024-09-23T18:18:51.923Z@@ -69,8 +69,8 @@
2792024-09-23T18:18:51.923Z #
2802024-09-23T18:18:51.924Z # For more on this and what to put here, see docs/how-to-run.adoc.
2812024-09-23T18:18:51.924Z [[internal_services_ip_pool_ranges]]
2822024-09-23T18:18:51.924Z-first = "192.168.1.20"
2832024-09-23T18:18:51.924Z-last = "192.168.1.29"
2842024-09-23T18:18:51.924Z+first = "10.151.2.164"
2852024-09-23T18:18:51.924Z+last = "10.151.2.173"
2862024-09-23T18:18:51.924Z
2872024-09-23T18:18:51.924Z # TODO - this configuration is subject to change going forward. Ultimately these
2882024-09-23T18:18:51.924Z # parameters should be provided to the control plane via wicket, but we need to
2892024-09-23T18:18:51.924Z@@ -91,8 +91,8 @@
2902024-09-23T18:18:51.924Z # A range of IP addresses used by Boundary Services on the external network. In
2912024-09-23T18:18:51.924Z # a real system, these would be addresses of the uplink ports on the Sidecar.
2922024-09-23T18:18:51.924Z # With softnpu, only one address is used.
2932024-09-23T18:18:51.924Z-infra_ip_first = "192.168.1.30"
2942024-09-23T18:18:51.924Z-infra_ip_last = "192.168.1.30"
2952024-09-23T18:18:51.924Z+infra_ip_first = "10.151.2.174"
2962024-09-23T18:18:51.924Z+infra_ip_last = "10.151.2.174"
2972024-09-23T18:18:51.924Z
2982024-09-23T18:18:51.924Z # Configurations for BGP routers to run on the scrimlets.
2992024-09-23T18:18:51.924Z bgp = []
3002024-09-23T18:18:51.924Z@@ -100,9 +100,9 @@
3012024-09-23T18:18:51.924Z # You can configure multiple uplinks by repeating the following stanza
3022024-09-23T18:18:51.924Z [[rack_network_config.ports]]
3032024-09-23T18:18:51.924Z # Routes associated with this port.
3042024-09-23T18:18:51.924Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
3052024-09-23T18:18:51.924Z+routes = [{nexthop = "10.151.2.100", destination = "0.0.0.0/0"}]
3062024-09-23T18:18:51.924Z # Addresses associated with this port.
3072024-09-23T18:18:51.924Z-addresses = [{address = "192.168.1.30/24"}]
3082024-09-23T18:18:51.924Z+addresses = [{address = "10.151.2.174/24"} ]
3092024-09-23T18:18:51.924Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
3102024-09-23T18:18:51.924Z port = "qsfp0"
3112024-09-23T18:18:51.924Z # The speed of this port.
3122024-09-23T18:18:51.924Z+ true
3132024-09-23T18:18:51.924Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml pkg/config.toml
3142024-09-23T18:18:51.924Za pkg/config-rss.toml 8K
3152024-09-23T18:18:51.924Za pkg/config.toml 4K
3162024-09-23T18:18:51.924Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~ pkg/config.toml pkg/config.toml~
3172024-09-23T18:18:51.924Z+ TLS_NAME=recovery.sys.oxide.test
3182024-09-23T18:18:51.924Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
3192024-09-23T18:18:52.367Z.+.......+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...........+.+...+..+....+............+...+..+....+.....+....+..+...+.+........+.......+...+.....+............+.+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+.......................................+..+......+...+....+..+.+...+..+.........+...+..........+...............+...+........................+........+......+...............+......................+..+.+.................+....+...+........+.............+......+.....+.+.....+....+......+..+............+....+.....+...................+...............+...+...+.....+....+.....+............+....+..+..........+.....+...............+..........+..+.+..+....+...+........+...+....+........+...+.+......+...+......+.........+........................+..+....+.....+....+..+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3202024-09-23T18:18:53.011Z.....+.+...+.....+...+.......+...+.......................+.+......+........+.+...+..+......+...+..........+..+.......+..+...+......+.+...+...+.....+...+.+.........+......+......+...+.....+.+.....+...............................+..+..........+.....+.+..+.......+........+...+.+.....+....+......+.....+.+...+.........+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+....+...+...+...+...........+....+......+.........+..+............+.......+...+..+...+.......+...........+...+.........+...+.........+.+.....+.......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..........+..............+....+........+..................+....+...+.....+..................+.......+...+..+.......+..+.........+.......+.....+...+......+.............+......+...........+...+..........+......+.....+....+......+......+......+...+............+..+...+..........+..+......+.+...+......+..+.........+....+..+....+........+...............+.+.....+.........+.+.....+.......+............+.....+..................+...+....+.......................+...+.+......+..............+.+..+.......+.........+......+........................+...........+....+......+.....+...+................+..+...+......+....+......+.....+......+.......+...+........+.+...+...+..............+...+..........+...........+.........+...+.......+.....................+...........+......+....+.........+.........+......+.........+..+.............+.....................+....................+.+...+...............+..+.+........+....+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3212024-09-23T18:18:53.034Z-----
3222024-09-23T18:18:53.059Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
3232024-09-23T18:18:53.059Za pkg/initial-tls-cert.pem 2K
3242024-09-23T18:18:53.059Za pkg/initial-tls-key.pem 4K
3252024-09-23T18:18:53.059Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
3262024-09-23T18:18:53.059Z+ rmdir pkg
3272024-09-23T18:18:53.059Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
3282024-09-23T18:18:53.059Z+ pfexec mkdir -p /usr/oxide
3292024-09-23T18:18:53.060Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0
3302024-09-23T18:18:53.709Z+ pfexec chmod +x /usr/oxide/oxide
3312024-09-23T18:18:53.732Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
3322024-09-23T18:19:12.017Z+ OMICRON_NO_UNINSTALL=1
3332024-09-23T18:19:12.017Z+ ptime -m pfexec ./target/release/omicron-package -t test install
3342024-09-23T18:19:12.040ZLogging to: /opt/oxide/work/out/LOG
3352024-09-23T18:19:12.604Z
3362024-09-23T18:19:12.604Zreal 0.581978518
3372024-09-23T18:19:12.604Zuser 0.133184255
3382024-09-23T18:19:12.628Zsys 2.987088042
3392024-09-23T18:19:12.628Ztrap 0.000007360
3402024-09-23T18:19:12.628Ztflt 0.000003869
3412024-09-23T18:19:12.628Zdflt 0.000700290
3422024-09-23T18:19:12.628Zkflt 0.000000000
3432024-09-23T18:19:12.628Zlock 52.361186538
3442024-09-23T18:19:12.628Zslp 0.770482908
3452024-09-23T18:19:12.628Zlat 0.003170559
3462024-09-23T18:19:12.628Zstop 0.197998643
3472024-09-23T18:19:12.628Z+ retry=0
3482024-09-23T18:19:12.628Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3492024-09-23T18:19:12.628Z+ [[ 0 -gt 30 ]]
3502024-09-23T18:19:12.628Z+ sleep 1
3512024-09-23T18:19:13.619Z+ retry=1
3522024-09-23T18:19:13.619Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3532024-09-23T18:19:13.642Z+ [[ 1 -gt 30 ]]
3542024-09-23T18:19:13.642Z+ sleep 1
3552024-09-23T18:19:14.634Z+ retry=2
3562024-09-23T18:19:14.634Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3572024-09-23T18:19:14.664Z+ [[ 2 -gt 30 ]]
3582024-09-23T18:19:14.664Z+ sleep 1
3592024-09-23T18:19:15.648Z+ retry=3
3602024-09-23T18:19:15.648Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3612024-09-23T18:19:35.540Z+ [[ 3 -gt 30 ]]
3622024-09-23T18:19:35.540Z+ sleep 1
3632024-09-23T18:19:36.544Z+ retry=4
3642024-09-23T18:19:36.544Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
3652024-09-23T18:19:36.567Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
3662024-09-23T18:19:36.590Zlocal v6:
3672024-09-23T18:19:36.590Zfde9:13fd:8ea1:ac02::1
3682024-09-23T18:19:36.591Zfe80::50a9:7fff:fe56:3495
3692024-09-23T18:19:36.591Zlocal v4:
3702024-09-23T18:19:36.591Z10.151.2.174
3712024-09-23T18:19:36.591Zrouter v6:
3722024-09-23T18:19:36.591Zrouter v4_idx:
3732024-09-23T18:19:36.591Z0.0.0.0/0 -> 0
3742024-09-23T18:19:36.591Zrouter v4_routes:
3752024-09-23T18:19:36.591Z0 -> 10.151.2.100 (2)
3762024-09-23T18:19:36.591Zresolver v4:
3772024-09-23T18:19:36.591Z10.151.2.100 -> 18:c0:4d:0d:a0:2a
3782024-09-23T18:19:36.591Z10.151.2.174 -> ce:73:54:5f:f9:23
3792024-09-23T18:19:36.591Zresolver v6:
3802024-09-23T18:19:36.591Zfe80::8852:17ff:fe04:b5a2 -> 8a:52:17:04:b5:a2
3812024-09-23T18:19:36.591Zfe80::50a9:7fff:fe56:3495 -> 52:a9:7f:56:34:95
3822024-09-23T18:19:36.591Znat v4:
3832024-09-23T18:19:36.591Znat_v6:
3842024-09-23T18:19:36.591Zport_mac:
3852024-09-23T18:19:36.591Z1: a8:40:25:18:2c:29
3862024-09-23T18:19:36.591Z1000: a8:40:25:18:2c:28
3872024-09-23T18:19:36.591Z2: a8:40:25:18:2c:49
3882024-09-23T18:19:36.591Zproxy arp:
3892024-09-23T18:19:36.591Z10.151.2.164/10.151.2.227: a8:e1:de:01:70:1d
3902024-09-23T18:19:36.591Z+ retry=0
3912024-09-23T18:19:36.615Z+++ zoneadm list -n
3922024-09-23T18:19:36.615Z+++ grep oxz_ntp
3932024-09-23T18:19:36.615Z++ pfexec svcs -z -Hostate oxide/ntp
3942024-09-23T18:19:36.615Zsvcs: zone '-Hostate': entity not found
3952024-09-23T18:19:36.615Z++ true
3962024-09-23T18:19:36.615Z+ [[ '' != online ]]
3972024-09-23T18:19:36.615Z+ [[ 0 -gt 60 ]]
3982024-09-23T18:19:36.615Z+ sleep 1
3992024-09-23T18:19:37.611Z+ retry=1
4002024-09-23T18:19:37.634Z+++ zoneadm list -n
4012024-09-23T18:19:37.634Z+++ grep oxz_ntp
4022024-09-23T18:19:37.634Z++ pfexec svcs -z -Hostate oxide/ntp
4032024-09-23T18:19:37.635Zsvcs: zone '-Hostate': entity not found
4042024-09-23T18:19:37.635Z++ true
4052024-09-23T18:19:37.635Z+ [[ '' != online ]]
4062024-09-23T18:19:37.635Z+ [[ 1 -gt 60 ]]
4072024-09-23T18:19:37.635Z+ sleep 1
4082024-09-23T18:19:38.631Z+ retry=2
4092024-09-23T18:19:38.655Z+++ zoneadm list -n
4102024-09-23T18:19:38.655Z+++ grep oxz_ntp
4112024-09-23T18:19:38.655Z++ pfexec svcs -z -Hostate oxide/ntp
4122024-09-23T18:19:38.655Zsvcs: zone '-Hostate': entity not found
4132024-09-23T18:19:38.655Z++ true
4142024-09-23T18:19:38.655Z+ [[ '' != online ]]
4152024-09-23T18:19:38.655Z+ [[ 2 -gt 60 ]]
4162024-09-23T18:19:38.655Z+ sleep 1
4172024-09-23T18:19:39.654Z+ retry=3
4182024-09-23T18:19:39.677Z+++ zoneadm list -n
4192024-09-23T18:19:39.677Z+++ grep oxz_ntp
4202024-09-23T18:19:39.677Z++ pfexec svcs -z oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e -Hostate oxide/ntp
4212024-09-23T18:19:39.700Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
4222024-09-23T18:19:39.700Z++ true
4232024-09-23T18:19:39.700Z+ [[ '' != online ]]
4242024-09-23T18:19:39.700Z+ [[ 3 -gt 60 ]]
4252024-09-23T18:19:39.700Z+ sleep 1
4262024-09-23T18:19:40.686Z+ retry=4
4272024-09-23T18:19:40.709Z+++ zoneadm list -n
4282024-09-23T18:19:40.709Z+++ grep oxz_ntp
4292024-09-23T18:19:40.709Z++ pfexec svcs -z oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e -Hostate oxide/ntp
4302024-09-23T18:19:40.732Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
4312024-09-23T18:19:40.732Z++ true
4322024-09-23T18:19:40.732Z+ [[ '' != online ]]
4332024-09-23T18:19:40.732Z+ [[ 4 -gt 60 ]]
4342024-09-23T18:19:40.732Z+ sleep 1
4352024-09-23T18:19:41.714Z+ retry=5
4362024-09-23T18:19:41.738Z+++ zoneadm list -n
4372024-09-23T18:19:41.738Z+++ grep oxz_ntp
4382024-09-23T18:19:41.738Z++ pfexec svcs -z oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e -Hostate oxide/ntp
4392024-09-23T18:19:41.761Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
4402024-09-23T18:19:41.761Z++ true
4412024-09-23T18:19:41.762Z+ [[ '' != online ]]
4422024-09-23T18:19:41.762Z+ [[ 5 -gt 60 ]]
4432024-09-23T18:19:41.762Z+ sleep 1
4442024-09-23T18:19:42.744Z+ retry=6
4452024-09-23T18:19:42.767Z+++ zoneadm list -n
4462024-09-23T18:19:42.767Z+++ grep oxz_ntp
4472024-09-23T18:19:42.767Z++ pfexec svcs -z oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e -Hostate oxide/ntp
4482024-09-23T18:19:42.767Z+ [[ offline != online ]]
4492024-09-23T18:19:42.767Z+ [[ 6 -gt 60 ]]
4502024-09-23T18:19:42.767Z+ sleep 1
4512024-09-23T18:19:43.770Z+ retry=7
4522024-09-23T18:19:43.793Z+++ zoneadm list -n
4532024-09-23T18:19:43.793Z+++ grep oxz_ntp
4542024-09-23T18:19:43.793Z++ pfexec svcs -z oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e -Hostate oxide/ntp
4552024-09-23T18:19:43.793Z+ [[ offline != online ]]
4562024-09-23T18:19:43.793Z+ [[ 7 -gt 60 ]]
4572024-09-23T18:19:43.793Z+ sleep 1
4582024-09-23T18:19:44.789Z+ retry=8
4592024-09-23T18:19:44.812Z+++ zoneadm list -n
4602024-09-23T18:19:44.812Z+++ grep oxz_ntp
4612024-09-23T18:19:44.812Z++ pfexec svcs -z oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e -Hostate oxide/ntp
4622024-09-23T18:19:44.812Z+ [[ offline != online ]]
4632024-09-23T18:19:44.812Z+ [[ 8 -gt 60 ]]
4642024-09-23T18:19:44.812Z+ sleep 1
4652024-09-23T18:19:45.812Z+ retry=9
4662024-09-23T18:19:45.835Z+++ zoneadm list -n
4672024-09-23T18:19:45.835Z+++ grep oxz_ntp
4682024-09-23T18:19:45.835Z++ pfexec svcs -z oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e -Hostate oxide/ntp
4692024-09-23T18:19:45.835Z+ [[ offline != online ]]
4702024-09-23T18:19:45.835Z+ [[ 9 -gt 60 ]]
4712024-09-23T18:19:45.835Z+ sleep 1
4722024-09-23T18:19:46.834Z+ retry=10
4732024-09-23T18:19:46.858Z+++ zoneadm list -n
4742024-09-23T18:19:46.858Z+++ grep oxz_ntp
4752024-09-23T18:19:46.858Z++ pfexec svcs -z oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e -Hostate oxide/ntp
4762024-09-23T18:19:46.858Z+ [[ offline != online ]]
4772024-09-23T18:19:46.858Z+ [[ 10 -gt 60 ]]
4782024-09-23T18:19:46.858Z+ sleep 1
4792024-09-23T18:19:47.857Z+ retry=11
4802024-09-23T18:19:47.880Z+++ zoneadm list -n
4812024-09-23T18:19:47.880Z+++ grep oxz_ntp
4822024-09-23T18:19:47.880Z++ pfexec svcs -z oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e -Hostate oxide/ntp
4832024-09-23T18:19:47.880Z+ [[ offline != online ]]
4842024-09-23T18:19:47.880Z+ [[ 11 -gt 60 ]]
4852024-09-23T18:19:47.880Z+ sleep 1
4862024-09-23T18:19:48.879Z+ retry=12
4872024-09-23T18:19:48.902Z+++ zoneadm list -n
4882024-09-23T18:19:48.902Z+++ grep oxz_ntp
4892024-09-23T18:19:48.902Z++ pfexec svcs -z oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e -Hostate oxide/ntp
4902024-09-23T18:19:48.902Z+ [[ offline != online ]]
4912024-09-23T18:19:48.902Z+ [[ 12 -gt 60 ]]
4922024-09-23T18:19:48.902Z+ sleep 1
4932024-09-23T18:19:49.901Z+ retry=13
4942024-09-23T18:19:49.924Z+++ zoneadm list -n
4952024-09-23T18:19:49.924Z+++ grep oxz_ntp
4962024-09-23T18:19:49.924Z++ pfexec svcs -z oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e -Hostate oxide/ntp
4972024-09-23T18:19:49.926Z+ [[ offline != online ]]
4982024-09-23T18:19:49.926Z+ [[ 13 -gt 60 ]]
4992024-09-23T18:19:49.926Z+ sleep 1
 
5002024-09-23T18:19:49.983Zsock console login: Sep 23 18:19:49 sock ip: ip: joining multicasts failed (4) on opte0 - will use link layer broadcasts for multicast
 
5012024-09-23T18:19:50.923Z+ retry=14
5022024-09-23T18:19:50.946Z+++ zoneadm list -n
5032024-09-23T18:19:50.946Z+++ grep oxz_ntp
5042024-09-23T18:19:50.946Z++ pfexec svcs -z oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e -Hostate oxide/ntp
5052024-09-23T18:19:50.946Z+ [[ online != online ]]
5062024-09-23T18:19:50.946Z+ echo 'Waited for chrony: 14s'
5072024-09-23T18:19:50.946Z+ retry=0
5082024-09-23T18:19:50.946ZWaited for chrony: 14s
5092024-09-23T18:19:50.946Z+ zoneadm list
5102024-09-23T18:19:50.969Z+ grep nexus
5112024-09-23T18:19:50.969Z+ [[ 0 -gt 300 ]]
5122024-09-23T18:19:50.969Z+ sleep 1
5132024-09-23T18:19:51.953Z+ retry=1
5142024-09-23T18:19:51.976Z+ zoneadm list
5152024-09-23T18:19:51.976Z+ grep nexus
5162024-09-23T18:19:51.976Z+ [[ 1 -gt 300 ]]
5172024-09-23T18:19:51.976Z+ sleep 1
5182024-09-23T18:19:52.965Z+ retry=2
5192024-09-23T18:19:52.988Z+ zoneadm list
5202024-09-23T18:19:52.988Z+ grep nexus
5212024-09-23T18:19:52.988Z+ [[ 2 -gt 300 ]]
5222024-09-23T18:19:52.988Z+ sleep 1
5232024-09-23T18:19:53.981Z+ retry=3
5242024-09-23T18:19:54.004Z+ zoneadm list
5252024-09-23T18:19:54.004Z+ grep nexus
5262024-09-23T18:19:54.004Z+ [[ 3 -gt 300 ]]
5272024-09-23T18:19:54.004Z+ sleep 1
5282024-09-23T18:19:54.996Z+ retry=4
5292024-09-23T18:19:55.020Z+ zoneadm list
5302024-09-23T18:19:55.020Z+ grep nexus
5312024-09-23T18:19:55.020Z+ [[ 4 -gt 300 ]]
5322024-09-23T18:19:55.020Z+ sleep 1
5332024-09-23T18:19:56.013Z+ retry=5
5342024-09-23T18:19:56.037Z+ zoneadm list
5352024-09-23T18:19:56.037Z+ grep nexus
5362024-09-23T18:19:56.127Z+ [[ 5 -gt 300 ]]
5372024-09-23T18:19:56.128Z+ sleep 1
5382024-09-23T18:19:57.146Z+ retry=6
5392024-09-23T18:19:57.171Z+ zoneadm list
5402024-09-23T18:19:57.171Z+ grep nexus
5412024-09-23T18:19:57.195Z+ [[ 6 -gt 300 ]]
5422024-09-23T18:19:57.195Z+ sleep 1
5432024-09-23T18:19:58.217Z+ retry=7
5442024-09-23T18:19:58.240Z+ zoneadm list
5452024-09-23T18:19:58.240Z+ grep nexus
5462024-09-23T18:19:58.240Z+ [[ 7 -gt 300 ]]
5472024-09-23T18:19:58.240Z+ sleep 1
5482024-09-23T18:19:59.234Z+ retry=8
5492024-09-23T18:19:59.257Z+ zoneadm list
5502024-09-23T18:19:59.257Z+ grep nexus
5512024-09-23T18:19:59.257Z+ [[ 8 -gt 300 ]]
5522024-09-23T18:19:59.257Z+ sleep 1
5532024-09-23T18:20:00.249Z+ retry=9
5542024-09-23T18:20:00.273Z+ zoneadm list
5552024-09-23T18:20:00.273Z+ grep nexus
5562024-09-23T18:20:00.273Z+ [[ 9 -gt 300 ]]
5572024-09-23T18:20:00.273Z+ sleep 1
5582024-09-23T18:20:01.267Z+ retry=10
5592024-09-23T18:20:01.291Z+ zoneadm list
5602024-09-23T18:20:01.291Z+ grep nexus
5612024-09-23T18:20:01.291Z+ [[ 10 -gt 300 ]]
5622024-09-23T18:20:01.291Z+ sleep 1
5632024-09-23T18:20:02.283Z+ retry=11
5642024-09-23T18:20:02.306Z+ zoneadm list
5652024-09-23T18:20:02.306Z+ grep nexus
5662024-09-23T18:20:02.306Z+ [[ 11 -gt 300 ]]
5672024-09-23T18:20:02.306Z+ sleep 1
5682024-09-23T18:20:03.297Z+ retry=12
5692024-09-23T18:20:03.321Z+ zoneadm list
5702024-09-23T18:20:03.321Z+ grep nexus
5712024-09-23T18:20:03.321Z+ [[ 12 -gt 300 ]]
5722024-09-23T18:20:03.321Z+ sleep 1
5732024-09-23T18:20:04.312Z+ retry=13
5742024-09-23T18:20:04.335Z+ zoneadm list
5752024-09-23T18:20:04.335Z+ grep nexus
5762024-09-23T18:20:04.335Z+ [[ 13 -gt 300 ]]
5772024-09-23T18:20:04.335Z+ sleep 1
5782024-09-23T18:20:05.328Z+ retry=14
5792024-09-23T18:20:05.352Z+ zoneadm list
5802024-09-23T18:20:05.352Z+ grep nexus
5812024-09-23T18:20:05.352Z+ [[ 14 -gt 300 ]]
5822024-09-23T18:20:05.352Z+ sleep 1
5832024-09-23T18:20:06.343Z+ retry=15
5842024-09-23T18:20:06.366Z+ zoneadm list
5852024-09-23T18:20:06.366Z+ grep nexus
5862024-09-23T18:20:06.366Z+ [[ 15 -gt 300 ]]
5872024-09-23T18:20:06.366Z+ sleep 1
5882024-09-23T18:20:07.357Z+ retry=16
5892024-09-23T18:20:07.380Z+ zoneadm list
5902024-09-23T18:20:07.380Z+ grep nexus
5912024-09-23T18:20:07.381Z+ [[ 16 -gt 300 ]]
5922024-09-23T18:20:07.381Z+ sleep 1
5932024-09-23T18:20:08.372Z+ retry=17
5942024-09-23T18:20:08.396Z+ zoneadm list
5952024-09-23T18:20:08.396Z+ grep nexus
5962024-09-23T18:20:08.396Z+ [[ 17 -gt 300 ]]
5972024-09-23T18:20:08.398Z+ sleep 1
5982024-09-23T18:20:09.433Z+ retry=18
5992024-09-23T18:20:09.433Z+ zoneadm list
6002024-09-23T18:20:09.433Z+ grep nexus
6012024-09-23T18:20:09.433Z+ [[ 18 -gt 300 ]]
6022024-09-23T18:20:09.433Z+ sleep 1
6032024-09-23T18:20:10.409Z+ retry=19
6042024-09-23T18:20:10.433Z+ grep nexus
6052024-09-23T18:20:10.651Z+ zoneadm list
6062024-09-23T18:20:10.796Z+ [[ 19 -gt 300 ]]
6072024-09-23T18:20:10.796Z+ sleep 1
6082024-09-23T18:20:11.807Z+ retry=20
6092024-09-23T18:20:11.831Z+ zoneadm list
6102024-09-23T18:20:11.831Z+ grep nexus
6112024-09-23T18:20:11.831Z+ [[ 20 -gt 300 ]]
6122024-09-23T18:20:11.831Z+ sleep 1
6132024-09-23T18:20:12.824Z+ retry=21
6142024-09-23T18:20:12.848Z+ zoneadm list
6152024-09-23T18:20:12.848Z+ grep nexus
6162024-09-23T18:20:12.848Z+ [[ 21 -gt 300 ]]
6172024-09-23T18:20:12.848Z+ sleep 1
6182024-09-23T18:20:13.845Z+ retry=22
6192024-09-23T18:20:13.869Z+ zoneadm list
6202024-09-23T18:20:13.869Z+ grep nexus
6212024-09-23T18:20:13.869Z+ [[ 22 -gt 300 ]]
6222024-09-23T18:20:13.869Z+ sleep 1
6232024-09-23T18:20:14.860Z+ retry=23
6242024-09-23T18:20:14.885Z+ zoneadm list
6252024-09-23T18:20:14.885Z+ grep nexus
6262024-09-23T18:20:14.885Z+ [[ 23 -gt 300 ]]
6272024-09-23T18:20:14.885Z+ sleep 1
6282024-09-23T18:20:15.875Z+ retry=24
6292024-09-23T18:20:16.922Z+ zoneadm list
6302024-09-23T18:20:16.922Z+ grep nexus
6312024-09-23T18:20:16.922Z+ [[ 24 -gt 300 ]]
6322024-09-23T18:20:16.922Z+ sleep 1
6332024-09-23T18:20:16.945Z+ retry=25
6342024-09-23T18:20:16.945Z+ zoneadm list
6352024-09-23T18:20:16.945Z+ grep nexus
6362024-09-23T18:20:16.945Z+ [[ 25 -gt 300 ]]
6372024-09-23T18:20:16.946Z+ sleep 1
6382024-09-23T18:20:17.902Z+ retry=26
6392024-09-23T18:20:17.926Z+ zoneadm list
6402024-09-23T18:20:17.926Z+ grep nexus
6412024-09-23T18:20:17.926Z+ [[ 26 -gt 300 ]]
6422024-09-23T18:20:17.926Z+ sleep 1
6432024-09-23T18:20:18.918Z+ retry=27
6442024-09-23T18:20:18.941Z+ zoneadm list
6452024-09-23T18:20:18.941Z+ grep nexus
6462024-09-23T18:20:18.941Z+ [[ 27 -gt 300 ]]
6472024-09-23T18:20:18.941Z+ sleep 1
6482024-09-23T18:20:19.935Z+ retry=28
6492024-09-23T18:20:19.963Z+ zoneadm list
6502024-09-23T18:20:19.963Z+ grep nexus
6512024-09-23T18:20:19.963Z+ [[ 28 -gt 300 ]]
6522024-09-23T18:20:19.963Z+ sleep 1
6532024-09-23T18:20:20.948Z+ retry=29
6542024-09-23T18:20:20.971Z+ zoneadm list
6552024-09-23T18:20:20.971Z+ grep nexus
6562024-09-23T18:20:20.971Z+ [[ 29 -gt 300 ]]
6572024-09-23T18:20:20.971Z+ sleep 1
6582024-09-23T18:20:21.961Z+ retry=30
6592024-09-23T18:20:21.985Z+ zoneadm list
6602024-09-23T18:20:21.985Z+ grep nexus
6612024-09-23T18:20:21.985Z+ [[ 30 -gt 300 ]]
6622024-09-23T18:20:21.985Z+ sleep 1
6632024-09-23T18:20:22.977Z+ retry=31
6642024-09-23T18:20:23.001Z+ zoneadm list
6652024-09-23T18:20:23.001Z+ grep nexus
6662024-09-23T18:20:23.001Z+ [[ 31 -gt 300 ]]
6672024-09-23T18:20:23.001Z+ sleep 1
6682024-09-23T18:20:23.991Z+ retry=32
6692024-09-23T18:20:24.017Z+ zoneadm list
6702024-09-23T18:20:24.017Z+ grep nexus
6712024-09-23T18:20:24.017Z+ [[ 32 -gt 300 ]]
6722024-09-23T18:20:24.017Z+ sleep 1
6732024-09-23T18:20:25.005Z+ retry=33
6742024-09-23T18:20:25.028Z+ zoneadm list
6752024-09-23T18:20:25.029Z+ grep nexus
6762024-09-23T18:20:25.029Z+ [[ 33 -gt 300 ]]
6772024-09-23T18:20:25.029Z+ sleep 1
6782024-09-23T18:20:26.019Z+ retry=34
6792024-09-23T18:20:26.043Z+ zoneadm list
6802024-09-23T18:20:26.043Z+ grep nexus
6812024-09-23T18:20:26.043Z+ [[ 34 -gt 300 ]]
6822024-09-23T18:20:26.044Z+ sleep 1
6832024-09-23T18:20:27.032Z+ retry=35
6842024-09-23T18:20:27.056Z+ zoneadm list
6852024-09-23T18:20:27.056Z+ grep nexus
6862024-09-23T18:20:27.056Z+ [[ 35 -gt 300 ]]
6872024-09-23T18:20:27.056Z+ sleep 1
6882024-09-23T18:20:28.045Z+ retry=36
6892024-09-23T18:20:28.069Z+ zoneadm list
6902024-09-23T18:20:28.069Z+ grep nexus
6912024-09-23T18:20:28.069Z+ [[ 36 -gt 300 ]]
6922024-09-23T18:20:28.069Z+ sleep 1
6932024-09-23T18:20:29.057Z+ retry=37
6942024-09-23T18:20:29.080Z+ zoneadm list
6952024-09-23T18:20:29.080Z+ grep nexus
6962024-09-23T18:20:29.080Z+ [[ 37 -gt 300 ]]
6972024-09-23T18:20:29.080Z+ sleep 1
6982024-09-23T18:20:30.077Z+ retry=38
6992024-09-23T18:20:30.101Z+ zoneadm list
7002024-09-23T18:20:30.101Z+ grep nexus
7012024-09-23T18:20:30.101Z+ [[ 38 -gt 300 ]]
7022024-09-23T18:20:30.102Z+ sleep 1
7032024-09-23T18:20:31.091Z+ retry=39
7042024-09-23T18:20:31.132Z+ zoneadm list
7052024-09-23T18:20:31.132Z+ grep nexus
7062024-09-23T18:20:31.132Z+ [[ 39 -gt 300 ]]
7072024-09-23T18:20:31.132Z+ sleep 1
7082024-09-23T18:20:32.096Z+ retry=40
7092024-09-23T18:20:32.120Z+ zoneadm list
7102024-09-23T18:20:32.120Z+ grep nexus
7112024-09-23T18:20:32.120Z+ [[ 40 -gt 300 ]]
7122024-09-23T18:20:32.120Z+ sleep 1
7132024-09-23T18:20:33.108Z+ retry=41
7142024-09-23T18:20:33.132Z+ zoneadm list
7152024-09-23T18:20:33.132Z+ grep nexus
7162024-09-23T18:20:33.132Z+ [[ 41 -gt 300 ]]
7172024-09-23T18:20:33.132Z+ sleep 1
7182024-09-23T18:20:34.120Z+ retry=42
7192024-09-23T18:20:34.144Z+ zoneadm list
7202024-09-23T18:20:34.144Z+ grep nexus
7212024-09-23T18:20:34.144Z+ [[ 42 -gt 300 ]]
7222024-09-23T18:20:34.144Z+ sleep 1
7232024-09-23T18:20:35.131Z+ retry=43
7242024-09-23T18:20:35.155Z+ zoneadm list
7252024-09-23T18:20:35.155Z+ grep nexus
7262024-09-23T18:20:35.155Z+ [[ 43 -gt 300 ]]
7272024-09-23T18:20:35.155Z+ sleep 1
7282024-09-23T18:20:36.143Z+ retry=44
7292024-09-23T18:20:36.167Z+ zoneadm list
7302024-09-23T18:20:36.167Z+ grep nexus
7312024-09-23T18:20:36.167Z+ [[ 44 -gt 300 ]]
7322024-09-23T18:20:36.167Z+ sleep 1
7332024-09-23T18:20:37.155Z+ retry=45
7342024-09-23T18:20:37.179Z+ zoneadm list
7352024-09-23T18:20:37.179Z+ grep nexus
7362024-09-23T18:20:37.179Z+ [[ 45 -gt 300 ]]
7372024-09-23T18:20:37.179Z+ sleep 1
7382024-09-23T18:20:38.168Z+ retry=46
7392024-09-23T18:20:38.191Z+ zoneadm list
7402024-09-23T18:20:38.191Z+ grep nexus
7412024-09-23T18:20:38.191Z+ [[ 46 -gt 300 ]]
7422024-09-23T18:20:38.191Z+ sleep 1
7432024-09-23T18:20:39.181Z+ retry=47
7442024-09-23T18:20:39.205Z+ zoneadm list
7452024-09-23T18:20:39.205Z+ grep nexus
7462024-09-23T18:20:39.205Z+ [[ 47 -gt 300 ]]
7472024-09-23T18:20:39.205Z+ sleep 1
7482024-09-23T18:20:40.209Z+ retry=48
7492024-09-23T18:20:40.263Z+ zoneadm list
7502024-09-23T18:20:40.263Z+ grep nexus
7512024-09-23T18:20:40.263Z+ [[ 48 -gt 300 ]]
7522024-09-23T18:20:40.263Z+ sleep 1
7532024-09-23T18:20:41.206Z+ retry=49
7542024-09-23T18:20:41.230Z+ zoneadm list
7552024-09-23T18:20:41.230Z+ grep nexus
7562024-09-23T18:20:41.230Z+ [[ 49 -gt 300 ]]
7572024-09-23T18:20:41.230Z+ sleep 1
7582024-09-23T18:20:42.220Z+ retry=50
7592024-09-23T18:20:42.244Z+ zoneadm list
7602024-09-23T18:20:42.244Z+ grep nexus
7612024-09-23T18:20:42.244Z+ [[ 50 -gt 300 ]]
7622024-09-23T18:20:42.244Z+ sleep 1
7632024-09-23T18:20:43.234Z+ retry=51
7642024-09-23T18:20:43.257Z+ zoneadm list
7652024-09-23T18:20:43.257Z+ grep nexus
7662024-09-23T18:20:43.257Z+ [[ 51 -gt 300 ]]
7672024-09-23T18:20:43.257Z+ sleep 1
7682024-09-23T18:20:44.246Z+ retry=52
7692024-09-23T18:20:44.269Z+ zoneadm list
7702024-09-23T18:20:44.269Z+ grep nexus
7712024-09-23T18:20:44.269Z+ [[ 52 -gt 300 ]]
7722024-09-23T18:20:44.269Z+ sleep 1
7732024-09-23T18:20:45.268Z+ retry=53
7742024-09-23T18:20:45.293Z+ zoneadm list
7752024-09-23T18:20:45.293Z+ grep nexus
7762024-09-23T18:20:45.293Z+ [[ 53 -gt 300 ]]
7772024-09-23T18:20:45.293Z+ sleep 1
7782024-09-23T18:20:46.291Z+ retry=54
7792024-09-23T18:20:46.315Z+ zoneadm list
7802024-09-23T18:20:46.315Z+ grep nexus
7812024-09-23T18:20:46.315Z+ [[ 54 -gt 300 ]]
7822024-09-23T18:20:46.315Z+ sleep 1
7832024-09-23T18:20:47.315Z+ retry=55
7842024-09-23T18:20:47.339Z+ grep nexus
7852024-09-23T18:20:47.378Z+ zoneadm list
7862024-09-23T18:20:47.461Z+ [[ 55 -gt 300 ]]
7872024-09-23T18:20:47.461Z+ sleep 1
7882024-09-23T18:20:48.464Z+ retry=56
7892024-09-23T18:20:48.488Z+ zoneadm list
7902024-09-23T18:20:48.488Z+ grep nexus
7912024-09-23T18:20:48.488Z+ [[ 56 -gt 300 ]]
7922024-09-23T18:20:48.488Z+ sleep 1
7932024-09-23T18:20:49.494Z+ retry=57
7942024-09-23T18:20:49.517Z+ zoneadm list
7952024-09-23T18:20:49.517Z+ grep nexus
7962024-09-23T18:20:49.517Z+ [[ 57 -gt 300 ]]
7972024-09-23T18:20:49.517Z+ sleep 1
7982024-09-23T18:20:50.517Z+ retry=58
7992024-09-23T18:20:50.540Z+ zoneadm list
8002024-09-23T18:20:50.540Z+ grep nexus
8012024-09-23T18:20:50.540Z+ [[ 58 -gt 300 ]]
8022024-09-23T18:20:50.541Z+ sleep 1
8032024-09-23T18:20:51.542Z+ retry=59
8042024-09-23T18:20:51.565Z+ zoneadm list
8052024-09-23T18:20:51.565Z+ grep nexus
8062024-09-23T18:20:51.565Zoxz_nexus_efc72cb3-831b-4b4a-acac-bb2cd9209275
8072024-09-23T18:20:51.565Zoxz_nexus_b9740b46-002a-4447-8a9d-58caea4925b7
8082024-09-23T18:20:51.565Z+ echo 'Waited for nexus: 59s'
8092024-09-23T18:20:51.565ZWaited for nexus: 59s
8102024-09-23T18:20:51.566Z+ export RUST_BACKTRACE=1
8112024-09-23T18:20:51.566Z+ RUST_BACKTRACE=1
8122024-09-23T18:20:51.566Z+ export E2E_TLS_CERT IPPOOL_START IPPOOL_END
8132024-09-23T18:20:51.566Z++ ./target/debug/bootstrap
8142024-09-23T18:20:51.632ZUsing external DNS server at 10.151.2.164:53
8152024-09-23T18:20:51.656Z2024-09-23 18:20:51.641808127 UTC: attempting to log into API
8162024-09-23T18:21:06.697Z2024-09-23 18:21:06.687251333 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8172024-09-23T18:21:07.698Z2024-09-23 18:21:07.688332880 UTC: attempting to log into API
 
8182024-09-23T18:21:14.146ZSep 23 18:21:13 sock ip: ip: joining multicasts failed (4) on opte4 - will use link layer broadcasts for multicast
8192024-09-23T18:21:15.042ZSep 23 18:21:14 sock ip: ip: joining multicasts failed (4) on opte3 - will use link layer broadcasts for multicast
8202024-09-23T18:21:15.076ZSep 23 18:21:14 sock ip: ip: joining multicasts failed (4) on opte1 - will use link layer broadcasts for multicast
8212024-09-23T18:21:16.761ZSep 23 18:21:16 sock ip: ip: joining multicasts failed (4) on opte5 - will use link layer broadcasts for multicast
8222024-09-23T18:21:16.946ZSep 23 18:21:16 sock ip: ip: joining multicasts failed (4) on opte2 - will use link layer broadcasts for multicast
 
8232024-09-23T18:21:22.734Z2024-09-23 18:21:22.721333333 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8242024-09-23T18:21:23.733Z2024-09-23 18:21:23.722051027 UTC: attempting to log into API
 
8252024-09-23T18:21:27.988ZSep 23 18:21:27 sock xde: WARNING: failed to generate descriptor for stateful action: ICMP:172.30.3.5:0:216.31.16.12:0 Unexpected { msg: "Expected ICMP Echo Request, found: destination unreachable" }
 
8262024-09-23T18:21:38.766Z2024-09-23 18:21:38.754099670 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8272024-09-23T18:21:39.766Z2024-09-23 18:21:39.755221727 UTC: attempting to log into API
 
8282024-09-23T18:21:46.251ZSep 23 18:21:29 sock last message repeated 1 time
8292024-09-23T18:21:46.282ZSep 23 18:21:45 sock xde: WARNING: failed to generate descriptor for stateful action: ICMP:172.30.3.5:0:104.167.215.195:0 Unexpected { msg: "Expected ICMP Echo Request, found: destination unreachable" }
 
8302024-09-23T18:21:54.798Z2024-09-23 18:21:54.787083523 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8312024-09-23T18:21:55.800Z2024-09-23 18:21:55.788061116 UTC: attempting to log into API
8322024-09-23T18:22:10.831Z2024-09-23 18:22:10.819863130 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8332024-09-23T18:22:11.832Z2024-09-23 18:22:11.821021418 UTC: attempting to log into API
8342024-09-23T18:22:12.683Z2024-09-23 18:22:12.671840710 UTC: login succeeded
8352024-09-23T18:22:12.716Zwaiting for nexus to come up...
8362024-09-23T18:22:12.799Zcreating IP pool... 10.151.2.175 - 10.151.2.227
8372024-09-23T18:22:13.020Zsetting up quotas...
8382024-09-23T18:22:13.074Zensuring datasets are ready...
8392024-09-23T18:22:15.955Zlet's roll.
8402024-09-23T18:22:15.979Z+ eval 'OXIDE_HOST=https://recovery.sys.oxide.test
8412024-09-23T18:22:15.979ZOXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.2.166
8422024-09-23T18:22:15.979ZOXIDE_TOKEN=oxide-token-a776e313ccc201b21e1521a034c6df008f919a4a'
8432024-09-23T18:22:15.979Z++ OXIDE_HOST=https://recovery.sys.oxide.test
8442024-09-23T18:22:15.979Z++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.2.166
8452024-09-23T18:22:15.979Z++ OXIDE_TOKEN=oxide-token-a776e313ccc201b21e1521a034c6df008f919a4a
8462024-09-23T18:22:15.979Z+ export OXIDE_HOST OXIDE_TOKEN
8472024-09-23T18:22:15.979Z+ retry=0
8482024-09-23T18:22:15.979Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8492024-09-23T18:22:15.979Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8502024-09-23T18:22:15.979Z+ [[ 0 -gt 60 ]]
8512024-09-23T18:22:15.979Z+ sleep 1
8522024-09-23T18:22:16.982Z+ retry=1
8532024-09-23T18:22:16.982Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8542024-09-23T18:22:17.006Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8552024-09-23T18:22:17.007Z+ [[ 1 -gt 60 ]]
8562024-09-23T18:22:17.007Z+ sleep 1
8572024-09-23T18:22:17.999Z+ retry=2
8582024-09-23T18:22:17.999Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8592024-09-23T18:22:18.023Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8602024-09-23T18:22:18.023Z+ [[ 2 -gt 60 ]]
8612024-09-23T18:22:18.023Z+ sleep 1
8622024-09-23T18:22:19.014Z+ retry=3
8632024-09-23T18:22:19.014Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8642024-09-23T18:22:19.038Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8652024-09-23T18:22:19.038Z+ [[ 3 -gt 60 ]]
8662024-09-23T18:22:19.038Z+ sleep 1
8672024-09-23T18:22:20.030Z+ retry=4
8682024-09-23T18:22:20.030Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8692024-09-23T18:22:20.054Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8702024-09-23T18:22:20.054Z+ [[ 4 -gt 60 ]]
8712024-09-23T18:22:20.054Z+ sleep 1
8722024-09-23T18:22:21.046Z+ retry=5
8732024-09-23T18:22:21.046Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8742024-09-23T18:22:21.070Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8752024-09-23T18:22:21.070Z+ [[ 5 -gt 60 ]]
8762024-09-23T18:22:21.070Z+ sleep 1
8772024-09-23T18:22:22.061Z+ retry=6
8782024-09-23T18:22:22.061Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8792024-09-23T18:22:22.086Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8802024-09-23T18:22:22.086Z+ [[ 6 -gt 60 ]]
8812024-09-23T18:22:22.086Z+ sleep 1
8822024-09-23T18:22:23.076Z+ retry=7
8832024-09-23T18:22:23.076Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8842024-09-23T18:22:23.099Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8852024-09-23T18:22:23.099Z+ [[ 7 -gt 60 ]]
8862024-09-23T18:22:23.099Z+ sleep 1
8872024-09-23T18:22:24.090Z+ retry=8
8882024-09-23T18:22:24.091Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8892024-09-23T18:22:24.115Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8902024-09-23T18:22:24.115Z+ [[ 8 -gt 60 ]]
8912024-09-23T18:22:24.115Z+ sleep 1
8922024-09-23T18:22:25.105Z+ retry=9
8932024-09-23T18:22:25.105Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8942024-09-23T18:22:25.129Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
8952024-09-23T18:22:25.129Z+ [[ 9 -gt 60 ]]
8962024-09-23T18:22:25.129Z+ sleep 1
8972024-09-23T18:22:26.121Z+ retry=10
8982024-09-23T18:22:26.121Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
8992024-09-23T18:22:26.145Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9002024-09-23T18:22:26.145Z+ [[ 10 -gt 60 ]]
9012024-09-23T18:22:26.145Z+ sleep 1
9022024-09-23T18:22:27.138Z+ retry=11
9032024-09-23T18:22:27.161Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9042024-09-23T18:22:27.161Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9052024-09-23T18:22:27.161Z+ [[ 11 -gt 60 ]]
9062024-09-23T18:22:27.162Z+ sleep 1
9072024-09-23T18:22:28.155Z+ retry=12
9082024-09-23T18:22:28.155Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9092024-09-23T18:22:28.178Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9102024-09-23T18:22:28.178Z+ [[ 12 -gt 60 ]]
9112024-09-23T18:22:28.178Z+ sleep 1
9122024-09-23T18:22:29.170Z+ retry=13
9132024-09-23T18:22:29.170Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9142024-09-23T18:22:29.193Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9152024-09-23T18:22:29.193Z+ [[ 13 -gt 60 ]]
9162024-09-23T18:22:29.193Z+ sleep 1
9172024-09-23T18:22:30.187Z+ retry=14
9182024-09-23T18:22:30.187Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9192024-09-23T18:22:30.210Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9202024-09-23T18:22:30.210Z+ [[ 14 -gt 60 ]]
9212024-09-23T18:22:30.210Z+ sleep 1
9222024-09-23T18:22:31.202Z+ retry=15
9232024-09-23T18:22:31.202Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9242024-09-23T18:22:31.226Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9252024-09-23T18:22:31.226Z+ [[ 15 -gt 60 ]]
9262024-09-23T18:22:31.226Z+ sleep 1
9272024-09-23T18:22:32.219Z+ retry=16
9282024-09-23T18:22:32.219Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9292024-09-23T18:22:32.242Zcurl: (7) Failed to connect to recovery.sys.oxide.test port 443 after 0 ms: Could not connect to server
9302024-09-23T18:22:32.242Z+ [[ 16 -gt 60 ]]
9312024-09-23T18:22:32.242Z+ sleep 1
9322024-09-23T18:22:33.237Z+ retry=17
9332024-09-23T18:22:33.237Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9342024-09-23T18:22:33.318Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images'
9352024-09-23T18:22:34.081Z{"status":"ok"}success
9362024-09-23T18:22:34.081ZProject {
9372024-09-23T18:22:34.081Z description: "some images",
9382024-09-23T18:22:34.081Z id: cd6bbd50-8316-4a79-9a61-13b986d52979,
9392024-09-23T18:22:34.081Z name: Name(
9402024-09-23T18:22:34.081Z "images",
9412024-09-23T18:22:34.081Z ),
9422024-09-23T18:22:34.081Z time_created: 2024-09-23T18:22:33.428829Z,
9432024-09-23T18:22:34.081Z time_modified: 2024-09-23T18:22:33.428829Z,
9442024-09-23T18:22:34.081Z}
9452024-09-23T18:22:34.105Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11
 
9462024-09-23T18:25:11.394ZSep 23 18:25:10 sock xde: WARNING: failed to generate descriptor for stateful action: ICMP:172.30.3.5:0:142.202.190.19:0 Unexpected { msg: "Expected ICMP Echo Request, found: destination unreachable" }
 
9472024-09-23T18:27:53.293Zdone!
9482024-09-23T18:27:53.397Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11
9492024-09-23T18:27:53.506Zsuccess
9502024-09-23T18:27:53.506ZImage {
9512024-09-23T18:27:53.506Z block_size: ByteCount(
9522024-09-23T18:27:53.506Z 512,
9532024-09-23T18:27:53.506Z ),
9542024-09-23T18:27:53.506Z description: "debian 11 original base image",
9552024-09-23T18:27:53.506Z digest: None,
9562024-09-23T18:27:53.506Z id: 5ff2f4a7-327a-466e-a1cd-168e1cff563e,
9572024-09-23T18:27:53.506Z name: Name(
9582024-09-23T18:27:53.506Z "debian11",
9592024-09-23T18:27:53.530Z ),
9602024-09-23T18:27:53.530Z os: "debian",
9612024-09-23T18:27:53.530Z project_id: None,
9622024-09-23T18:27:53.530Z size: ByteCount(
9632024-09-23T18:27:53.530Z 2147483648,
9642024-09-23T18:27:53.531Z ),
9652024-09-23T18:27:53.531Z time_created: 2024-09-23T18:27:53.227803Z,
9662024-09-23T18:27:53.531Z time_modified: 2024-09-23T18:27:53.488431Z,
9672024-09-23T18:27:53.531Z url: None,
9682024-09-23T18:27:53.531Z version: "11",
9692024-09-23T18:27:53.531Z}
9702024-09-23T18:27:53.531Z+ for test_bin in tests/*
9712024-09-23T18:27:53.531Z+ ./tests/bootstrap-4bd4b6817bc5b95b
9722024-09-23T18:27:53.531Z
9732024-09-23T18:27:53.531Zrunning 0 tests
9742024-09-23T18:27:53.531Z
9752024-09-23T18:27:53.531Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
9762024-09-23T18:27:53.531Z
9772024-09-23T18:27:53.531Z+ for test_bin in tests/*
9782024-09-23T18:27:53.531Z+ ./tests/commtest-5ebf0895cc3e3c74
9792024-09-23T18:27:53.555Z
9802024-09-23T18:27:53.555Zrunning 0 tests
9812024-09-23T18:27:53.555Z
9822024-09-23T18:27:53.555Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
9832024-09-23T18:27:53.555Z
9842024-09-23T18:27:53.555Z+ for test_bin in tests/*
9852024-09-23T18:27:53.555Z+ ./tests/dhcp_server-9a6f8575738de90f
9862024-09-23T18:27:53.555Z
9872024-09-23T18:27:53.555Zrunning 0 tests
9882024-09-23T18:27:53.555Z
9892024-09-23T18:27:53.555Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
9902024-09-23T18:27:53.555Z
9912024-09-23T18:27:53.555Z+ for test_bin in tests/*
9922024-09-23T18:27:53.555Z+ ./tests/end_to_end_tests-cdbb8d2022e3297e
9932024-09-23T18:27:53.595Z
9942024-09-23T18:27:53.596Zrunning 2 tests
9952024-09-23T18:27:53.846Ztest no_spoof::no_spoof_here ... ok
9962024-09-23T18:28:20.953Ztest instance_launch::instance_launch ... ok
9972024-09-23T18:28:20.953Z
9982024-09-23T18:28:20.953Ztest result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 27.36s
9992024-09-23T18:28:20.953Z
10002024-09-23T18:28:20.979Z+ _exit_trap
10012024-09-23T18:28:20.979Z+ local status=0
10022024-09-23T18:28:20.979Z+ set +o errexit
10032024-09-23T18:28:20.979Z+ pfexec svcadm -Z disable -s cron
10042024-09-23T18:28:22.356Z+ [[ 0 -eq 0 ]]
10052024-09-23T18:28:22.356Z+ exit 0
10062024-09-23T18:28:22.381Zprocess exited: duration 596069 ms, exit code 0
 
10072024-09-23T18:28:22.486Zfound 108 output files
10082024-09-23T18:28:22.486Zuploading: /var/svc/log/oxide-sled-agent:default.log (841530 bytes)
10092024-09-23T18:28:23.770Zuploaded: /var/svc/log/oxide-sled-agent:default.log
10102024-09-23T18:28:23.770Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log (4707923 bytes)
10112024-09-23T18:28:24.188Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log" changed size mid upload: 4707923 -> 4721422
10122024-09-23T18:28:25.258Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log
10132024-09-23T18:28:25.258Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log (37178 bytes)
10142024-09-23T18:28:26.420Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-lldpd:default.log
10152024-09-23T18:28:26.420Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log (183249 bytes)
10162024-09-23T18:28:26.602Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log" changed size mid upload: 183249 -> 184813
10172024-09-23T18:28:27.672Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log
10182024-09-23T18:28:27.672Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log (233980 bytes)
10192024-09-23T18:28:27.899Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log" changed size mid upload: 233980 -> 236326
10202024-09-23T18:28:28.971Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log
10212024-09-23T18:28:28.971Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log (1427523 bytes)
10222024-09-23T18:28:29.277Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log" changed size mid upload: 1427523 -> 1441887
10232024-09-23T18:28:30.347Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log
10242024-09-23T18:28:30.347Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log (3464522 bytes)
10252024-09-23T18:28:30.741Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log" changed size mid upload: 3464522 -> 3518616
10262024-09-23T18:28:31.814Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log
10272024-09-23T18:28:31.814Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log (3333 bytes)
10282024-09-23T18:28:32.907Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log
10292024-09-23T18:28:32.907Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log (260995 bytes)
10302024-09-23T18:28:33.111Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log" changed size mid upload: 260995 -> 266170
10312024-09-23T18:28:34.183Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log
10322024-09-23T18:28:34.183Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log (7855 bytes)
10332024-09-23T18:28:34.255Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log" changed size mid upload: 7855 -> 8440
10342024-09-23T18:28:35.320Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log
10352024-09-23T18:28:35.320Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log (3977 bytes)
10362024-09-23T18:28:36.414Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log
10372024-09-23T18:28:36.414Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-zone-network-setup:default.log (8131 bytes)
10382024-09-23T18:28:37.530Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-zone-network-setup:default.log
10392024-09-23T18:28:37.530Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_clickhouse_f141ec5c-de3f-4dea-aa70-9c7e0ad6beee/root/var/svc/log/oxide-clickhouse-admin:default.log (871 bytes)
10402024-09-23T18:28:38.625Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_clickhouse_f141ec5c-de3f-4dea-aa70-9c7e0ad6beee/root/var/svc/log/oxide-clickhouse-admin:default.log
10412024-09-23T18:28:38.626Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_clickhouse_f141ec5c-de3f-4dea-aa70-9c7e0ad6beee/root/var/svc/log/oxide-clickhouse:default.log (3421409 bytes)
10422024-09-23T18:28:38.999Zupload warning: file "/pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_clickhouse_f141ec5c-de3f-4dea-aa70-9c7e0ad6beee/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 3421409 -> 3574867
10432024-09-23T18:28:40.068Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_clickhouse_f141ec5c-de3f-4dea-aa70-9c7e0ad6beee/root/var/svc/log/oxide-clickhouse:default.log
10442024-09-23T18:28:40.068Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_clickhouse_f141ec5c-de3f-4dea-aa70-9c7e0ad6beee/root/var/svc/log/oxide-zone-network-setup:default.log (1685 bytes)
10452024-09-23T18:28:41.162Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_clickhouse_f141ec5c-de3f-4dea-aa70-9c7e0ad6beee/root/var/svc/log/oxide-zone-network-setup:default.log
10462024-09-23T18:28:41.162Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_cockroachdb_3f116413-4e92-4582-b3a5-e25e835c42da/root/var/svc/log/oxide-cockroach-admin:default.log (3727 bytes)
10472024-09-23T18:28:42.259Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_cockroachdb_3f116413-4e92-4582-b3a5-e25e835c42da/root/var/svc/log/oxide-cockroach-admin:default.log
10482024-09-23T18:28:42.259Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_cockroachdb_3f116413-4e92-4582-b3a5-e25e835c42da/root/var/svc/log/oxide-cockroachdb:default.log (4314 bytes)
10492024-09-23T18:28:43.353Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_cockroachdb_3f116413-4e92-4582-b3a5-e25e835c42da/root/var/svc/log/oxide-cockroachdb:default.log
10502024-09-23T18:28:43.353Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_cockroachdb_3f116413-4e92-4582-b3a5-e25e835c42da/root/var/svc/log/oxide-zone-network-setup:default.log (1681 bytes)
10512024-09-23T18:28:44.448Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_cockroachdb_3f116413-4e92-4582-b3a5-e25e835c42da/root/var/svc/log/oxide-zone-network-setup:default.log
10522024-09-23T18:28:44.448Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_crucible_46c88b0c-8be0-41ef-83a1-5caeaba47dc8/root/var/svc/log/oxide-crucible-agent:default.log (23900 bytes)
10532024-09-23T18:28:45.587Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_crucible_46c88b0c-8be0-41ef-83a1-5caeaba47dc8/root/var/svc/log/oxide-crucible-agent:default.log
10542024-09-23T18:28:45.587Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_crucible_46c88b0c-8be0-41ef-83a1-5caeaba47dc8/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10552024-09-23T18:28:46.683Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_crucible_46c88b0c-8be0-41ef-83a1-5caeaba47dc8/root/var/svc/log/oxide-crucible-downstairs:default.log
10562024-09-23T18:28:46.683Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_crucible_46c88b0c-8be0-41ef-83a1-5caeaba47dc8/root/var/svc/log/oxide-crucible-downstairs:downstairs-331eaaf3-0df3-4137-81d2-e459a6448132.log (3020 bytes)
10572024-09-23T18:28:47.777Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_crucible_46c88b0c-8be0-41ef-83a1-5caeaba47dc8/root/var/svc/log/oxide-crucible-downstairs:downstairs-331eaaf3-0df3-4137-81d2-e459a6448132.log
10582024-09-23T18:28:47.777Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_crucible_46c88b0c-8be0-41ef-83a1-5caeaba47dc8/root/var/svc/log/oxide-crucible-downstairs:downstairs-c8c865ea-2510-46cb-b625-006aee166a82.log (5996 bytes)
10592024-09-23T18:28:48.871Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_crucible_46c88b0c-8be0-41ef-83a1-5caeaba47dc8/root/var/svc/log/oxide-crucible-downstairs:downstairs-c8c865ea-2510-46cb-b625-006aee166a82.log
10602024-09-23T18:28:48.871Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_crucible_46c88b0c-8be0-41ef-83a1-5caeaba47dc8/root/var/svc/log/oxide-zone-network-setup:default.log (1674 bytes)
10612024-09-23T18:28:49.964Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_crucible_46c88b0c-8be0-41ef-83a1-5caeaba47dc8/root/var/svc/log/oxide-zone-network-setup:default.log
10622024-09-23T18:28:49.964Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_external_dns_544f6393-b5b8-4964-b9b8-d8467761342e/root/var/svc/log/oxide-external_dns:default.log (26903 bytes)
10632024-09-23T18:28:50.077Zupload warning: file "/pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_external_dns_544f6393-b5b8-4964-b9b8-d8467761342e/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 26903 -> 30678
10642024-09-23T18:28:51.148Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_external_dns_544f6393-b5b8-4964-b9b8-d8467761342e/root/var/svc/log/oxide-external_dns:default.log
10652024-09-23T18:28:51.148Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_external_dns_544f6393-b5b8-4964-b9b8-d8467761342e/root/var/svc/log/oxide-opte-interface-setup:default.log (1156 bytes)
10662024-09-23T18:28:52.243Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_external_dns_544f6393-b5b8-4964-b9b8-d8467761342e/root/var/svc/log/oxide-opte-interface-setup:default.log
10672024-09-23T18:28:52.243Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_external_dns_544f6393-b5b8-4964-b9b8-d8467761342e/root/var/svc/log/oxide-zone-network-setup:default.log (1697 bytes)
10682024-09-23T18:28:53.337Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_external_dns_544f6393-b5b8-4964-b9b8-d8467761342e/root/var/svc/log/oxide-zone-network-setup:default.log
10692024-09-23T18:28:53.337Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_internal_dns_8a315cf1-5df7-4f3d-9fde-62f2891368e1/root/var/svc/log/oxide-internal_dns:default.log (406854 bytes)
10702024-09-23T18:28:53.566Zupload warning: file "/pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_internal_dns_8a315cf1-5df7-4f3d-9fde-62f2891368e1/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 406854 -> 410611
10712024-09-23T18:28:54.637Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_internal_dns_8a315cf1-5df7-4f3d-9fde-62f2891368e1/root/var/svc/log/oxide-internal_dns:default.log
10722024-09-23T18:28:54.637Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_internal_dns_8a315cf1-5df7-4f3d-9fde-62f2891368e1/root/var/svc/log/oxide-zone-network-setup:default.log (1680 bytes)
10732024-09-23T18:28:55.731Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_internal_dns_8a315cf1-5df7-4f3d-9fde-62f2891368e1/root/var/svc/log/oxide-zone-network-setup:default.log
10742024-09-23T18:28:55.731Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/svc/log/oxide-chrony-setup:default.log (14495 bytes)
10752024-09-23T18:28:56.850Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/svc/log/oxide-chrony-setup:default.log
10762024-09-23T18:28:56.850Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/svc/log/oxide-ntp:default.log (3719 bytes)
10772024-09-23T18:28:56.896Zupload warning: file "/pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/svc/log/oxide-ntp:default.log" changed size mid upload: 3719 -> 3835
10782024-09-23T18:28:57.966Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/svc/log/oxide-ntp:default.log
10792024-09-23T18:28:57.966Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/svc/log/oxide-opte-interface-setup:default.log (1126 bytes)
10802024-09-23T18:28:59.064Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/svc/log/oxide-opte-interface-setup:default.log
10812024-09-23T18:28:59.064Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/svc/log/oxide-zone-network-setup:default.log (1636 bytes)
10822024-09-23T18:29:00.156Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/svc/log/oxide-zone-network-setup:default.log
10832024-09-23T18:29:00.156Zuploading: /pool/ext/17d503a6-b44e-42a3-ae74-5b19be755ad5/crypt/zone/oxz_crucible_517b3055-a59f-450e-a349-fc1fc2615512/root/var/svc/log/oxide-crucible-agent:default.log (20176 bytes)
10842024-09-23T18:29:01.296Zuploaded: /pool/ext/17d503a6-b44e-42a3-ae74-5b19be755ad5/crypt/zone/oxz_crucible_517b3055-a59f-450e-a349-fc1fc2615512/root/var/svc/log/oxide-crucible-agent:default.log
10852024-09-23T18:29:01.296Zuploading: /pool/ext/17d503a6-b44e-42a3-ae74-5b19be755ad5/crypt/zone/oxz_crucible_517b3055-a59f-450e-a349-fc1fc2615512/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10862024-09-23T18:29:02.389Zuploaded: /pool/ext/17d503a6-b44e-42a3-ae74-5b19be755ad5/crypt/zone/oxz_crucible_517b3055-a59f-450e-a349-fc1fc2615512/root/var/svc/log/oxide-crucible-downstairs:default.log
10872024-09-23T18:29:02.389Zuploading: /pool/ext/17d503a6-b44e-42a3-ae74-5b19be755ad5/crypt/zone/oxz_crucible_517b3055-a59f-450e-a349-fc1fc2615512/root/var/svc/log/oxide-crucible-downstairs:downstairs-6e3a6323-3ebc-46d5-a9fe-0fd316930238.log (6078 bytes)
10882024-09-23T18:29:03.484Zuploaded: /pool/ext/17d503a6-b44e-42a3-ae74-5b19be755ad5/crypt/zone/oxz_crucible_517b3055-a59f-450e-a349-fc1fc2615512/root/var/svc/log/oxide-crucible-downstairs:downstairs-6e3a6323-3ebc-46d5-a9fe-0fd316930238.log
10892024-09-23T18:29:03.484Zuploading: /pool/ext/17d503a6-b44e-42a3-ae74-5b19be755ad5/crypt/zone/oxz_crucible_517b3055-a59f-450e-a349-fc1fc2615512/root/var/svc/log/oxide-crucible-downstairs:snapshot-6e3a6323-3ebc-46d5-a9fe-0fd316930238-bf6b8e04-a5bf-448b-a96c-5d70aa90f8a7.log (5688 bytes)
10902024-09-23T18:29:04.577Zuploaded: /pool/ext/17d503a6-b44e-42a3-ae74-5b19be755ad5/crypt/zone/oxz_crucible_517b3055-a59f-450e-a349-fc1fc2615512/root/var/svc/log/oxide-crucible-downstairs:snapshot-6e3a6323-3ebc-46d5-a9fe-0fd316930238-bf6b8e04-a5bf-448b-a96c-5d70aa90f8a7.log
10912024-09-23T18:29:04.577Zuploading: /pool/ext/17d503a6-b44e-42a3-ae74-5b19be755ad5/crypt/zone/oxz_crucible_517b3055-a59f-450e-a349-fc1fc2615512/root/var/svc/log/oxide-zone-network-setup:default.log (1675 bytes)
10922024-09-23T18:29:05.671Zuploaded: /pool/ext/17d503a6-b44e-42a3-ae74-5b19be755ad5/crypt/zone/oxz_crucible_517b3055-a59f-450e-a349-fc1fc2615512/root/var/svc/log/oxide-zone-network-setup:default.log
10932024-09-23T18:29:05.671Zuploading: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_crucible_44aeefad-3a65-438e-8790-1c7a579bbb49/root/var/svc/log/oxide-crucible-agent:default.log (36919 bytes)
10942024-09-23T18:29:06.834Zuploaded: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_crucible_44aeefad-3a65-438e-8790-1c7a579bbb49/root/var/svc/log/oxide-crucible-agent:default.log
10952024-09-23T18:29:06.834Zuploading: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_crucible_44aeefad-3a65-438e-8790-1c7a579bbb49/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
10962024-09-23T18:29:07.928Zuploaded: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_crucible_44aeefad-3a65-438e-8790-1c7a579bbb49/root/var/svc/log/oxide-crucible-downstairs:default.log
10972024-09-23T18:29:07.928Zuploading: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_crucible_44aeefad-3a65-438e-8790-1c7a579bbb49/root/var/svc/log/oxide-crucible-downstairs:downstairs-327e9947-f7e1-4b97-92df-e5d731ca17a8.log (5993 bytes)
10982024-09-23T18:29:09.020Zuploaded: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_crucible_44aeefad-3a65-438e-8790-1c7a579bbb49/root/var/svc/log/oxide-crucible-downstairs:downstairs-327e9947-f7e1-4b97-92df-e5d731ca17a8.log
10992024-09-23T18:29:09.020Zuploading: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_crucible_44aeefad-3a65-438e-8790-1c7a579bbb49/root/var/svc/log/oxide-crucible-downstairs:downstairs-5974210c-41ae-4667-b206-8171012260ea.log (6076 bytes)
11002024-09-23T18:29:10.114Zuploaded: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_crucible_44aeefad-3a65-438e-8790-1c7a579bbb49/root/var/svc/log/oxide-crucible-downstairs:downstairs-5974210c-41ae-4667-b206-8171012260ea.log
11012024-09-23T18:29:10.114Zuploading: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_crucible_44aeefad-3a65-438e-8790-1c7a579bbb49/root/var/svc/log/oxide-crucible-downstairs:snapshot-5974210c-41ae-4667-b206-8171012260ea-bf6b8e04-a5bf-448b-a96c-5d70aa90f8a7.log (5688 bytes)
11022024-09-23T18:29:11.207Zuploaded: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_crucible_44aeefad-3a65-438e-8790-1c7a579bbb49/root/var/svc/log/oxide-crucible-downstairs:snapshot-5974210c-41ae-4667-b206-8171012260ea-bf6b8e04-a5bf-448b-a96c-5d70aa90f8a7.log
11032024-09-23T18:29:11.207Zuploading: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_crucible_44aeefad-3a65-438e-8790-1c7a579bbb49/root/var/svc/log/oxide-zone-network-setup:default.log (1674 bytes)
11042024-09-23T18:29:12.301Zuploaded: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_crucible_44aeefad-3a65-438e-8790-1c7a579bbb49/root/var/svc/log/oxide-zone-network-setup:default.log
11052024-09-23T18:29:12.301Zuploading: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_nexus_efc72cb3-831b-4b4a-acac-bb2cd9209275/root/var/svc/log/oxide-nexus:default.log (36631082 bytes)
11062024-09-23T18:29:13.880Zupload warning: file "/pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_nexus_efc72cb3-831b-4b4a-acac-bb2cd9209275/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 36631082 -> 36943927
11072024-09-23T18:29:14.953Zuploaded: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_nexus_efc72cb3-831b-4b4a-acac-bb2cd9209275/root/var/svc/log/oxide-nexus:default.log
11082024-09-23T18:29:14.985Zuploading: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_nexus_efc72cb3-831b-4b4a-acac-bb2cd9209275/root/var/svc/log/oxide-opte-interface-setup:default.log (1135 bytes)
11092024-09-23T18:29:16.060Zuploaded: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_nexus_efc72cb3-831b-4b4a-acac-bb2cd9209275/root/var/svc/log/oxide-opte-interface-setup:default.log
11102024-09-23T18:29:16.060Zuploading: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_nexus_efc72cb3-831b-4b4a-acac-bb2cd9209275/root/var/svc/log/oxide-zone-network-setup:default.log (1654 bytes)
11112024-09-23T18:29:17.182Zuploaded: /pool/ext/1f7ede35-40db-4a1a-8087-fc87b6705700/crypt/zone/oxz_nexus_efc72cb3-831b-4b4a-acac-bb2cd9209275/root/var/svc/log/oxide-zone-network-setup:default.log
11122024-09-23T18:29:17.182Zuploading: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_cockroachdb_349f8cac-ee32-4b29-afb5-5e8a126d2e81/root/var/svc/log/oxide-cockroach-admin:default.log (3727 bytes)
11132024-09-23T18:29:18.349Zuploaded: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_cockroachdb_349f8cac-ee32-4b29-afb5-5e8a126d2e81/root/var/svc/log/oxide-cockroach-admin:default.log
11142024-09-23T18:29:18.349Zuploading: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_cockroachdb_349f8cac-ee32-4b29-afb5-5e8a126d2e81/root/var/svc/log/oxide-cockroachdb:default.log (4314 bytes)
11152024-09-23T18:29:19.451Zuploaded: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_cockroachdb_349f8cac-ee32-4b29-afb5-5e8a126d2e81/root/var/svc/log/oxide-cockroachdb:default.log
11162024-09-23T18:29:19.451Zuploading: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_cockroachdb_349f8cac-ee32-4b29-afb5-5e8a126d2e81/root/var/svc/log/oxide-zone-network-setup:default.log (1682 bytes)
11172024-09-23T18:29:20.570Zuploaded: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_cockroachdb_349f8cac-ee32-4b29-afb5-5e8a126d2e81/root/var/svc/log/oxide-zone-network-setup:default.log
11182024-09-23T18:29:20.570Zuploading: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_crucible_d28c60ef-c4db-4c3e-8e22-baaa61f68157/root/var/svc/log/oxide-crucible-agent:default.log (8485 bytes)
11192024-09-23T18:29:21.687Zuploaded: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_crucible_d28c60ef-c4db-4c3e-8e22-baaa61f68157/root/var/svc/log/oxide-crucible-agent:default.log
11202024-09-23T18:29:21.687Zuploading: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_crucible_d28c60ef-c4db-4c3e-8e22-baaa61f68157/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11212024-09-23T18:29:22.784Zuploaded: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_crucible_d28c60ef-c4db-4c3e-8e22-baaa61f68157/root/var/svc/log/oxide-crucible-downstairs:default.log
11222024-09-23T18:29:22.784Zuploading: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_crucible_d28c60ef-c4db-4c3e-8e22-baaa61f68157/root/var/svc/log/oxide-crucible-downstairs:downstairs-df3bb6f5-1eb8-4c38-be05-972fb2a29e1d.log (3020 bytes)
11232024-09-23T18:29:23.887Zuploaded: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_crucible_d28c60ef-c4db-4c3e-8e22-baaa61f68157/root/var/svc/log/oxide-crucible-downstairs:downstairs-df3bb6f5-1eb8-4c38-be05-972fb2a29e1d.log
11242024-09-23T18:29:23.887Zuploading: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_crucible_d28c60ef-c4db-4c3e-8e22-baaa61f68157/root/var/svc/log/oxide-zone-network-setup:default.log (1675 bytes)
11252024-09-23T18:29:24.982Zuploaded: /pool/ext/4edf146e-3ed8-4c23-b131-d4ed33864007/crypt/zone/oxz_crucible_d28c60ef-c4db-4c3e-8e22-baaa61f68157/root/var/svc/log/oxide-zone-network-setup:default.log
11262024-09-23T18:29:24.982Zuploading: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_cockroachdb_02a2a6f5-4966-4fbe-96a2-cc8c17266bdf/root/var/svc/log/oxide-cockroach-admin:default.log (3727 bytes)
11272024-09-23T18:29:26.077Zuploaded: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_cockroachdb_02a2a6f5-4966-4fbe-96a2-cc8c17266bdf/root/var/svc/log/oxide-cockroach-admin:default.log
11282024-09-23T18:29:26.077Zuploading: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_cockroachdb_02a2a6f5-4966-4fbe-96a2-cc8c17266bdf/root/var/svc/log/oxide-cockroachdb:default.log (4288 bytes)
11292024-09-23T18:29:27.173Zuploaded: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_cockroachdb_02a2a6f5-4966-4fbe-96a2-cc8c17266bdf/root/var/svc/log/oxide-cockroachdb:default.log
11302024-09-23T18:29:27.173Zuploading: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_cockroachdb_02a2a6f5-4966-4fbe-96a2-cc8c17266bdf/root/var/svc/log/oxide-zone-network-setup:default.log (1682 bytes)
11312024-09-23T18:29:28.267Zuploaded: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_cockroachdb_02a2a6f5-4966-4fbe-96a2-cc8c17266bdf/root/var/svc/log/oxide-zone-network-setup:default.log
11322024-09-23T18:29:28.267Zuploading: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_crucible_96a2bbcf-5e5b-4bf1-82f3-5b2051b02712/root/var/svc/log/oxide-crucible-agent:default.log (19467 bytes)
11332024-09-23T18:29:29.404Zuploaded: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_crucible_96a2bbcf-5e5b-4bf1-82f3-5b2051b02712/root/var/svc/log/oxide-crucible-agent:default.log
11342024-09-23T18:29:29.404Zuploading: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_crucible_96a2bbcf-5e5b-4bf1-82f3-5b2051b02712/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11352024-09-23T18:29:30.499Zuploaded: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_crucible_96a2bbcf-5e5b-4bf1-82f3-5b2051b02712/root/var/svc/log/oxide-crucible-downstairs:default.log
11362024-09-23T18:29:30.499Zuploading: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_crucible_96a2bbcf-5e5b-4bf1-82f3-5b2051b02712/root/var/svc/log/oxide-crucible-downstairs:downstairs-0f546427-ab01-4d36-81df-a087473a1144.log (3124 bytes)
11372024-09-23T18:29:31.595Zuploaded: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_crucible_96a2bbcf-5e5b-4bf1-82f3-5b2051b02712/root/var/svc/log/oxide-crucible-downstairs:downstairs-0f546427-ab01-4d36-81df-a087473a1144.log
11382024-09-23T18:29:31.595Zuploading: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_crucible_96a2bbcf-5e5b-4bf1-82f3-5b2051b02712/root/var/svc/log/oxide-crucible-downstairs:downstairs-45bd6f39-1fd5-4cf5-a49d-c0043d55a8fd.log (3019 bytes)
11392024-09-23T18:29:32.688Zuploaded: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_crucible_96a2bbcf-5e5b-4bf1-82f3-5b2051b02712/root/var/svc/log/oxide-crucible-downstairs:downstairs-45bd6f39-1fd5-4cf5-a49d-c0043d55a8fd.log
11402024-09-23T18:29:32.689Zuploading: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_crucible_96a2bbcf-5e5b-4bf1-82f3-5b2051b02712/root/var/svc/log/oxide-zone-network-setup:default.log (1675 bytes)
11412024-09-23T18:29:33.782Zuploaded: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_crucible_96a2bbcf-5e5b-4bf1-82f3-5b2051b02712/root/var/svc/log/oxide-zone-network-setup:default.log
11422024-09-23T18:29:33.811Zuploading: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_oximeter_43821df6-7a7b-49df-9a53-ebf281e00a55/root/var/svc/log/oxide-oximeter:default.log (1812855 bytes)
11432024-09-23T18:29:34.143Zupload warning: file "/pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_oximeter_43821df6-7a7b-49df-9a53-ebf281e00a55/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 1812855 -> 2120398
11442024-09-23T18:29:35.212Zuploaded: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_oximeter_43821df6-7a7b-49df-9a53-ebf281e00a55/root/var/svc/log/oxide-oximeter:default.log
11452024-09-23T18:29:35.212Zuploading: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_oximeter_43821df6-7a7b-49df-9a53-ebf281e00a55/root/var/svc/log/oxide-zone-network-setup:default.log (1673 bytes)
11462024-09-23T18:29:36.355Zuploaded: /pool/ext/67a20504-7be3-48ff-bb32-7f9411b1342b/crypt/zone/oxz_oximeter_43821df6-7a7b-49df-9a53-ebf281e00a55/root/var/svc/log/oxide-zone-network-setup:default.log
11472024-09-23T18:29:36.355Zuploading: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_crucible_2818b80d-465e-48ce-92dc-2436b4b704cd/root/var/svc/log/oxide-crucible-agent:default.log (12504 bytes)
11482024-09-23T18:29:37.485Zuploaded: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_crucible_2818b80d-465e-48ce-92dc-2436b4b704cd/root/var/svc/log/oxide-crucible-agent:default.log
11492024-09-23T18:29:37.485Zuploading: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_crucible_2818b80d-465e-48ce-92dc-2436b4b704cd/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11502024-09-23T18:29:38.580Zuploaded: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_crucible_2818b80d-465e-48ce-92dc-2436b4b704cd/root/var/svc/log/oxide-crucible-downstairs:default.log
11512024-09-23T18:29:38.604Zuploading: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_crucible_2818b80d-465e-48ce-92dc-2436b4b704cd/root/var/svc/log/oxide-crucible-downstairs:downstairs-f7bbeb5c-0f98-4681-8486-56046ce3ad65.log (5995 bytes)
11522024-09-23T18:29:39.675Zuploaded: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_crucible_2818b80d-465e-48ce-92dc-2436b4b704cd/root/var/svc/log/oxide-crucible-downstairs:downstairs-f7bbeb5c-0f98-4681-8486-56046ce3ad65.log
11532024-09-23T18:29:39.675Zuploading: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_crucible_2818b80d-465e-48ce-92dc-2436b4b704cd/root/var/svc/log/oxide-zone-network-setup:default.log (1675 bytes)
11542024-09-23T18:29:40.769Zuploaded: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_crucible_2818b80d-465e-48ce-92dc-2436b4b704cd/root/var/svc/log/oxide-zone-network-setup:default.log
11552024-09-23T18:29:40.769Zuploading: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_nexus_b9740b46-002a-4447-8a9d-58caea4925b7/root/var/svc/log/oxide-nexus:default.log (2860034 bytes)
11562024-09-23T18:29:41.145Zupload warning: file "/pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_nexus_b9740b46-002a-4447-8a9d-58caea4925b7/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 2860034 -> 3358795
11572024-09-23T18:29:42.214Zuploaded: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_nexus_b9740b46-002a-4447-8a9d-58caea4925b7/root/var/svc/log/oxide-nexus:default.log
11582024-09-23T18:29:42.214Zuploading: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_nexus_b9740b46-002a-4447-8a9d-58caea4925b7/root/var/svc/log/oxide-opte-interface-setup:default.log (1134 bytes)
11592024-09-23T18:29:43.307Zuploaded: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_nexus_b9740b46-002a-4447-8a9d-58caea4925b7/root/var/svc/log/oxide-opte-interface-setup:default.log
11602024-09-23T18:29:43.307Zuploading: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_nexus_b9740b46-002a-4447-8a9d-58caea4925b7/root/var/svc/log/oxide-zone-network-setup:default.log (1654 bytes)
11612024-09-23T18:29:44.402Zuploaded: /pool/ext/73f13013-70b2-4020-853f-5a70920c296f/crypt/zone/oxz_nexus_b9740b46-002a-4447-8a9d-58caea4925b7/root/var/svc/log/oxide-zone-network-setup:default.log
11622024-09-23T18:29:44.402Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_cockroachdb_ffb68cf7-819b-4482-96ea-a92802127e4d/root/var/svc/log/oxide-cockroach-admin:default.log (3727 bytes)
11632024-09-23T18:29:45.496Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_cockroachdb_ffb68cf7-819b-4482-96ea-a92802127e4d/root/var/svc/log/oxide-cockroach-admin:default.log
11642024-09-23T18:29:45.496Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_cockroachdb_ffb68cf7-819b-4482-96ea-a92802127e4d/root/var/svc/log/oxide-cockroachdb:default.log (4315 bytes)
11652024-09-23T18:29:46.590Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_cockroachdb_ffb68cf7-819b-4482-96ea-a92802127e4d/root/var/svc/log/oxide-cockroachdb:default.log
11662024-09-23T18:29:46.590Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_cockroachdb_ffb68cf7-819b-4482-96ea-a92802127e4d/root/var/svc/log/oxide-zone-network-setup:default.log (1681 bytes)
11672024-09-23T18:29:47.684Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_cockroachdb_ffb68cf7-819b-4482-96ea-a92802127e4d/root/var/svc/log/oxide-zone-network-setup:default.log
11682024-09-23T18:29:47.684Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_crucible_b473542b-f56f-44f7-ac97-717a7c8f5b46/root/var/svc/log/oxide-crucible-agent:default.log (12500 bytes)
11692024-09-23T18:29:48.797Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_crucible_b473542b-f56f-44f7-ac97-717a7c8f5b46/root/var/svc/log/oxide-crucible-agent:default.log
11702024-09-23T18:29:48.797Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_crucible_b473542b-f56f-44f7-ac97-717a7c8f5b46/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
11712024-09-23T18:29:49.890Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_crucible_b473542b-f56f-44f7-ac97-717a7c8f5b46/root/var/svc/log/oxide-crucible-downstairs:default.log
11722024-09-23T18:29:49.890Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_crucible_b473542b-f56f-44f7-ac97-717a7c8f5b46/root/var/svc/log/oxide-crucible-downstairs:downstairs-54613bbd-d5bd-47fc-93f3-2a844dea187b.log (3125 bytes)
11732024-09-23T18:29:50.983Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_crucible_b473542b-f56f-44f7-ac97-717a7c8f5b46/root/var/svc/log/oxide-crucible-downstairs:downstairs-54613bbd-d5bd-47fc-93f3-2a844dea187b.log
11742024-09-23T18:29:50.983Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_crucible_b473542b-f56f-44f7-ac97-717a7c8f5b46/root/var/svc/log/oxide-zone-network-setup:default.log (1675 bytes)
11752024-09-23T18:29:52.076Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_crucible_b473542b-f56f-44f7-ac97-717a7c8f5b46/root/var/svc/log/oxide-zone-network-setup:default.log
11762024-09-23T18:29:52.076Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_crucible_pantry_4c6fe4b9-aa70-4c11-a8cb-78727c3b2349/root/var/svc/log/oxide-crucible-pantry:default.log (782 bytes)
11772024-09-23T18:29:53.171Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_crucible_pantry_4c6fe4b9-aa70-4c11-a8cb-78727c3b2349/root/var/svc/log/oxide-crucible-pantry:default.log
11782024-09-23T18:29:53.171Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_crucible_pantry_4c6fe4b9-aa70-4c11-a8cb-78727c3b2349/root/var/svc/log/oxide-zone-network-setup:default.log (1717 bytes)
11792024-09-23T18:29:54.264Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_crucible_pantry_4c6fe4b9-aa70-4c11-a8cb-78727c3b2349/root/var/svc/log/oxide-zone-network-setup:default.log
11802024-09-23T18:29:54.264Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_external_dns_cd3b9879-6155-40d6-93a9-0c64a75a3e6e/root/var/svc/log/oxide-external_dns:default.log (26905 bytes)
11812024-09-23T18:29:54.377Zupload warning: file "/pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_external_dns_cd3b9879-6155-40d6-93a9-0c64a75a3e6e/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 26905 -> 34458
11822024-09-23T18:29:55.447Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_external_dns_cd3b9879-6155-40d6-93a9-0c64a75a3e6e/root/var/svc/log/oxide-external_dns:default.log
11832024-09-23T18:29:55.447Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_external_dns_cd3b9879-6155-40d6-93a9-0c64a75a3e6e/root/var/svc/log/oxide-opte-interface-setup:default.log (1155 bytes)
11842024-09-23T18:29:56.540Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_external_dns_cd3b9879-6155-40d6-93a9-0c64a75a3e6e/root/var/svc/log/oxide-opte-interface-setup:default.log
11852024-09-23T18:29:56.540Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_external_dns_cd3b9879-6155-40d6-93a9-0c64a75a3e6e/root/var/svc/log/oxide-zone-network-setup:default.log (1697 bytes)
11862024-09-23T18:29:57.637Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_external_dns_cd3b9879-6155-40d6-93a9-0c64a75a3e6e/root/var/svc/log/oxide-zone-network-setup:default.log
11872024-09-23T18:29:57.637Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_internal_dns_df62fab1-4575-460a-9193-f90470451eed/root/var/svc/log/oxide-internal_dns:default.log (406847 bytes)
11882024-09-23T18:29:57.867Zupload warning: file "/pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_internal_dns_df62fab1-4575-460a-9193-f90470451eed/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 406847 -> 414359
11892024-09-23T18:29:58.944Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_internal_dns_df62fab1-4575-460a-9193-f90470451eed/root/var/svc/log/oxide-internal_dns:default.log
11902024-09-23T18:29:58.944Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_internal_dns_df62fab1-4575-460a-9193-f90470451eed/root/var/svc/log/oxide-zone-network-setup:default.log (1679 bytes)
11912024-09-23T18:30:00.039Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_internal_dns_df62fab1-4575-460a-9193-f90470451eed/root/var/svc/log/oxide-zone-network-setup:default.log
11922024-09-23T18:30:00.039Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_nexus_501a8d46-a8ba-41dd-888b-350783c6dce3/root/var/svc/log/oxide-nexus:default.log (5139325 bytes)
11932024-09-23T18:30:00.510Zupload warning: file "/pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_nexus_501a8d46-a8ba-41dd-888b-350783c6dce3/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 5139325 -> 5876580
11942024-09-23T18:30:01.580Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_nexus_501a8d46-a8ba-41dd-888b-350783c6dce3/root/var/svc/log/oxide-nexus:default.log
11952024-09-23T18:30:01.580Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_nexus_501a8d46-a8ba-41dd-888b-350783c6dce3/root/var/svc/log/oxide-opte-interface-setup:default.log (1135 bytes)
11962024-09-23T18:30:02.673Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_nexus_501a8d46-a8ba-41dd-888b-350783c6dce3/root/var/svc/log/oxide-opte-interface-setup:default.log
11972024-09-23T18:30:02.673Zuploading: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_nexus_501a8d46-a8ba-41dd-888b-350783c6dce3/root/var/svc/log/oxide-zone-network-setup:default.log (1654 bytes)
11982024-09-23T18:30:03.767Zuploaded: /pool/ext/b5c5f1b9-c752-4261-a49d-06cee1ff1f1a/crypt/zone/oxz_nexus_501a8d46-a8ba-41dd-888b-350783c6dce3/root/var/svc/log/oxide-zone-network-setup:default.log
11992024-09-23T18:30:03.767Zuploading: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_7c0b2a6b-4c9a-4820-a10b-e91676739409/root/var/svc/log/oxide-crucible-agent:default.log (28517 bytes)
12002024-09-23T18:30:04.906Zuploaded: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_7c0b2a6b-4c9a-4820-a10b-e91676739409/root/var/svc/log/oxide-crucible-agent:default.log
12012024-09-23T18:30:04.906Zuploading: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_7c0b2a6b-4c9a-4820-a10b-e91676739409/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12022024-09-23T18:30:06.002Zuploaded: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_7c0b2a6b-4c9a-4820-a10b-e91676739409/root/var/svc/log/oxide-crucible-downstairs:default.log
12032024-09-23T18:30:06.002Zuploading: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_7c0b2a6b-4c9a-4820-a10b-e91676739409/root/var/svc/log/oxide-crucible-downstairs:downstairs-1b756892-1a53-4137-8ea8-7284a952e257.log (6077 bytes)
12042024-09-23T18:30:07.095Zuploaded: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_7c0b2a6b-4c9a-4820-a10b-e91676739409/root/var/svc/log/oxide-crucible-downstairs:downstairs-1b756892-1a53-4137-8ea8-7284a952e257.log
12052024-09-23T18:30:07.095Zuploading: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_7c0b2a6b-4c9a-4820-a10b-e91676739409/root/var/svc/log/oxide-crucible-downstairs:downstairs-f831f550-a0fa-4294-9f94-6f7ab66079ba.log (3127 bytes)
12062024-09-23T18:30:08.188Zuploaded: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_7c0b2a6b-4c9a-4820-a10b-e91676739409/root/var/svc/log/oxide-crucible-downstairs:downstairs-f831f550-a0fa-4294-9f94-6f7ab66079ba.log
12072024-09-23T18:30:08.188Zuploading: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_7c0b2a6b-4c9a-4820-a10b-e91676739409/root/var/svc/log/oxide-crucible-downstairs:snapshot-1b756892-1a53-4137-8ea8-7284a952e257-bf6b8e04-a5bf-448b-a96c-5d70aa90f8a7.log (5685 bytes)
12082024-09-23T18:30:09.283Zuploaded: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_7c0b2a6b-4c9a-4820-a10b-e91676739409/root/var/svc/log/oxide-crucible-downstairs:snapshot-1b756892-1a53-4137-8ea8-7284a952e257-bf6b8e04-a5bf-448b-a96c-5d70aa90f8a7.log
12092024-09-23T18:30:09.283Zuploading: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_7c0b2a6b-4c9a-4820-a10b-e91676739409/root/var/svc/log/oxide-zone-network-setup:default.log (1673 bytes)
12102024-09-23T18:30:10.380Zuploaded: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_7c0b2a6b-4c9a-4820-a10b-e91676739409/root/var/svc/log/oxide-zone-network-setup:default.log
12112024-09-23T18:30:10.380Zuploading: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_pantry_6837578d-2f18-4d5d-82e1-b69d1cbf6f4c/root/var/svc/log/oxide-crucible-pantry:default.log (783 bytes)
12122024-09-23T18:30:11.476Zuploaded: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_pantry_6837578d-2f18-4d5d-82e1-b69d1cbf6f4c/root/var/svc/log/oxide-crucible-pantry:default.log
12132024-09-23T18:30:11.476Zuploading: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_pantry_6837578d-2f18-4d5d-82e1-b69d1cbf6f4c/root/var/svc/log/oxide-zone-network-setup:default.log (1715 bytes)
12142024-09-23T18:30:12.570Zuploaded: /pool/ext/c6321c72-f89d-471a-8986-1f1bf65a3de2/crypt/zone/oxz_crucible_pantry_6837578d-2f18-4d5d-82e1-b69d1cbf6f4c/root/var/svc/log/oxide-zone-network-setup:default.log
12152024-09-23T18:30:12.570Zuploading: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_cockroachdb_65e6efb1-be8e-412e-b4dc-184f63bb1b9e/root/var/svc/log/oxide-cockroach-admin:default.log (3726 bytes)
12162024-09-23T18:30:13.663Zuploaded: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_cockroachdb_65e6efb1-be8e-412e-b4dc-184f63bb1b9e/root/var/svc/log/oxide-cockroach-admin:default.log
12172024-09-23T18:30:13.663Zuploading: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_cockroachdb_65e6efb1-be8e-412e-b4dc-184f63bb1b9e/root/var/svc/log/oxide-cockroachdb:default.log (4314 bytes)
12182024-09-23T18:30:14.763Zuploaded: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_cockroachdb_65e6efb1-be8e-412e-b4dc-184f63bb1b9e/root/var/svc/log/oxide-cockroachdb:default.log
12192024-09-23T18:30:14.763Zuploading: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_cockroachdb_65e6efb1-be8e-412e-b4dc-184f63bb1b9e/root/var/svc/log/oxide-zone-network-setup:default.log (1682 bytes)
12202024-09-23T18:30:15.860Zuploaded: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_cockroachdb_65e6efb1-be8e-412e-b4dc-184f63bb1b9e/root/var/svc/log/oxide-zone-network-setup:default.log
12212024-09-23T18:30:15.860Zuploading: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_crucible_5d81eec0-e923-40e4-875c-abea1d4aed53/root/var/svc/log/oxide-crucible-agent:default.log (1556 bytes)
12222024-09-23T18:30:16.953Zuploaded: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_crucible_5d81eec0-e923-40e4-875c-abea1d4aed53/root/var/svc/log/oxide-crucible-agent:default.log
12232024-09-23T18:30:16.954Zuploading: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_crucible_5d81eec0-e923-40e4-875c-abea1d4aed53/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes)
12242024-09-23T18:30:18.048Zuploaded: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_crucible_5d81eec0-e923-40e4-875c-abea1d4aed53/root/var/svc/log/oxide-crucible-downstairs:default.log
12252024-09-23T18:30:18.049Zuploading: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_crucible_5d81eec0-e923-40e4-875c-abea1d4aed53/root/var/svc/log/oxide-zone-network-setup:default.log (1672 bytes)
12262024-09-23T18:30:19.143Zuploaded: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_crucible_5d81eec0-e923-40e4-875c-abea1d4aed53/root/var/svc/log/oxide-zone-network-setup:default.log
12272024-09-23T18:30:19.143Zuploading: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_crucible_pantry_336574a9-87bb-450e-9ef7-3aa3e0e79905/root/var/svc/log/oxide-crucible-pantry:default.log (763913 bytes)
12282024-09-23T18:30:20.470Zuploaded: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_crucible_pantry_336574a9-87bb-450e-9ef7-3aa3e0e79905/root/var/svc/log/oxide-crucible-pantry:default.log
12292024-09-23T18:30:20.470Zuploading: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_crucible_pantry_336574a9-87bb-450e-9ef7-3aa3e0e79905/root/var/svc/log/oxide-zone-network-setup:default.log (1711 bytes)
12302024-09-23T18:30:21.564Zuploaded: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_crucible_pantry_336574a9-87bb-450e-9ef7-3aa3e0e79905/root/var/svc/log/oxide-zone-network-setup:default.log
12312024-09-23T18:30:21.564Zuploading: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_internal_dns_a816df87-b330-4b29-8c67-45ce2059ce69/root/var/svc/log/oxide-internal_dns:default.log (406840 bytes)
12322024-09-23T18:30:21.795Zupload warning: file "/pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_internal_dns_a816df87-b330-4b29-8c67-45ce2059ce69/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 406840 -> 414352
12332024-09-23T18:30:22.866Zuploaded: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_internal_dns_a816df87-b330-4b29-8c67-45ce2059ce69/root/var/svc/log/oxide-internal_dns:default.log
12342024-09-23T18:30:22.890Zuploading: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_internal_dns_a816df87-b330-4b29-8c67-45ce2059ce69/root/var/svc/log/oxide-zone-network-setup:default.log (1680 bytes)
12352024-09-23T18:30:23.964Zuploaded: /pool/ext/e381d5a5-91f1-4b1c-b0ad-128e813a0a94/crypt/zone/oxz_internal_dns_a816df87-b330-4b29-8c67-45ce2059ce69/root/var/svc/log/oxide-zone-network-setup:default.log
12362024-09-23T18:30:23.964Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/log/chrony/measurements.log (141932 bytes)
12372024-09-23T18:30:24.149Zupload warning: file "/pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/log/chrony/measurements.log" changed size mid upload: 141932 -> 175634
12382024-09-23T18:30:25.221Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/log/chrony/measurements.log
12392024-09-23T18:30:25.221Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/log/chrony/statistics.log (107055 bytes)
12402024-09-23T18:30:25.402Zupload warning: file "/pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/log/chrony/statistics.log" changed size mid upload: 107055 -> 132795
12412024-09-23T18:30:26.475Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/log/chrony/statistics.log
12422024-09-23T18:30:26.475Zuploading: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/log/chrony/tracking.log (18481 bytes)
12432024-09-23T18:30:26.567Zupload warning: file "/pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/log/chrony/tracking.log" changed size mid upload: 18481 -> 20857
12442024-09-23T18:30:27.637Zuploaded: /pool/ext/016c0356-f4b4-4d6b-8c24-edfd2951cfe3/crypt/zone/oxz_ntp_dc6ddc3e-c0dc-4d16-bf6e-b2db4c94de9e/root/var/log/chrony/tracking.log