01HP2REPP05MAA7BWHDSMAYYSH: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log*",
        "%/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/oxide-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_*/root/var/svc/log/system-illumos-*.log*",
        "%/pool/ext/*/crypt/zone/oxz_ntp_*/root/var/log/chrony/*.log*",
        "!/pool/ext/*/crypt/zone/oxz_propolis-server_*/root/var/svc/log/*.log*",
        "%/pool/ext/*/crypt/debug/global/oxide-sled-agent:default.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/oxide-*.log.*",
        "%/pool/ext/*/crypt/debug/oxz_*/system-illumos-*.log.*",
        "!/pool/ext/*/crypt/debug/oxz_propolis-server_*/*.log.*",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-2.0-opte-0.28",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01HP2RFHWD34JV30MRR2CYT3PG

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12024-02-07T22:24:47.007Zjob dependencies complete; ready to run (waiting for 40 m 2 s)
22024-02-07T22:27:39.764Zjob assigned to worker 01HP2TSYCNPXAG40AHCGGW6YCE (queued for 2 m 52 s)
32024-02-07T22:27:39.770Zhttp://172.20.0.5:9969/ipxe/sock... ok
42024-02-07T22:27:39.811Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/kernel/amd64/unix... ok
52024-02-07T22:27:39.845Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/amd64/boot_archive... 2%Error: No response to keepalive - Terminating session
62024-02-07T22:27:39.878ZError: No response de-activating SOL payload
72024-02-07T22:27:39.912Z[SOL Session operational. Use ~? for help]
82024-02-07T22:27:39.956Zcpu0: microcode has been updated from version 0xa001133 to 0xa0011d1
92024-02-07T22:27:40.199ZOxide Helios Version helios-2.0.22451 64-bit
102024-02-07T22:27:40.269ZConfiguring devices.
112024-02-07T22:27:40.312Z##################################################
122024-02-07T22:27:40.354Z#### ############# ##
132024-02-07T22:27:40.421Z### ### ############ ##
142024-02-07T22:27:40.455Z## ### # ## ### ## Oxide Computer Company ##
152024-02-07T22:27:40.494Z## ## ## ### # ### ##
162024-02-07T22:27:40.533Z## # ### #### #### This Station Under ##
172024-02-07T22:27:40.569Z### ### #### # ### Computer Control ##
182024-02-07T22:27:40.608Z#### #### ### ## ##
192024-02-07T22:27:40.646Z##################################################
202024-02-07T22:27:40.684ZHostname: sock
212024-02-07T22:27:40.724ZLoading smf(7) service descriptions:
222024-02-07T22:27:42.443Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap-5c5e256b526718ba.gz
232024-02-07T22:27:45.307Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap-5c5e256b526718ba.gz
242024-02-07T22:27:45.307Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
252024-02-07T22:28:29.104Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
262024-02-07T22:28:29.139Zdownloading input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-398b34350dfa5341.gz
272024-02-07T22:28:53.870Zdownloaded input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-398b34350dfa5341.gz
282024-02-07T22:28:53.870Zdownloading input: /input/ci-tools/work/caboose-util.gz
292024-02-07T22:28:54.854Zdownloaded input: /input/ci-tools/work/caboose-util.gz
302024-02-07T22:28:54.879Zdownloading input: /input/ci-tools/work/tufaceous.gz
312024-02-07T22:28:58.699Zdownloaded input: /input/ci-tools/work/tufaceous.gz
322024-02-07T22:28:58.733Zdownloading input: /input/package/work/version.txt
332024-02-07T22:28:58.759Zdownloaded input: /input/package/work/version.txt
342024-02-07T22:28:58.759Zdownloading input: /input/package/work/package.tar.gz
352024-02-07T22:29:17.630Zdownloaded input: /input/package/work/package.tar.gz
362024-02-07T22:29:17.656Zdownloading input: /input/package/work/global-zone-packages.tar.gz
372024-02-07T22:29:41.447Zdownloaded input: /input/package/work/global-zone-packages.tar.gz
382024-02-07T22:29:41.447Zdownloading input: /input/package/work/trampoline-global-zone-packages.tar.gz
392024-02-07T22:29:47.864Zdownloaded input: /input/package/work/trampoline-global-zone-packages.tar.gz
402024-02-07T22:29:47.888Zdownloading input: /input/package/work/zones/clickhouse.tar.gz
412024-02-07T22:30:04.925Zsock console login: Error: No response to keepalive - Terminating session
422024-02-07T22:30:14.846ZError: No response de-activating SOL payload
432024-02-07T22:30:15.949Z[SOL Session operational. Use ~? for help]
442024-02-07T22:30:23.283Zdownloaded input: /input/package/work/zones/clickhouse.tar.gz
452024-02-07T22:30:23.310Zdownloading input: /input/package/work/zones/clickhouse_keeper.tar.gz
462024-02-07T22:31:00.434Zdownloaded input: /input/package/work/zones/clickhouse_keeper.tar.gz
472024-02-07T22:31:00.434Zdownloading input: /input/package/work/zones/cockroachdb.tar.gz
482024-02-07T22:31:24.659Zdownloaded input: /input/package/work/zones/cockroachdb.tar.gz
492024-02-07T22:31:24.659Zdownloading input: /input/package/work/zones/crucible-pantry.tar.gz
502024-02-07T22:31:27.770Zdownloaded input: /input/package/work/zones/crucible-pantry.tar.gz
512024-02-07T22:31:27.796Zdownloading input: /input/package/work/zones/crucible.tar.gz
522024-02-07T22:31:33.879Zdownloaded input: /input/package/work/zones/crucible.tar.gz
532024-02-07T22:31:33.880Zdownloading input: /input/package/work/zones/external-dns.tar.gz
542024-02-07T22:31:38.201Zdownloaded input: /input/package/work/zones/external-dns.tar.gz
552024-02-07T22:31:38.201Zdownloading input: /input/package/work/zones/internal-dns.tar.gz
562024-02-07T22:31:43.497Zdownloaded input: /input/package/work/zones/internal-dns.tar.gz
572024-02-07T22:31:43.497Zdownloading input: /input/package/work/zones/ntp.tar.gz
582024-02-07T22:31:43.542Zdownloaded input: /input/package/work/zones/ntp.tar.gz
592024-02-07T22:31:43.542Zdownloading input: /input/package/work/zones/omicron-gateway-asic.tar.gz
602024-02-07T22:31:47.078Zdownloaded input: /input/package/work/zones/omicron-gateway-asic.tar.gz
612024-02-07T22:31:47.078Zdownloading input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
622024-02-07T22:31:50.653Zdownloaded input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz
632024-02-07T22:31:50.653Zdownloading input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
642024-02-07T22:32:10.755Zdownloaded input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz
652024-02-07T22:32:10.755Zdownloading input: /input/package/work/zones/omicron-nexus.tar.gz
662024-02-07T22:32:27.291Zdownloaded input: /input/package/work/zones/omicron-nexus.tar.gz
672024-02-07T22:32:27.291Zdownloading input: /input/package/work/zones/overlay.tar.gz
682024-02-07T22:32:27.386Zdownloaded input: /input/package/work/zones/overlay.tar.gz
692024-02-07T22:32:27.387Zdownloading input: /input/package/work/zones/oximeter.tar.gz
702024-02-07T22:32:39.253Zdownloaded input: /input/package/work/zones/oximeter.tar.gz
712024-02-07T22:32:39.253Zdownloading input: /input/package/work/zones/propolis-server.tar.gz
722024-02-07T22:32:53.094Zdownloaded input: /input/package/work/zones/propolis-server.tar.gz
732024-02-07T22:32:53.094Zdownloading input: /input/package/work/zones/switch-asic.tar.gz
742024-02-07T22:33:04.851ZError: No response to keepalive - Terminating session
752024-02-07T22:33:14.773ZError: No response de-activating SOL payload
762024-02-07T22:33:16.162Z[SOL Session operational. Use ~? for help]
772024-02-07T22:33:49.360Zdownloaded input: /input/package/work/zones/switch-asic.tar.gz
782024-02-07T22:33:49.360Zdownloading input: /input/package/work/zones/switch-softnpu.tar.gz
792024-02-07T22:34:40.474Zdownloaded input: /input/package/work/zones/switch-softnpu.tar.gz
 
802024-02-07T22:34:40.499Zstarting task 0: "setup"
812024-02-07T22:34:40.499Z++ uname -s
822024-02-07T22:34:40.499Z+ kern=SunOS
832024-02-07T22:34:40.499Z+ case "$kern" in
842024-02-07T22:34:40.499Z+ groupadd -g 12345 build
852024-02-07T22:34:40.500Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
862024-02-07T22:34:42.496Z+ zfs create -o mountpoint=/work rpool/work
872024-02-07T22:34:42.529Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
882024-02-07T22:34:42.530Z+ home_fs=zfs
892024-02-07T22:34:42.530Z+ [[ zfs == autofs ]]
902024-02-07T22:34:42.530Z+ mkdir -p /home/build
912024-02-07T22:34:42.530Z+ chown build:build /home/build /work
922024-02-07T22:34:44.526Z+ chmod 0700 /home/build /work
932024-02-07T22:34:44.552Zprocess exited: duration 4055 ms, exit code 0
 
942024-02-07T22:34:44.602Zstarting task 1: "authentication"
952024-02-07T22:34:44.644Zprocess exited: duration 42 ms, exit code 0
 
962024-02-07T22:34:44.695Zstarting task 2: "build"
972024-02-07T22:34:44.719Z+ trap _exit_trap EXIT
982024-02-07T22:34:44.719Z+ set +x
992024-02-07T22:35:39.630Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
1002024-02-07T22:35:39.873Zsecpolicy_fs_linkdir: 0x55 = 0x55
1012024-02-07T22:35:39.874Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
1022024-02-07T22:35:39.874Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
1032024-02-07T22:35:39.874Zsecpolicy_fs_linkdir+3: 0 = 0xe5
1042024-02-07T22:35:39.874Zsecpolicy_fs_linkdir+4: 0 = 0xb8
1052024-02-07T22:35:39.874Zsecpolicy_fs_linkdir+5: 0 = 0x1
1062024-02-07T22:35:39.874Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
1072024-02-07T22:35:39.874Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
1082024-02-07T22:35:39.874Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
1092024-02-07T22:35:39.874Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
1102024-02-07T22:35:39.874Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
1112024-02-07T22:35:39.899Z+ [[ -d /opt/oxide ]]
1122024-02-07T22:35:39.899Z+ pfexec mv /opt/oxide /opt/oxide-underneath
1132024-02-07T22:35:39.899Z+ pfexec mkdir /opt/oxide
1142024-02-07T22:35:39.899Z+ pfexec mount -F tmpfs -O swap /opt/oxide
1152024-02-07T22:35:39.899Z+ [[ -d /opt/oxide-underneath ]]
1162024-02-07T22:35:39.899Z+ cd /opt/oxide-underneath
1172024-02-07T22:35:39.899Z+ pfexec tar ceEp@/f - .
1182024-02-07T22:35:39.899Z+ cd /opt/oxide
1192024-02-07T22:35:39.899Z+ pfexec tar xveEp@/f -
1202024-02-07T22:35:39.942Zx ., 0 bytes, 0 tape blocks
1212024-02-07T22:35:39.970Zx ./opte, 0 bytes, 0 tape blocks
1222024-02-07T22:35:39.970Zx ./opte/bin, 0 bytes, 0 tape blocks
1232024-02-07T22:35:39.970Zx ./opte/bin/opteadm, 51799224 bytes, 101171 tape blocks
1242024-02-07T22:35:39.971Z+ pfexec rm -rf /opt/oxide-underneath
1252024-02-07T22:35:39.971Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
1262024-02-07T22:35:39.971Z+ pfexec mkdir /opt/oxide/work
1272024-02-07T22:35:39.971Z+ pfexec chown build:build /opt/oxide/work
1282024-02-07T22:35:39.995Z+ cd /opt/oxide/work
1292024-02-07T22:35:39.995Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
1302024-02-07T22:35:41.665ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
1312024-02-07T22:35:41.665Zx out/mg-ddm-gz.tar, 64939008 bytes, 126834 tape blocks
1322024-02-07T22:35:41.665Zx out/omicron-sled-agent.tar, 122618368 bytes, 239489 tape blocks
1332024-02-07T22:35:41.665Zx out/oxlog.tar, 8382464 bytes, 16372 tape blocks
1342024-02-07T22:35:41.665Zx out/target/test, 75 bytes, 1 tape blocks
1352024-02-07T22:35:41.665Zx out/npuzone/npuzone, 37184104 bytes, 72626 tape blocks
1362024-02-07T22:35:41.665Zx package-manifest.toml, 23732 bytes, 47 tape blocks
1372024-02-07T22:35:41.666Zx smf/sled-agent/non-gimlet/config.toml, 3448 bytes, 7 tape blocks
1382024-02-07T22:35:41.666Zx target/release/omicron-package, 45712928 bytes, 89284 tape blocks
1392024-02-07T22:35:41.666Zx tools/create_virtual_hardware.sh, 3184 bytes, 7 tape blocks
1402024-02-07T22:35:41.666Zx tools/virtual_hardware.sh, 3416 bytes, 7 tape blocks
1412024-02-07T22:35:41.666Zx tools/scrimlet/create-softnpu-zone.sh, 627 bytes, 2 tape blocks
1422024-02-07T22:35:41.666Zx tools/scrimlet/destroy-softnpu-zone.sh, 203 bytes, 1 tape blocks
1432024-02-07T22:35:41.666Zx tools/scrimlet/softnpu-init.sh, 2157 bytes, 5 tape blocks
1442024-02-07T22:35:41.666Zx tools/scrimlet/softnpu-zone.txt, 330 bytes, 1 tape blocks
1452024-02-07T22:35:41.666Zx tools/scrimlet/softnpu.toml, 171 bytes, 1 tape blocks
1462024-02-07T22:35:42.096Z
1472024-02-07T22:35:42.096Zreal 1.690304118
1482024-02-07T22:35:42.096Zuser 1.519102621
1492024-02-07T22:35:42.097Zsys 0.300355693
1502024-02-07T22:35:42.097Ztrap 0.000050482
1512024-02-07T22:35:42.097Ztflt 0.000000000
1522024-02-07T22:35:42.097Zdflt 0.000000000
1532024-02-07T22:35:42.097Zkflt 0.000000000
1542024-02-07T22:35:42.097Zlock 0.000000000
1552024-02-07T22:35:42.097Zslp 1.520093456
1562024-02-07T22:35:42.097Zlat 0.038810113
1572024-02-07T22:35:42.098Zstop 0.000125122
1582024-02-07T22:35:42.098Z+ cp /input/package/work/zones/clickhouse_keeper.tar.gz /input/package/work/zones/clickhouse.tar.gz /input/package/work/zones/cockroachdb.tar.gz /input/package/work/zones/crucible-pantry.tar.gz /input/package/work/zones/crucible.tar.gz /input/package/work/zones/external-dns.tar.gz /input/package/work/zones/internal-dns.tar.gz /input/package/work/zones/ntp.tar.gz /input/package/work/zones/omicron-gateway-asic.tar.gz /input/package/work/zones/omicron-gateway-softnpu.tar.gz /input/package/work/zones/omicron-nexus-single-sled.tar.gz /input/package/work/zones/omicron-nexus.tar.gz /input/package/work/zones/overlay.tar.gz /input/package/work/zones/oximeter.tar.gz /input/package/work/zones/propolis-server.tar.gz /input/package/work/zones/switch-asic.tar.gz /input/package/work/zones/switch-softnpu.tar.gz out/
1592024-02-07T22:35:42.891Z+ mv out/omicron-nexus-single-sled.tar.gz out/omicron-nexus.tar.gz
1602024-02-07T22:35:42.915Z+ mkdir tests
1612024-02-07T22:35:42.915Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1622024-02-07T22:35:42.915Z+ ptime -m gunzip
1632024-02-07T22:35:42.915Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-5c5e256b526718ba
1642024-02-07T22:35:43.151Z
1652024-02-07T22:35:43.216Zreal 0.233726878
1662024-02-07T22:35:43.216Zuser 0.213113192
1672024-02-07T22:35:43.216Zsys 0.020375521
1682024-02-07T22:35:43.216Ztrap 0.000016261
1692024-02-07T22:35:43.216Ztflt 0.000000000
1702024-02-07T22:35:43.217Zdflt 0.000000000
1712024-02-07T22:35:43.217Zkflt 0.000000000
1722024-02-07T22:35:43.217Zlock 0.000000000
1732024-02-07T22:35:43.217Zslp 0.000021480
1742024-02-07T22:35:43.217Zlat 0.000023250
1752024-02-07T22:35:43.217Zstop 0.000118692
1762024-02-07T22:35:43.217Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-5c5e256b526718ba
1772024-02-07T22:35:43.218Z+ chmod a+x tests/bootstrap-5c5e256b526718ba
1782024-02-07T22:35:43.218Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1792024-02-07T22:35:43.219Z+ ptime -m gunzip
1802024-02-07T22:35:43.219Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1812024-02-07T22:35:45.656Z
1822024-02-07T22:35:45.681Zreal 2.493914204
1832024-02-07T22:35:45.681Zuser 2.333994105
1842024-02-07T22:35:45.681Zsys 0.159496114
1852024-02-07T22:35:45.681Ztrap 0.000191873
1862024-02-07T22:35:45.681Ztflt 0.000000000
1872024-02-07T22:35:45.681Zdflt 0.000000000
1882024-02-07T22:35:45.681Zkflt 0.000000000
1892024-02-07T22:35:45.682Zlock 0.000000000
1902024-02-07T22:35:45.682Zslp 0.000019520
1912024-02-07T22:35:45.682Zlat 0.000022980
1922024-02-07T22:35:45.682Zstop 0.000129042
1932024-02-07T22:35:45.682Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1942024-02-07T22:35:45.682Z+ chmod a+x tests/bootstrap
1952024-02-07T22:35:45.682Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1962024-02-07T22:35:45.682Z+ ptime -m gunzip
1972024-02-07T22:35:45.682Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-398b34350dfa5341
1982024-02-07T22:35:48.245Z
1992024-02-07T22:35:48.272Zreal 2.577473276
2002024-02-07T22:35:48.272Zuser 2.397703124
2012024-02-07T22:35:48.272Zsys 0.179320662
2022024-02-07T22:35:48.272Ztrap 0.000206226
2032024-02-07T22:35:48.272Ztflt 0.000000000
2042024-02-07T22:35:48.272Zdflt 0.000000000
2052024-02-07T22:35:48.272Zkflt 0.000000000
2062024-02-07T22:35:48.272Zlock 0.000000000
2072024-02-07T22:35:48.272Zslp 0.000023520
2082024-02-07T22:35:48.272Zlat 0.000032550
2092024-02-07T22:35:48.272Zstop 0.000128152
2102024-02-07T22:35:48.272Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-398b34350dfa5341
2112024-02-07T22:35:48.272Z+ chmod a+x tests/end_to_end_tests-398b34350dfa5341
2122024-02-07T22:35:48.273Z+ bmat address ls
2132024-02-07T22:35:48.273ZNAME CIDR FIRST LAST COUNT
2142024-02-07T22:35:48.273Zextra 10.151.2.0/24 10.151.2.164 10.151.2.227 64
2152024-02-07T22:35:48.273Z++ bmat address ls -f extra -Ho first,last,count
2162024-02-07T22:35:48.273Z+ set -- 10.151.2.164 10.151.2.227 64
2172024-02-07T22:35:48.273Z+ EXTRA_IP_START=10.151.2.164
2182024-02-07T22:35:48.273Z+ EXTRA_IP_END=10.151.2.227
2192024-02-07T22:35:48.273Z+ EXTRA_IP_COUNT=64
2202024-02-07T22:35:48.273Z+ (( EXTRA_IP_COUNT >= 32 ))
2212024-02-07T22:35:48.273Z+ EXTRA_IP_BASE=10.151.2
2222024-02-07T22:35:48.273Z+ EXTRA_IP_FOCTET=164
2232024-02-07T22:35:48.273Z+ EXTRA_IP_LOCTET=227
2242024-02-07T22:35:48.273Z+ SERVICE_IP_POOL_START=10.151.2.164
2252024-02-07T22:35:48.273Z+ SERVICE_IP_POOL_END=10.151.2.173
2262024-02-07T22:35:48.273Z+ DNS_IP1=10.151.2.164
2272024-02-07T22:35:48.273Z+ DNS_IP2=10.151.2.165
2282024-02-07T22:35:48.274Z+ UPLINK_IP=10.151.2.174
2292024-02-07T22:35:48.274Z+ IPPOOL_START=10.151.2.175
2302024-02-07T22:35:48.274Z+ IPPOOL_END=10.151.2.227
2312024-02-07T22:35:48.298Z++ ipadm show-addr -po type,addr
2322024-02-07T22:35:48.298Z++ awk '-F[:/]' '$1 == "dhcp" {print $2}'
2332024-02-07T22:35:48.298Z+ GATEWAY_IP=10.151.2.100
2342024-02-07T22:35:48.298Z+ [[ -n 10.151.2.100 ]]
2352024-02-07T22:35:48.298Z+ ping -s 10.151.2.100 56 1
2362024-02-07T22:35:48.298ZPING 10.151.2.100: 56 data bytes
2372024-02-07T22:35:48.298Z64 bytes from 10.151.2.100: icmp_seq=0. time=0.011 ms
2382024-02-07T22:35:48.299Z
2392024-02-07T22:35:48.299Z----10.151.2.100 PING Statistics----
2402024-02-07T22:35:48.299Z1 packets transmitted, 1 packets received, 0% packet loss
2412024-02-07T22:35:48.299Zround-trip (ms) min/avg/max/stddev = 0.011/0.011/0.011/-nan
2422024-02-07T22:35:48.299Z++ arp -an
2432024-02-07T22:35:48.299Z++ awk -vgw=10.151.2.100 '$2 == gw {print $NF}'
2442024-02-07T22:35:48.299Z+ GATEWAY_MAC=18:c0:4d:0d:a0:2a
2452024-02-07T22:35:48.299Z+ routeadm -e ipv4-forwarding -u
2462024-02-07T22:35:48.407Z+ PXA_START=10.151.2.164
2472024-02-07T22:35:48.407Z+ PXA_END=10.151.2.227
2482024-02-07T22:35:48.433Z+ export GATEWAY_IP GATEWAY_MAC PXA_START PXA_END
2492024-02-07T22:35:48.433Z+ pfexec zpool create -f scratch c1t1d0 c2t1d0
2502024-02-07T22:35:48.614Z+ ZPOOL_VDEV_DIR=/scratch
2512024-02-07T22:35:48.614Z+ ptime -m pfexec ./tools/create_virtual_hardware.sh
2522024-02-07T22:35:48.641Z+++ dirname ./tools/create_virtual_hardware.sh
2532024-02-07T22:35:48.641Z++ cd ./tools
2542024-02-07T22:35:48.641Z++ pwd
2552024-02-07T22:35:48.641Z+ SOURCE_DIR=/opt/oxide/work/tools
2562024-02-07T22:35:48.642Z+ OMICRON_TOP=/opt/oxide/work/tools/..
2572024-02-07T22:35:48.642Z+ SOFTNPU_MODE=zone
2582024-02-07T22:35:48.642Z+ . /opt/oxide/work/tools/virtual_hardware.sh
2592024-02-07T22:35:48.642Z++ MARKER=/etc/opt/oxide/NO_INSTALL
2602024-02-07T22:35:48.642Z++ [[ -f /etc/opt/oxide/NO_INSTALL ]]
2612024-02-07T22:35:48.642Z++ dladm show-phys -p -o LINK
2622024-02-07T22:35:48.642Z++ head -1
2632024-02-07T22:35:48.642Z+ PHYSICAL_LINK=igb0
2642024-02-07T22:35:48.642Z+ echo 'Using igb0 as physical link'
2652024-02-07T22:35:48.643ZUsing igb0 as physical link
2662024-02-07T22:35:48.643Z+ ensure_run_as_root
2672024-02-07T22:35:48.643Z++ id -u
2682024-02-07T22:35:48.643Z+ [[ 0 -ne 0 ]]
2692024-02-07T22:35:48.643Z+ ensure_zpools
2702024-02-07T22:35:48.643Z+ ZPOOL_TYPES=('oxp_' 'oxi_')
2712024-02-07T22:35:48.643Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
2722024-02-07T22:35:48.643Z+ readarray -t ZPOOLS
2732024-02-07T22:35:48.668Z++ grep '"oxp_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
2742024-02-07T22:35:48.668Z++ sed 's/[ ",]//g'
2752024-02-07T22:35:48.668Z+ for ZPOOL in "${ZPOOLS[@]}"
2762024-02-07T22:35:48.668Z+ echo 'Zpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]'
2772024-02-07T22:35:48.668ZZpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]
2782024-02-07T22:35:48.668Z+ VDEV_PATH=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2792024-02-07T22:35:48.668Z+ [[ -f /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
2802024-02-07T22:35:48.668Z+ dd if=/dev/zero of=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
2812024-02-07T22:35:48.668Z0+0 records in
2822024-02-07T22:35:48.668Z0+0 records out
2832024-02-07T22:35:48.668Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
2842024-02-07T22:35:48.668Z+ success 'ZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
2852024-02-07T22:35:48.668Z+ set +x
2862024-02-07T22:35:48.669ZZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
2872024-02-07T22:35:48.669Z++ zpool list -o name
2882024-02-07T22:35:48.669Z++ grep oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
2892024-02-07T22:35:48.669Z+ [[ -z '' ]]
2902024-02-07T22:35:48.669Z+ zpool create -f oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2912024-02-07T22:35:48.699Z+ success 'ZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
2922024-02-07T22:35:48.725Z+ set +x
2932024-02-07T22:35:48.725ZZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists
2942024-02-07T22:35:48.725Z+ for ZPOOL in "${ZPOOLS[@]}"
2952024-02-07T22:35:48.725Z+ echo 'Zpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
2962024-02-07T22:35:48.725ZZpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]
2972024-02-07T22:35:48.725Z+ VDEV_PATH=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
2982024-02-07T22:35:48.725Z+ [[ -f /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
2992024-02-07T22:35:48.725Z+ dd if=/dev/zero of=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3002024-02-07T22:35:48.726Z0+0 records in
3012024-02-07T22:35:48.726Z0+0 records out
3022024-02-07T22:35:48.726Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3032024-02-07T22:35:48.726Z+ success 'ZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3042024-02-07T22:35:48.726Z+ set +x
3052024-02-07T22:35:48.726ZZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3062024-02-07T22:35:48.727Z++ zpool list -o name
3072024-02-07T22:35:48.727Z++ grep oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
3082024-02-07T22:35:48.727Z+ [[ -z '' ]]
3092024-02-07T22:35:48.727Z+ zpool create -f oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3102024-02-07T22:35:48.753Z+ success 'ZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3112024-02-07T22:35:48.753Z+ set +x
3122024-02-07T22:35:48.753ZZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3132024-02-07T22:35:48.753Z+ for ZPOOL in "${ZPOOLS[@]}"
3142024-02-07T22:35:48.753Z+ echo 'Zpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3152024-02-07T22:35:48.753ZZpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]
3162024-02-07T22:35:48.753Z+ VDEV_PATH=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3172024-02-07T22:35:48.753Z+ [[ -f /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3182024-02-07T22:35:48.754Z+ dd if=/dev/zero of=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3192024-02-07T22:35:48.754Z0+0 records in
3202024-02-07T22:35:48.754Z0+0 records out
3212024-02-07T22:35:48.754Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3222024-02-07T22:35:48.754Z+ success 'ZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3232024-02-07T22:35:48.754Z+ set +x
3242024-02-07T22:35:48.754ZZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3252024-02-07T22:35:48.754Z++ zpool list -o name
3262024-02-07T22:35:48.754Z++ grep oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
3272024-02-07T22:35:48.778Z+ [[ -z '' ]]
3282024-02-07T22:35:48.778Z+ zpool create -f oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3292024-02-07T22:35:48.804Z+ success 'ZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3302024-02-07T22:35:48.804Z+ set +x
3312024-02-07T22:35:48.804ZZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3322024-02-07T22:35:48.804Z+ for ZPOOL in "${ZPOOLS[@]}"
3332024-02-07T22:35:48.804Z+ echo 'Zpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3342024-02-07T22:35:48.804ZZpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]
3352024-02-07T22:35:48.804Z+ VDEV_PATH=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3362024-02-07T22:35:48.805Z+ [[ -f /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3372024-02-07T22:35:48.805Z+ dd if=/dev/zero of=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3382024-02-07T22:35:48.805Z0+0 records in
3392024-02-07T22:35:48.805Z0+0 records out
3402024-02-07T22:35:48.805Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3412024-02-07T22:35:48.805Z+ success 'ZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3422024-02-07T22:35:48.805Z+ set +x
3432024-02-07T22:35:48.806ZZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3442024-02-07T22:35:48.806Z++ zpool list -o name
3452024-02-07T22:35:48.806Z++ grep oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03
3462024-02-07T22:35:48.806Z+ [[ -z '' ]]
3472024-02-07T22:35:48.806Z+ zpool create -f oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3482024-02-07T22:35:48.830Z+ success 'ZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3492024-02-07T22:35:48.830Z+ set +x
3502024-02-07T22:35:48.830ZZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3512024-02-07T22:35:48.830Z+ for ZPOOL in "${ZPOOLS[@]}"
3522024-02-07T22:35:48.831Z+ echo 'Zpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]'
3532024-02-07T22:35:48.831ZZpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]
3542024-02-07T22:35:48.831Z+ VDEV_PATH=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3552024-02-07T22:35:48.831Z+ [[ -f /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
3562024-02-07T22:35:48.831Z+ dd if=/dev/zero of=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G
3572024-02-07T22:35:48.831Z0+0 records in
3582024-02-07T22:35:48.831Z0+0 records out
3592024-02-07T22:35:48.831Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
3602024-02-07T22:35:48.831Z+ success 'ZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
3612024-02-07T22:35:48.831Z+ set +x
3622024-02-07T22:35:48.831ZZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
3632024-02-07T22:35:48.831Z++ zpool list -o name
3642024-02-07T22:35:48.831Z++ grep oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03
3652024-02-07T22:35:48.868Z+ [[ -z '' ]]
3662024-02-07T22:35:48.868Z+ zpool create -f oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
3672024-02-07T22:35:48.892Z+ success 'ZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
3682024-02-07T22:35:48.892Z+ set +x
3692024-02-07T22:35:48.892ZZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
3702024-02-07T22:35:48.892Z+ for ZPOOL in "${ZPOOLS[@]}"
3712024-02-07T22:35:48.892Z+ echo 'Zpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]'
3722024-02-07T22:35:48.892ZZpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]
3732024-02-07T22:35:48.892Z+ VDEV_PATH=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3742024-02-07T22:35:48.892Z+ [[ -f /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev ]]
3752024-02-07T22:35:48.893Z+ dd if=/dev/zero of=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev bs=1 count=0 seek=20G
3762024-02-07T22:35:48.893Z0+0 records in
3772024-02-07T22:35:48.893Z0+0 records out
3782024-02-07T22:35:48.893Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3792024-02-07T22:35:48.893Z+ success 'ZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists'
3802024-02-07T22:35:48.893Z+ set +x
3812024-02-07T22:35:48.893ZZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists
3822024-02-07T22:35:48.893Z++ zpool list -o name
3832024-02-07T22:35:48.893Z++ grep oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930
3842024-02-07T22:35:48.893Z+ [[ -z '' ]]
3852024-02-07T22:35:48.893Z+ zpool create -f oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev
3862024-02-07T22:35:48.925Z+ success 'ZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists'
3872024-02-07T22:35:48.925Z+ set +x
3882024-02-07T22:35:48.925ZZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists
3892024-02-07T22:35:48.925Z+ for ZPOOL in "${ZPOOLS[@]}"
3902024-02-07T22:35:48.925Z+ echo 'Zpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]'
3912024-02-07T22:35:48.926ZZpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]
3922024-02-07T22:35:48.926Z+ VDEV_PATH=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
3932024-02-07T22:35:48.926Z+ [[ -f /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev ]]
3942024-02-07T22:35:48.926Z+ dd if=/dev/zero of=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev bs=1 count=0 seek=20G
3952024-02-07T22:35:48.926Z0+0 records in
3962024-02-07T22:35:48.926Z0+0 records out
3972024-02-07T22:35:48.926Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3982024-02-07T22:35:48.927Z+ success 'ZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists'
3992024-02-07T22:35:48.927Z+ set +x
4002024-02-07T22:35:48.927ZZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists
4012024-02-07T22:35:48.927Z++ zpool list -o name
4022024-02-07T22:35:48.927Z++ grep oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60
4032024-02-07T22:35:48.956Z+ [[ -z '' ]]
4042024-02-07T22:35:48.956Z+ zpool create -f oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev
4052024-02-07T22:35:48.957Z+ success 'ZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists'
4062024-02-07T22:35:48.957Z+ set +x
4072024-02-07T22:35:48.957ZZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists
4082024-02-07T22:35:48.957Z+ for ZPOOL in "${ZPOOLS[@]}"
4092024-02-07T22:35:48.957Z+ echo 'Zpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]'
4102024-02-07T22:35:48.957Z+ VDEV_PATH=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4112024-02-07T22:35:48.957Z+ [[ -f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev ]]
4122024-02-07T22:35:48.957Z+ dd if=/dev/zero of=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev bs=1 count=0 seek=20G
4132024-02-07T22:35:48.957ZZpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]
4142024-02-07T22:35:48.957Z0+0 records in
4152024-02-07T22:35:48.957Z0+0 records out
4162024-02-07T22:35:48.981Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4172024-02-07T22:35:48.981Z+ success 'ZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists'
4182024-02-07T22:35:48.982Z+ set +x
4192024-02-07T22:35:48.982ZZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists
4202024-02-07T22:35:48.982Z++ zpool list -o name
4212024-02-07T22:35:48.982Z++ grep oxp_31bd71cd-4736-4a12-a387-9b74b050396f
4222024-02-07T22:35:48.982Z+ [[ -z '' ]]
4232024-02-07T22:35:48.982Z+ zpool create -f oxp_31bd71cd-4736-4a12-a387-9b74b050396f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev
4242024-02-07T22:35:49.011Z+ success 'ZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists'
4252024-02-07T22:35:49.012Z+ set +x
4262024-02-07T22:35:49.012ZZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists
4272024-02-07T22:35:49.012Z+ for ZPOOL in "${ZPOOLS[@]}"
4282024-02-07T22:35:49.012Z+ echo 'Zpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]'
4292024-02-07T22:35:49.012ZZpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]
4302024-02-07T22:35:49.012Z+ VDEV_PATH=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4312024-02-07T22:35:49.012Z+ [[ -f /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev ]]
4322024-02-07T22:35:49.012Z+ dd if=/dev/zero of=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev bs=1 count=0 seek=20G
4332024-02-07T22:35:49.012Z0+0 records in
4342024-02-07T22:35:49.012Z0+0 records out
4352024-02-07T22:35:49.012Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4362024-02-07T22:35:49.012Z+ success 'ZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists'
4372024-02-07T22:35:49.012Z+ set +x
4382024-02-07T22:35:49.012ZZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists
4392024-02-07T22:35:49.012Z++ zpool list -o name
4402024-02-07T22:35:49.013Z++ grep oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7
4412024-02-07T22:35:49.041Z+ [[ -z '' ]]
4422024-02-07T22:35:49.041Z+ zpool create -f oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev
4432024-02-07T22:35:49.041Z+ success 'ZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists'
4442024-02-07T22:35:49.041Z+ set +x
4452024-02-07T22:35:49.041ZZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists
4462024-02-07T22:35:49.041Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
4472024-02-07T22:35:49.041Z+ readarray -t ZPOOLS
4482024-02-07T22:35:49.071Z++ grep '"oxi_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
4492024-02-07T22:35:49.071Z++ sed 's/[ ",]//g'
4502024-02-07T22:35:49.071Z+ for ZPOOL in "${ZPOOLS[@]}"
4512024-02-07T22:35:49.072Z+ echo 'Zpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4522024-02-07T22:35:49.072ZZpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]
4532024-02-07T22:35:49.072Z+ VDEV_PATH=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4542024-02-07T22:35:49.072Z+ [[ -f /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4552024-02-07T22:35:49.072Z+ dd if=/dev/zero of=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4562024-02-07T22:35:49.072Z0+0 records in
4572024-02-07T22:35:49.072Z0+0 records out
4582024-02-07T22:35:49.072Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
4592024-02-07T22:35:49.072Z+ success 'ZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4602024-02-07T22:35:49.072Z+ set +x
4612024-02-07T22:35:49.072ZZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4622024-02-07T22:35:49.072Z++ zpool list -o name
4632024-02-07T22:35:49.072Z++ grep oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b
4642024-02-07T22:35:49.072Z+ [[ -z '' ]]
4652024-02-07T22:35:49.072Z+ zpool create -f oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4662024-02-07T22:35:49.101Z+ success 'ZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4672024-02-07T22:35:49.101Z+ set +x
4682024-02-07T22:35:49.101ZZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4692024-02-07T22:35:49.101Z+ for ZPOOL in "${ZPOOLS[@]}"
4702024-02-07T22:35:49.101Z+ echo 'Zpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]'
4712024-02-07T22:35:49.101ZZpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]
4722024-02-07T22:35:49.101Z+ VDEV_PATH=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4732024-02-07T22:35:49.101Z+ [[ -f /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
4742024-02-07T22:35:49.101Z+ dd if=/dev/zero of=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G
4752024-02-07T22:35:49.101Z0+0 records in
4762024-02-07T22:35:49.101Z0+0 records out
4772024-02-07T22:35:49.101Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
4782024-02-07T22:35:49.101Z+ success 'ZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
4792024-02-07T22:35:49.101Z+ set +x
4802024-02-07T22:35:49.102ZZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
4812024-02-07T22:35:49.102Z++ zpool list -o name
4822024-02-07T22:35:49.102Z++ grep oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b
4832024-02-07T22:35:49.129Z+ [[ -z '' ]]
4842024-02-07T22:35:49.129Z+ zpool create -f oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
4852024-02-07T22:35:49.154Z+ success 'ZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
4862024-02-07T22:35:49.154Z+ set +x
4872024-02-07T22:35:49.154ZZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists
4882024-02-07T22:35:49.154Z+ [[ zone == \z\o\n\e ]]
4892024-02-07T22:35:49.154Z+ ensure_simulated_links igb0
4902024-02-07T22:35:49.154Z+ local PHYSICAL_LINK=igb0
4912024-02-07T22:35:49.154Z+ INDICES=("0" "1")
4922024-02-07T22:35:49.154Z+ for I in "${INDICES[@]}"
4932024-02-07T22:35:49.154Z++ get_simnet_name_if_exists net0
4942024-02-07T22:35:49.154Z++ dladm show-simnet -p -o LINK net0
4952024-02-07T22:35:49.154Z++ echo ''
4962024-02-07T22:35:49.154Z+ [[ -z '' ]]
4972024-02-07T22:35:49.155Z+ dladm create-simnet -t net0
4982024-02-07T22:35:49.155Z+ dladm create-simnet -t sc0_0
4992024-02-07T22:35:49.155Z+ dladm modify-simnet -t -p net0 sc0_0
5002024-02-07T22:35:49.155Z+ dladm set-linkprop -p mtu=9000 sc0_0
5012024-02-07T22:35:49.155Z+ success 'Simnet net0/sc0_0 exists'
5022024-02-07T22:35:49.155Z+ set +x
5032024-02-07T22:35:49.155ZSimnet net0/sc0_0 exists
5042024-02-07T22:35:49.155Z+ for I in "${INDICES[@]}"
5052024-02-07T22:35:49.179Z++ get_simnet_name_if_exists net1
5062024-02-07T22:35:49.180Z++ dladm show-simnet -p -o LINK net1
5072024-02-07T22:35:49.180Z++ echo ''
5082024-02-07T22:35:49.180Z+ [[ -z '' ]]
5092024-02-07T22:35:49.180Z+ dladm create-simnet -t net1
5102024-02-07T22:35:49.180Z+ dladm create-simnet -t sc1_0
5112024-02-07T22:35:49.180Z+ dladm modify-simnet -t -p net1 sc1_0
5122024-02-07T22:35:49.180Z+ dladm set-linkprop -p mtu=9000 sc1_0
5132024-02-07T22:35:49.180Z+ success 'Simnet net1/sc1_0 exists'
5142024-02-07T22:35:49.180Z+ set +x
5152024-02-07T22:35:49.180ZSimnet net1/sc1_0 exists
5162024-02-07T22:35:49.180Z++ get_vnic_name_if_exists sc0_1
5172024-02-07T22:35:49.180Z++ dladm show-vnic -p -o LINK sc0_1
5182024-02-07T22:35:49.209Z++ echo ''
5192024-02-07T22:35:49.209Z+ [[ -z '' ]]
5202024-02-07T22:35:49.209Z+ dladm create-vnic -t sc0_1 -l igb0 -m a8:e1:de:01:70:1d
5212024-02-07T22:35:49.209Z+ [[ -v PROMISC_FILT_OFF ]]
5222024-02-07T22:35:49.209Z+ success 'Vnic sc0_1 exists'
5232024-02-07T22:35:49.209Z+ set +x
5242024-02-07T22:35:49.209ZVnic sc0_1 exists
5252024-02-07T22:35:49.209Z+ warn_if_no_proxy_arp
5262024-02-07T22:35:49.209Z+ [[ -v PXA_START ]]
5272024-02-07T22:35:49.209Z+ [[ -v PXA_END ]]
5282024-02-07T22:35:49.209Z+ ensure_softnpu_zone
5292024-02-07T22:35:49.209Z+ zoneadm list
5302024-02-07T22:35:49.209Z+ grep -q sidecar_softnpu
5312024-02-07T22:35:49.209Z+ [[ -f out/npuzone/npuzone ]]
5322024-02-07T22:35:49.210Z+ out/npuzone/npuzone create sidecar --omicron-zone --ports sc0_0,tfportrear0_0 --ports sc0_1,tfportqsfp0_0 --sidecar-lite-commit e3ea4b495ba0a71801ded0776ae4bbd31df57e26 --softnpu-commit dbab082dfa89da5db5ca2325c257089d2f130092
5332024-02-07T22:35:49.234Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/softnpu/image/dbab082dfa89da5db5ca2325c257089d2f130092/softnpu.sha256.txt
5342024-02-07T22:35:56.104Zdownloading softnpu ...done
5352024-02-07T22:35:56.128Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/libsidecar_lite.so.sha256.txt
5362024-02-07T22:36:02.358Zdownloading asic_program.so ...done
5372024-02-07T22:36:02.391Zremote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/scadm.sha256.txt
 
5382024-02-07T22:36:04.396ZError: No response to keepalive - Terminating session
 
5392024-02-07T22:36:09.654Zdownloading scadm ...done
5402024-02-07T22:36:09.681Zcreating port tfportrear0_0 for sc0_0
5412024-02-07T22:36:09.681Zcreating port tfportqsfp0_0 for sc0_1
5422024-02-07T22:36:09.682Zsoftnpu zone setup
5432024-02-07T22:36:09.682Zconfigure zone
5442024-02-07T22:36:09.711Zinstall zone
5452024-02-07T22:36:10.730Zboot zone
5462024-02-07T22:36:13.832Z[sidecar_softnpu] svcs milestone/network
5472024-02-07T22:36:13.858ZSTATE STIME FMRI
5482024-02-07T22:36:13.858Zonline 22:36:11 svc:/milestone/network:default
5492024-02-07T22:36:13.858Z[sidecar_softnpu] /softnpu/npu start
5502024-02-07T22:36:13.885Z
5512024-02-07T22:36:13.909Z+ /opt/oxide/work/tools/scrimlet/softnpu-init.sh
5522024-02-07T22:36:13.909Z+ GATEWAY_IP=10.151.2.100
5532024-02-07T22:36:13.909Z+ echo 'Using 10.151.2.100 as gateway ip'
5542024-02-07T22:36:13.909ZUsing 10.151.2.100 as gateway ip
5552024-02-07T22:36:13.909Z+ [[ ! -v GATEWAY_MAC ]]
5562024-02-07T22:36:13.910Z+ GATEWAY_MAC=18:c0:4d:0d:a0:2a
5572024-02-07T22:36:13.910Z++ grep -c -E '^([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}$' /dev/fd/63
5582024-02-07T22:36:13.910Z+++ echo 18:c0:4d:0d:a0:2a
5592024-02-07T22:36:13.910Z+ COUNT=1
5602024-02-07T22:36:13.910Z+ [[ 1 -ne 1 ]]
5612024-02-07T22:36:13.910Z+ echo 'Using 18:c0:4d:0d:a0:2a as gateway mac'
5622024-02-07T22:36:13.910ZUsing 18:c0:4d:0d:a0:2a as gateway mac
5632024-02-07T22:36:13.910Z+ z_scadm add-arp-entry 10.151.2.100 18:c0:4d:0d:a0:2a
5642024-02-07T22:36:13.910Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-arp-entry 10.151.2.100 18:c0:4d:0d:a0:2a
5652024-02-07T22:36:13.935Z+ PXA_MAC=a8:e1:de:01:70:1d
5662024-02-07T22:36:13.935Z+ [[ -v PXA_START ]]
5672024-02-07T22:36:13.935Z+ z_scadm add-proxy-arp 10.151.2.164 10.151.2.227 a8:e1:de:01:70:1d
5682024-02-07T22:36:13.935Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-proxy-arp 10.151.2.164 10.151.2.227 a8:e1:de:01:70:1d
5692024-02-07T22:36:13.960Z+ z_scadm dump-state
5702024-02-07T22:36:13.960Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
5712024-02-07T22:36:14.114Zlocal v6:
5722024-02-07T22:36:14.114Zlocal v4:
5732024-02-07T22:36:14.115Zrouter v6:
5742024-02-07T22:36:14.115Zrouter v4_idx:
5752024-02-07T22:36:14.115Zrouter v4_routes:
5762024-02-07T22:36:14.115Zresolver v4:
5772024-02-07T22:36:14.115Z10.151.2.100 -> 18:c0:4d:0d:a0:2a
5782024-02-07T22:36:14.115Zresolver v6:
5792024-02-07T22:36:14.115Znat v4:
5802024-02-07T22:36:14.115Znat_v6:
5812024-02-07T22:36:14.115Zport_mac:
5822024-02-07T22:36:14.116Zproxy arp:
5832024-02-07T22:36:14.116Z10.151.2.164/10.151.2.227: a8:e1:de:01:70:1d
5842024-02-07T22:36:14.142Z+ success 'softnpu zone exists'
5852024-02-07T22:36:14.142Z+ set +x
5862024-02-07T22:36:14.142Zsoftnpu zone exists
5872024-02-07T22:36:14.143Z
5882024-02-07T22:36:14.143Zreal 25.372445568
5892024-02-07T22:36:14.143Zuser 0.673721060
5902024-02-07T22:36:14.143Zsys 1.504368270
5912024-02-07T22:36:14.143Ztrap 0.000081797
5922024-02-07T22:36:14.143Ztflt 0.000133405
5932024-02-07T22:36:14.143Zdflt 0.001359375
5942024-02-07T22:36:14.144Zkflt 0.000003820
5952024-02-07T22:36:14.144Zlock 19:43.260748542
5962024-02-07T22:36:14.144Zslp 1:15.883180464
5972024-02-07T22:36:14.144Zlat 0.066499846
5982024-02-07T22:36:14.144Zstop 0.329576788
5992024-02-07T22:36:14.144Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml
6002024-02-07T22:36:14.144Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
6012024-02-07T22:36:14.144Z+ SILO_NAME=recovery
6022024-02-07T22:36:14.145Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
6032024-02-07T22:36:14.145Z+ EXTERNAL_DNS_DOMAIN=oxide.test
6042024-02-07T22:36:14.145Z+ sed -i~ '
6052024-02-07T22:36:14.145Z /^external_dns_ips/c\
6062024-02-07T22:36:14.145Zexternal_dns_ips = [ "10.151.2.164", "10.151.2.165" ]
6072024-02-07T22:36:14.145Z /^\[\[internal_services_ip_pool_ranges/,/^$/ {
6082024-02-07T22:36:14.145Z /^first/c\
6092024-02-07T22:36:14.146Zfirst = "10.151.2.164"
6102024-02-07T22:36:14.146Z /^last/c\
6112024-02-07T22:36:14.146Zlast = "10.151.2.173"
6122024-02-07T22:36:14.146Z }
6132024-02-07T22:36:14.146Z /^\[rack_network_config/,/^$/ {
6142024-02-07T22:36:14.146Z /^infra_ip_first/c\
6152024-02-07T22:36:14.146Zinfra_ip_first = "10.151.2.174"
6162024-02-07T22:36:14.146Z /^infra_ip_last/c\
6172024-02-07T22:36:14.147Zinfra_ip_last = "10.151.2.174"
6182024-02-07T22:36:14.147Z }
6192024-02-07T22:36:14.147Z /^\[\[rack_network_config.ports/,/^$/ {
6202024-02-07T22:36:14.147Z /^routes/c\
6212024-02-07T22:36:14.147Zroutes = \[{nexthop = "10.151.2.100", destination = "0.0.0.0/0"}\]
6222024-02-07T22:36:14.147Z /^addresses/c\
6232024-02-07T22:36:14.147Zaddresses = \["10.151.2.174/24"\]
6242024-02-07T22:36:14.148Z }
6252024-02-07T22:36:14.148Z' pkg/config-rss.toml
6262024-02-07T22:36:14.148Z+ diff -u pkg/config-rss.toml~ pkg/config-rss.toml
6272024-02-07T22:36:14.148Z--- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006
6282024-02-07T22:36:14.148Z+++ pkg/config-rss.toml Wed Feb 7 22:36:13 2024
6292024-02-07T22:36:14.148Z@@ -41,7 +41,7 @@
6302024-02-07T22:36:14.148Z # the DNS domain delegated to the rack by the customer. Each of these addresses
6312024-02-07T22:36:14.148Z # must be contained in one of the "internal services" IP Pool ranges listed
6322024-02-07T22:36:14.149Z # below.
6332024-02-07T22:36:14.149Z-external_dns_ips = [ "192.168.1.20", "192.168.1.21" ]
6342024-02-07T22:36:14.149Z+external_dns_ips = [ "10.151.2.164", "10.151.2.165" ]
6352024-02-07T22:36:14.149Z
6362024-02-07T22:36:14.149Z # Initial TLS certificates for the external API
6372024-02-07T22:36:14.149Z #
6382024-02-07T22:36:14.149Z@@ -77,8 +77,8 @@
6392024-02-07T22:36:14.150Z #
6402024-02-07T22:36:14.150Z # For more on this and what to put here, see docs/how-to-run.adoc.
6412024-02-07T22:36:14.150Z [[internal_services_ip_pool_ranges]]
6422024-02-07T22:36:14.150Z-first = "192.168.1.20"
6432024-02-07T22:36:14.150Z-last = "192.168.1.29"
6442024-02-07T22:36:14.150Z+first = "10.151.2.164"
6452024-02-07T22:36:14.150Z+last = "10.151.2.173"
6462024-02-07T22:36:14.151Z
6472024-02-07T22:36:14.151Z # TODO - this configuration is subject to change going forward. Ultimately these
6482024-02-07T22:36:14.151Z # parameters should be provided to the control plane via wicket, but we need to
6492024-02-07T22:36:14.151Z@@ -92,8 +92,8 @@
6502024-02-07T22:36:14.151Z # A range of IP addresses used by Boundary Services on the external network. In
6512024-02-07T22:36:14.151Z # a real system, these would be addresses of the uplink ports on the Sidecar.
6522024-02-07T22:36:14.151Z # With softnpu, only one address is used.
6532024-02-07T22:36:14.151Z-infra_ip_first = "192.168.1.30"
6542024-02-07T22:36:14.151Z-infra_ip_last = "192.168.1.30"
6552024-02-07T22:36:14.152Z+infra_ip_first = "10.151.2.174"
6562024-02-07T22:36:14.152Z+infra_ip_last = "10.151.2.174"
6572024-02-07T22:36:14.152Z
6582024-02-07T22:36:14.152Z # Configurations for BGP routers to run on the scrimlets.
6592024-02-07T22:36:14.152Z bgp = []
6602024-02-07T22:36:14.152Z@@ -101,9 +101,9 @@
6612024-02-07T22:36:14.152Z # You can configure multiple uplinks by repeating the following stanza
6622024-02-07T22:36:14.152Z [[rack_network_config.ports]]
6632024-02-07T22:36:14.152Z # Routes associated with this port.
6642024-02-07T22:36:14.152Z-routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
6652024-02-07T22:36:14.153Z+routes = [{nexthop = "10.151.2.100", destination = "0.0.0.0/0"}]
6662024-02-07T22:36:14.153Z # Addresses associated with this port.
6672024-02-07T22:36:14.153Z-addresses = ["192.168.1.30/24"]
6682024-02-07T22:36:14.153Z+addresses = ["10.151.2.174/24"]
6692024-02-07T22:36:14.153Z # Name of the uplink port. This should always be "qsfp0" when using softnpu.
6702024-02-07T22:36:14.153Z port = "qsfp0"
6712024-02-07T22:36:14.153Z # The speed of this port.
6722024-02-07T22:36:14.154Z+ true
6732024-02-07T22:36:14.154Z+ tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml
6742024-02-07T22:36:14.154Za pkg/config-rss.toml 7K
6752024-02-07T22:36:14.154Z+ rm -f pkg/config-rss.toml pkg/config-rss.toml~
6762024-02-07T22:36:14.154Z+ TLS_NAME=recovery.sys.oxide.test
6772024-02-07T22:36:14.154Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
 
6782024-02-07T22:36:14.319ZError: No response de-activating SOL payload
 
6792024-02-07T22:36:14.441Z...........+....+..+.......+.....+.......+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+...+.....+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+...+..........................................+.............+..+.+.........+........+...+...+............+...+.+.........+...........+....+.....+....+..+......+.......+...+...............+..............+...+.+......+........+.......+.........+.....+.......+..+...+...+.+........+..........+...+...+............+.....+.........+......+.......+...+.....+............+.+...........+.+.....+....+.....+.............+........+...+....+..+...+............+..........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6802024-02-07T22:36:14.604Z.+......................+...+..+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*........+..........+......+.....+.+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+........+..........+...........+.+........+.+..............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6812024-02-07T22:36:14.628Z-----
6822024-02-07T22:36:14.628Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6832024-02-07T22:36:14.654Za pkg/initial-tls-cert.pem 2K
6842024-02-07T22:36:14.654Za pkg/initial-tls-key.pem 4K
6852024-02-07T22:36:14.654Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
6862024-02-07T22:36:14.654Z+ rmdir pkg
6872024-02-07T22:36:14.654Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
6882024-02-07T22:36:14.654Z+ pfexec mkdir -p /usr/oxide
6892024-02-07T22:36:14.654Z+ pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0
6902024-02-07T22:36:15.318Z+ pfexec chmod +x /usr/oxide/oxide
6912024-02-07T22:36:15.344Z+ curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw
 
6922024-02-07T22:36:15.680Z[SOL Session operational. Use ~? for help]
 
6932024-02-07T22:36:33.628Z+ OMICRON_NO_UNINSTALL=1
6942024-02-07T22:36:33.628Z+ ptime -m pfexec ./target/release/omicron-package -t test install
6952024-02-07T22:36:33.957ZLogging to: /opt/oxide/work/out/LOG
6962024-02-07T22:36:34.159Z
6972024-02-07T22:36:34.197Zreal 0.531492890
6982024-02-07T22:36:34.197Zuser 0.107985665
6992024-02-07T22:36:34.197Zsys 2.227880033
7002024-02-07T22:36:34.197Ztrap 0.000000609
7012024-02-07T22:36:34.197Ztflt 0.000375614
7022024-02-07T22:36:34.197Zdflt 0.000406205
7032024-02-07T22:36:34.197Zkflt 0.000000000
7042024-02-07T22:36:34.197Zlock 48.257348284
7052024-02-07T22:36:34.197Zslp 0.735342796
7062024-02-07T22:36:34.197Zlat 0.003017479
7072024-02-07T22:36:34.198Zstop 0.195774905
7082024-02-07T22:36:34.198Z+ retry=0
7092024-02-07T22:36:34.198Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7102024-02-07T22:36:34.198Z+ [[ 0 -gt 30 ]]
7112024-02-07T22:36:34.198Z+ sleep 1
7122024-02-07T22:36:35.168Z+ retry=1
7132024-02-07T22:36:35.168Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7142024-02-07T22:36:35.200Z+ [[ 1 -gt 30 ]]
7152024-02-07T22:36:35.201Z+ sleep 1
7162024-02-07T22:36:36.181Z+ retry=2
7172024-02-07T22:36:36.181Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7182024-02-07T22:36:36.207Z+ [[ 2 -gt 30 ]]
7192024-02-07T22:36:36.207Z+ sleep 1
7202024-02-07T22:36:37.198Z+ retry=3
7212024-02-07T22:36:37.198Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7222024-02-07T22:36:37.230Z+ [[ 3 -gt 30 ]]
7232024-02-07T22:36:37.230Z+ sleep 1
7242024-02-07T22:36:38.216Z+ retry=4
7252024-02-07T22:36:38.217Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
7262024-02-07T22:37:03.393Z+ pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state
7272024-02-07T22:37:03.423Zlocal v6:
7282024-02-07T22:37:03.423Zfdfa:3d1e:58d8:f51c::1
7292024-02-07T22:37:03.423Zfe80::8cf1:99ff:fe5f:1531
7302024-02-07T22:37:03.423Zlocal v4:
7312024-02-07T22:37:03.423Z10.151.2.174
7322024-02-07T22:37:03.423Zrouter v6:
7332024-02-07T22:37:03.423Zfd00:1122:3344:2::/64 -> fe80::dcbc:1aff:fe4c:2363 (1)
7342024-02-07T22:37:03.423Zfdb0:18c0:4d0d:a02a::/64 -> fe80::dcbc:1aff:fe4c:2363 (1)
7352024-02-07T22:37:03.424Zfd00:1122:3344:101::/64 -> fe80::dcbc:1aff:fe4c:2363 (1)
7362024-02-07T22:37:03.424Zfd00:1122:3344:1::/64 -> fe80::dcbc:1aff:fe4c:2363 (1)
7372024-02-07T22:37:03.424Zfd00:1122:3344:3::/64 -> fe80::dcbc:1aff:fe4c:2363 (1)
7382024-02-07T22:37:03.424Zrouter v4_idx:
7392024-02-07T22:37:03.424Z0.0.0.0/0 -> 0
7402024-02-07T22:37:03.424Zrouter v4_routes:
7412024-02-07T22:37:03.424Z0 -> 10.151.2.100 (2)
7422024-02-07T22:37:03.424Zresolver v4:
7432024-02-07T22:37:03.424Z10.151.2.100 -> 18:c0:4d:0d:a0:2a
7442024-02-07T22:37:03.425Z10.151.2.174 -> 5e:f2:4d:1e:c5:fb
7452024-02-07T22:37:03.425Zresolver v6:
7462024-02-07T22:37:03.425Zfe80::8cf1:99ff:fe5f:1531 -> 8e:f1:99:5f:15:31
7472024-02-07T22:37:03.425Zfe80::dcbc:1aff:fe4c:2363 -> de:bc:1a:4c:23:63
7482024-02-07T22:37:03.425Znat v4:
7492024-02-07T22:37:03.425Z10.151.2.169 0/16383 -> fd00:1122:3344:101::1 100/a8:40:25:ff:87:cd
7502024-02-07T22:37:03.425Znat_v6:
7512024-02-07T22:37:03.425Zport_mac:
7522024-02-07T22:37:03.425Z1: a8:40:25:38:0c:e8
7532024-02-07T22:37:03.426Z1000: a8:40:25:38:0c:e7
7542024-02-07T22:37:03.426Zproxy arp:
7552024-02-07T22:37:03.426Z10.151.2.164/10.151.2.227: a8:e1:de:01:70:1d
7562024-02-07T22:37:03.426Z+ retry=0
7572024-02-07T22:37:03.426Z+++ zoneadm list -n
7582024-02-07T22:37:03.464Z+++ grep oxz_ntp
7592024-02-07T22:37:03.464Z++ pfexec svcs -z oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60 -Hostate oxide/ntp
7602024-02-07T22:37:03.464Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7612024-02-07T22:37:03.465Z++ true
7622024-02-07T22:37:03.465Z+ [[ '' != online ]]
7632024-02-07T22:37:03.465Z+ [[ 0 -gt 60 ]]
7642024-02-07T22:37:03.465Z+ sleep 1
7652024-02-07T22:37:04.501Z+ retry=1
7662024-02-07T22:37:04.526Z+++ zoneadm list -n
7672024-02-07T22:37:04.526Z+++ grep oxz_ntp
7682024-02-07T22:37:04.526Z++ pfexec svcs -z oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60 -Hostate oxide/ntp
7692024-02-07T22:37:04.526Zsvcs: Pattern 'oxide/ntp' doesn't match any instances
7702024-02-07T22:37:04.526Z++ true
7712024-02-07T22:37:04.526Z+ [[ '' != online ]]
7722024-02-07T22:37:04.526Z+ [[ 1 -gt 60 ]]
7732024-02-07T22:37:04.526Z+ sleep 1
7742024-02-07T22:37:05.475Z+ retry=2
7752024-02-07T22:37:05.498Z+++ zoneadm list -n
7762024-02-07T22:37:05.498Z+++ grep oxz_ntp
7772024-02-07T22:37:05.498Z++ pfexec svcs -z oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60 -Hostate oxide/ntp
7782024-02-07T22:37:05.499Z+ [[ disabled != online ]]
7792024-02-07T22:37:05.499Z+ [[ 2 -gt 60 ]]
7802024-02-07T22:37:05.499Z+ sleep 1
7812024-02-07T22:37:06.497Z+ retry=3
7822024-02-07T22:37:06.521Z+++ zoneadm list -n
7832024-02-07T22:37:06.521Z+++ grep oxz_ntp
7842024-02-07T22:37:06.521Z++ pfexec svcs -z oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60 -Hostate oxide/ntp
7852024-02-07T22:37:06.521Z+ [[ disabled != online ]]
7862024-02-07T22:37:06.521Z+ [[ 3 -gt 60 ]]
7872024-02-07T22:37:06.522Z+ sleep 1
7882024-02-07T22:37:07.523Z+ retry=4
7892024-02-07T22:37:07.549Z+++ zoneadm list -n
7902024-02-07T22:37:07.549Z+++ grep oxz_ntp
7912024-02-07T22:37:07.549Z++ pfexec svcs -z oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60 -Hostate oxide/ntp
7922024-02-07T22:37:07.549Z+ [[ disabled != online ]]
7932024-02-07T22:37:07.549Z+ [[ 4 -gt 60 ]]
7942024-02-07T22:37:07.549Z+ sleep 1
7952024-02-07T22:37:08.954Z+ retry=5
7962024-02-07T22:37:08.990Z+++ zoneadm list -n
7972024-02-07T22:37:08.990Z+++ grep oxz_ntp
7982024-02-07T22:37:08.990Z++ pfexec svcs -z oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60 -Hostate oxide/ntp
7992024-02-07T22:37:08.990Z+ [[ disabled != online ]]
8002024-02-07T22:37:08.991Z+ [[ 5 -gt 60 ]]
8012024-02-07T22:37:08.991Z+ sleep 1
8022024-02-07T22:37:09.563Z+ retry=6
8032024-02-07T22:37:09.590Z+++ zoneadm list -n
8042024-02-07T22:37:09.590Z+++ grep oxz_ntp
8052024-02-07T22:37:09.590Z++ pfexec svcs -z oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60 -Hostate oxide/ntp
8062024-02-07T22:37:09.590Z+ [[ disabled != online ]]
8072024-02-07T22:37:09.590Z+ [[ 6 -gt 60 ]]
8082024-02-07T22:37:09.590Z+ sleep 1
8092024-02-07T22:37:10.586Z+ retry=7
8102024-02-07T22:37:10.612Z+++ zoneadm list -n
8112024-02-07T22:37:10.612Z+++ grep oxz_ntp
8122024-02-07T22:37:10.612Z++ pfexec svcs -z oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60 -Hostate oxide/ntp
8132024-02-07T22:37:10.612Z+ [[ disabled != online ]]
8142024-02-07T22:37:10.612Z+ [[ 7 -gt 60 ]]
8152024-02-07T22:37:10.612Z+ sleep 1
8162024-02-07T22:37:11.606Z+ retry=8
8172024-02-07T22:37:11.630Z+++ zoneadm list -n
8182024-02-07T22:37:11.631Z+++ grep oxz_ntp
8192024-02-07T22:37:11.631Z++ pfexec svcs -z oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60 -Hostate oxide/ntp
8202024-02-07T22:37:11.631Z+ [[ disabled != online ]]
8212024-02-07T22:37:11.631Z+ [[ 8 -gt 60 ]]
8222024-02-07T22:37:11.631Z+ sleep 1
8232024-02-07T22:37:12.627Z+ retry=9
8242024-02-07T22:37:12.654Z+++ zoneadm list -n
8252024-02-07T22:37:12.654Z+++ grep oxz_ntp
8262024-02-07T22:37:12.654Z++ pfexec svcs -z oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60 -Hostate oxide/ntp
8272024-02-07T22:37:12.654Z+ [[ disabled != online ]]
8282024-02-07T22:37:12.654Z+ [[ 9 -gt 60 ]]
8292024-02-07T22:37:12.654Z+ sleep 1
 
8302024-02-07T22:37:13.510ZFeb 7 22:37:12 sock ip: ip: joining multicasts failed (4) on vopte0 - will use link layer broadcasts for multicast
 
8312024-02-07T22:37:13.657Z+ retry=10
8322024-02-07T22:37:13.685Z+++ zoneadm list -n
8332024-02-07T22:37:13.685Z+++ grep oxz_ntp
8342024-02-07T22:37:13.685Z++ pfexec svcs -z oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60 -Hostate oxide/ntp
8352024-02-07T22:37:13.685Z+ [[ online != online ]]
8362024-02-07T22:37:13.685Z+ echo 'Waited for chrony: 10s'
8372024-02-07T22:37:13.686ZWaited for chrony: 10s
8382024-02-07T22:37:13.686Z+ export RUST_BACKTRACE=1
8392024-02-07T22:37:13.686Z+ RUST_BACKTRACE=1
8402024-02-07T22:37:13.686Z+ export E2E_TLS_CERT IPPOOL_START IPPOOL_END
8412024-02-07T22:37:13.686Z++ ./tests/bootstrap
8422024-02-07T22:37:13.711ZUsing external DNS server at 10.151.2.164:53
8432024-02-07T22:37:13.712Z2024-02-07 22:37:13.271969655 UTC: attempting to log into API
8442024-02-07T22:37:28.730Z2024-02-07 22:37:28.878279348 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8452024-02-07T22:37:29.733Z2024-02-07 22:37:29.880221716 UTC: attempting to log into API
8462024-02-07T22:37:44.765Z2024-02-07 22:37:44.912036788 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8472024-02-07T22:37:45.766Z2024-02-07 22:37:45.912955569 UTC: attempting to log into API
8482024-02-07T22:38:00.798Z2024-02-07 22:38:00.944821052 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8492024-02-07T22:38:01.798Z2024-02-07 22:38:01.945767763 UTC: attempting to log into API
8502024-02-07T22:38:16.832Z2024-02-07 22:38:16.976991336 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8512024-02-07T22:38:17.830Z2024-02-07 22:38:17.977857985 UTC: attempting to log into API
8522024-02-07T22:38:35.301Z2024-02-07 22:38:32.613328637 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8532024-02-07T22:38:35.301Z2024-02-07 22:38:33.617121162 UTC: attempting to log into API
8542024-02-07T22:38:48.894Z2024-02-07 22:38:48.689047266 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8552024-02-07T22:38:49.893Z2024-02-07 22:38:49.692560123 UTC: attempting to log into API
 
8562024-02-07T22:39:04.888ZError: No response to keepalive - Terminating session
 
8572024-02-07T22:39:04.926Z2024-02-07 22:39:04.764385722 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8582024-02-07T22:39:05.926Z2024-02-07 22:39:05.768145731 UTC: attempting to log into API
 
8592024-02-07T22:39:14.819ZError: No response de-activating SOL payload
8602024-02-07T22:39:15.937Z[SOL Session operational. Use ~? for help]
 
8612024-02-07T22:39:20.959Z2024-02-07 22:39:20.839548485 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8622024-02-07T22:39:21.961Z2024-02-07 22:39:21.843295847 UTC: attempting to log into API
8632024-02-07T22:39:36.991Z2024-02-07 22:39:36.915153169 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8642024-02-07T22:39:37.996Z2024-02-07 22:39:37.918943260 UTC: attempting to log into API
8652024-02-07T22:39:53.020Z2024-02-07 22:39:52.990480145 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8662024-02-07T22:39:54.027Z2024-02-07 22:39:53.995484885 UTC: attempting to log into API
8672024-02-07T22:40:09.077Z2024-02-07 22:40:09.066568473 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8682024-02-07T22:40:10.079Z2024-02-07 22:40:10.067504704 UTC: attempting to log into API
8692024-02-07T22:40:25.226Z2024-02-07 22:40:25.198360498 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8702024-02-07T22:40:26.217Z2024-02-07 22:40:26.199856058 UTC: attempting to log into API
8712024-02-07T22:40:41.260Z2024-02-07 22:40:41.248977520 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8722024-02-07T22:40:42.272Z2024-02-07 22:40:42.250033173 UTC: attempting to log into API
8732024-02-07T22:40:57.475Z2024-02-07 22:40:57.353697789 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8742024-02-07T22:40:58.390Z2024-02-07 22:40:58.377187501 UTC: attempting to log into API
8752024-02-07T22:41:13.500Z2024-02-07 22:41:13.489339179 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8762024-02-07T22:41:14.502Z2024-02-07 22:41:14.489790192 UTC: attempting to log into API
8772024-02-07T22:41:29.756Z2024-02-07 22:41:29.523647869 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8782024-02-07T22:41:30.638Z2024-02-07 22:41:30.524761043 UTC: attempting to log into API
8792024-02-07T22:41:45.748Z2024-02-07 22:41:45.558022029 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8802024-02-07T22:41:57.631Z2024-02-07 22:41:46.559263625 UTC: attempting to log into API
8812024-02-07T22:42:01.634Z2024-02-07 22:42:01.591484375 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out
8822024-02-07T22:42:02.835Z2024-02-07 22:42:02.592619069 UTC: attempting to log into API
8832024-02-07T22:42:06.365Z2024-02-07 22:42:02.648759217 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: tcp connect error: Connection refused (os error 146)
8842024-02-07T22:42:06.472Z2024-02-07 22:42:03.654179804 UTC: attempting to log into API
8852024-02-07T22:42:06.472Z2024-02-07 22:42:03.686573481 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: tcp connect error: Connection refused (os error 146)
8862024-02-07T22:42:06.472Z2024-02-07 22:42:04.687800677 UTC: attempting to log into API
8872024-02-07T22:42:06.472Z2024-02-07 22:42:05.499938758 UTC: login succeeded
8882024-02-07T22:42:06.472Zwaiting for nexus to come up...
8892024-02-07T22:42:06.472Zcreating IP pool... 10.151.2.175 - 10.151.2.227
8902024-02-07T22:42:06.472Zsetting up quotas...
8912024-02-07T22:42:06.472Zensuring datasets are ready...
8922024-02-07T22:42:07.873Zlet's roll.
8932024-02-07T22:42:07.918Z+ eval 'OXIDE_HOST=https://recovery.sys.oxide.test
8942024-02-07T22:42:07.918ZOXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.2.166
8952024-02-07T22:42:07.919ZOXIDE_TOKEN=oxide-token-def51726d63244f81b903bafded79f23becad98d'
8962024-02-07T22:42:07.919Z++ OXIDE_HOST=https://recovery.sys.oxide.test
8972024-02-07T22:42:07.919Z++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.2.166
8982024-02-07T22:42:07.920Z++ OXIDE_TOKEN=oxide-token-def51726d63244f81b903bafded79f23becad98d
8992024-02-07T22:42:07.920Z+ export OXIDE_HOST OXIDE_TOKEN
9002024-02-07T22:42:07.920Z+ retry=0
9012024-02-07T22:42:07.920Z+ curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem
9022024-02-07T22:42:08.008Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images'
9032024-02-07T22:42:08.312Z{"status":"ok"}success
9042024-02-07T22:42:08.313ZProject {
9052024-02-07T22:42:08.313Z description: "some images",
9062024-02-07T22:42:08.313Z id: 7067ac43-48ea-4390-a9e1-af3bdc344173,
9072024-02-07T22:42:08.314Z name: Name(
9082024-02-07T22:42:08.314Z "images",
9092024-02-07T22:42:08.314Z ),
9102024-02-07T22:42:08.314Z time_created: 2024-02-07T22:42:08.059389Z,
9112024-02-07T22:42:08.315Z time_modified: 2024-02-07T22:42:08.059389Z,
9122024-02-07T22:42:08.316Z}
9132024-02-07T22:42:08.365Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11
 
9142024-02-07T22:42:20.807ZError: No response to keepalive - Terminating session
9152024-02-07T22:42:30.740ZError: No response de-activating SOL payload
9162024-02-07T22:42:31.854Z[SOL Session operational. Use ~? for help]
9172024-02-07T22:45:20.591ZError: No response to keepalive - Terminating session
9182024-02-07T22:45:30.785ZError: No response de-activating SOL payload
9192024-02-07T22:45:32.265Z[SOL Session operational. Use ~? for help]
 
9202024-02-07T22:47:23.146Zdone!
9212024-02-07T22:47:23.241Z+ /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11
9222024-02-07T22:47:23.356Zsuccess
9232024-02-07T22:47:23.356ZImage {
9242024-02-07T22:47:23.357Z block_size: ByteCount(
9252024-02-07T22:47:23.384Z 512,
9262024-02-07T22:47:23.384Z ),
9272024-02-07T22:47:23.384Z description: "debian 11 original base image",
9282024-02-07T22:47:23.385Z digest: None,
9292024-02-07T22:47:23.385Z id: 09596501-57e4-4e2c-9e2f-46cd77765e44,
9302024-02-07T22:47:23.385Z name: Name(
9312024-02-07T22:47:23.385Z "debian11",
9322024-02-07T22:47:23.385Z ),
9332024-02-07T22:47:23.385Z os: "debian",
9342024-02-07T22:47:23.385Z project_id: None,
9352024-02-07T22:47:23.385Z size: ByteCount(
9362024-02-07T22:47:23.385Z 2147483648,
9372024-02-07T22:47:23.385Z ),
9382024-02-07T22:47:23.385Z time_created: 2024-02-07T22:47:23.100829Z,
9392024-02-07T22:47:23.385Z time_modified: 2024-02-07T22:47:23.325639Z,
9402024-02-07T22:47:23.385Z url: None,
9412024-02-07T22:47:23.385Z version: "11",
9422024-02-07T22:47:23.385Z}
9432024-02-07T22:47:23.385Z+ rm ./tests/bootstrap
9442024-02-07T22:47:23.413Z+ for test_bin in tests/*
9452024-02-07T22:47:23.413Z+ ./tests/bootstrap-5c5e256b526718ba
9462024-02-07T22:47:23.492Z
9472024-02-07T22:47:23.492Zrunning 0 tests
9482024-02-07T22:47:23.493Z
9492024-02-07T22:47:23.518Ztest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
9502024-02-07T22:47:23.518Z
9512024-02-07T22:47:23.518Z+ for test_bin in tests/*
9522024-02-07T22:47:23.518Z+ ./tests/end_to_end_tests-398b34350dfa5341
9532024-02-07T22:47:23.874Z
9542024-02-07T22:47:23.874Zrunning 2 tests
9552024-02-07T22:47:24.118Ztest no_spoof::no_spoof_here ... ok
9562024-02-07T22:47:56.983Ztest instance_launch::instance_launch ... ok
9572024-02-07T22:47:56.983Z
9582024-02-07T22:47:56.984Ztest result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 33.10s
9592024-02-07T22:47:56.984Z
9602024-02-07T22:47:57.011Z+ _exit_trap
9612024-02-07T22:47:57.011Z+ local status=0
9622024-02-07T22:47:57.011Z+ [[ 0 -eq 0 ]]
9632024-02-07T22:47:57.011Z+ exit 0
9642024-02-07T22:47:57.011Zprocess exited: duration 792284 ms, exit code 0
 
9652024-02-07T22:47:57.104Zfound 131 output files
9662024-02-07T22:47:57.104Zuploading: /var/svc/log/oxide-sled-agent:default.log (731130 bytes)
9672024-02-07T22:47:59.108Zuploaded: /var/svc/log/oxide-sled-agent:default.log
9682024-02-07T22:47:59.108Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log (733800 bytes)
9692024-02-07T22:47:59.940Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log" changed size mid upload: 733800 -> 746112
9702024-02-07T22:48:01.449Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log
9712024-02-07T22:48:01.449Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log.0 (2080539 bytes)
9722024-02-07T22:48:03.791Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log.0
9732024-02-07T22:48:03.791Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log (0 bytes)
9742024-02-07T22:48:04.880Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log
9752024-02-07T22:48:04.910Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log.0 (7871 bytes)
9762024-02-07T22:48:06.396Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log.0
9772024-02-07T22:48:06.396Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log (6269 bytes)
9782024-02-07T22:48:07.525Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log
9792024-02-07T22:48:07.525Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log.0 (16774 bytes)
9802024-02-07T22:48:08.661Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log.0
9812024-02-07T22:48:08.661Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log (345008 bytes)
9822024-02-07T22:48:08.887Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log" changed size mid upload: 345008 -> 366572
9832024-02-07T22:48:09.967Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log
9842024-02-07T22:48:09.968Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log.0 (1056119 bytes)
9852024-02-07T22:48:12.137Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log.0
9862024-02-07T22:48:12.138Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log (220802 bytes)
9872024-02-07T22:48:12.170Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log" changed size mid upload: 220802 -> 239484
9882024-02-07T22:48:13.435Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log
9892024-02-07T22:48:13.436Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log.0 (641961 bytes)
9902024-02-07T22:48:15.938Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log.0
9912024-02-07T22:48:15.938Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log (0 bytes)
9922024-02-07T22:48:16.614Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log
9932024-02-07T22:48:16.615Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log.0 (1388 bytes)
9942024-02-07T22:48:18.121Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log.0
9952024-02-07T22:48:18.122Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log (84795 bytes)
9962024-02-07T22:48:18.307Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log" changed size mid upload: 84795 -> 96402
9972024-02-07T22:48:19.377Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log
9982024-02-07T22:48:19.377Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log.0 (236091 bytes)
9992024-02-07T22:48:20.624Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log.0
10002024-02-07T22:48:20.624Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log (0 bytes)
10012024-02-07T22:48:20.709ZError: No response to keepalive - Terminating session
10022024-02-07T22:48:21.692Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log
10032024-02-07T22:48:21.692Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log.0 (617 bytes)
10042024-02-07T22:48:22.799Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log.0
10052024-02-07T22:48:22.799Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log (0 bytes)
10062024-02-07T22:48:23.877Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log
10072024-02-07T22:48:23.877Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log.0 (5392 bytes)
10082024-02-07T22:48:24.985Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log.0
10092024-02-07T22:48:24.985Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_b1c4b662-da79-4780-b7c5-cec8adfb3b9a/root/var/svc/log/oxide-clickhouse:default.log (502796 bytes)
10102024-02-07T22:48:26.468Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_b1c4b662-da79-4780-b7c5-cec8adfb3b9a/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 502796 -> 580289
10112024-02-07T22:48:27.574Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_b1c4b662-da79-4780-b7c5-cec8adfb3b9a/root/var/svc/log/oxide-clickhouse:default.log
10122024-02-07T22:48:27.574Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_b1c4b662-da79-4780-b7c5-cec8adfb3b9a/root/var/svc/log/oxide-clickhouse:default.log.0 (528779 bytes)
10132024-02-07T22:48:29.663Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_b1c4b662-da79-4780-b7c5-cec8adfb3b9a/root/var/svc/log/oxide-clickhouse:default.log.0
10142024-02-07T22:48:29.663Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_b1c4b662-da79-4780-b7c5-cec8adfb3b9a/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
10152024-02-07T22:48:30.734Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_b1c4b662-da79-4780-b7c5-cec8adfb3b9a/root/var/svc/log/oxide-zone-network-setup:default.log
10162024-02-07T22:48:30.775Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_b1c4b662-da79-4780-b7c5-cec8adfb3b9a/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1709 bytes)
10172024-02-07T22:48:30.927ZError: No response de-activating SOL payload
10182024-02-07T22:48:31.845Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_b1c4b662-da79-4780-b7c5-cec8adfb3b9a/root/var/svc/log/oxide-zone-network-setup:default.log.0
10192024-02-07T22:48:31.877Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_4eb224d1-21de-4e8a-9a6b-9c7734a936f7/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
10202024-02-07T22:48:32.049Z[SOL Session operational. Use ~? for help]
10212024-02-07T22:48:32.933Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_4eb224d1-21de-4e8a-9a6b-9c7734a936f7/root/var/svc/log/oxide-cockroachdb:default.log
10222024-02-07T22:48:32.934Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_4eb224d1-21de-4e8a-9a6b-9c7734a936f7/root/var/svc/log/oxide-cockroachdb:default.log.0 (4406 bytes)
10232024-02-07T22:48:34.037Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_4eb224d1-21de-4e8a-9a6b-9c7734a936f7/root/var/svc/log/oxide-cockroachdb:default.log.0
10242024-02-07T22:48:34.037Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_4eb224d1-21de-4e8a-9a6b-9c7734a936f7/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
10252024-02-07T22:48:35.133Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_4eb224d1-21de-4e8a-9a6b-9c7734a936f7/root/var/svc/log/oxide-zone-network-setup:default.log
10262024-02-07T22:48:35.134Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_4eb224d1-21de-4e8a-9a6b-9c7734a936f7/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1710 bytes)
10272024-02-07T22:48:36.246Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_4eb224d1-21de-4e8a-9a6b-9c7734a936f7/root/var/svc/log/oxide-zone-network-setup:default.log.0
10282024-02-07T22:48:36.246Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b27d56e-022f-48f2-a40d-9e80a205cc9f/root/var/svc/log/oxide-crucible-agent:default.log (0 bytes)
10292024-02-07T22:48:45.895Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b27d56e-022f-48f2-a40d-9e80a205cc9f/root/var/svc/log/oxide-crucible-agent:default.log
10302024-02-07T22:48:45.895Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b27d56e-022f-48f2-a40d-9e80a205cc9f/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
10312024-02-07T22:48:47.763Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b27d56e-022f-48f2-a40d-9e80a205cc9f/root/var/svc/log/oxide-crucible-agent:default.log.0
10322024-02-07T22:48:47.763Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b27d56e-022f-48f2-a40d-9e80a205cc9f/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
10332024-02-07T22:49:02.801Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b27d56e-022f-48f2-a40d-9e80a205cc9f/root/var/svc/log/oxide-crucible-downstairs:default.log
10342024-02-07T22:49:02.801Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b27d56e-022f-48f2-a40d-9e80a205cc9f/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
10352024-02-07T22:49:04.020Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_9b27d56e-022f-48f2-a40d-9e80a205cc9f/root/var/svc/log/oxide-crucible-downstairs:default.log.0
10362024-02-07T22:49:04.021Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_6c172128-e4c9-4ec7-b22d-f507a5c16c59/root/var/svc/log/oxide-external_dns:default.log (11333 bytes)
10372024-02-07T22:49:04.118Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_6c172128-e4c9-4ec7-b22d-f507a5c16c59/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 11333 -> 15111
10382024-02-07T22:49:05.660Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_6c172128-e4c9-4ec7-b22d-f507a5c16c59/root/var/svc/log/oxide-external_dns:default.log
10392024-02-07T22:49:05.661Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_6c172128-e4c9-4ec7-b22d-f507a5c16c59/root/var/svc/log/oxide-external_dns:default.log.0 (15613 bytes)
10402024-02-07T22:49:06.872Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_6c172128-e4c9-4ec7-b22d-f507a5c16c59/root/var/svc/log/oxide-external_dns:default.log.0
10412024-02-07T22:49:06.872Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_0ab16e51-98ef-47c9-8483-3b7345a3f73e/root/var/svc/log/oxide-internal_dns:default.log (11264 bytes)
10422024-02-07T22:49:06.981Zupload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_0ab16e51-98ef-47c9-8483-3b7345a3f73e/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 11264 -> 15020
10432024-02-07T22:49:08.096Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_0ab16e51-98ef-47c9-8483-3b7345a3f73e/root/var/svc/log/oxide-internal_dns:default.log
10442024-02-07T22:49:08.096Zuploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_0ab16e51-98ef-47c9-8483-3b7345a3f73e/root/var/svc/log/oxide-internal_dns:default.log.0 (102467 bytes)
10452024-02-07T22:49:09.462Zuploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_0ab16e51-98ef-47c9-8483-3b7345a3f73e/root/var/svc/log/oxide-internal_dns:default.log.0
10462024-02-07T22:49:09.462Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_ce2d4fed-394a-468c-9614-f5ababd0bbac/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
10472024-02-07T22:49:10.534Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_ce2d4fed-394a-468c-9614-f5ababd0bbac/root/var/svc/log/oxide-cockroachdb:default.log
10482024-02-07T22:49:10.534Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_ce2d4fed-394a-468c-9614-f5ababd0bbac/root/var/svc/log/oxide-cockroachdb:default.log.0 (4406 bytes)
10492024-02-07T22:49:11.648Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_ce2d4fed-394a-468c-9614-f5ababd0bbac/root/var/svc/log/oxide-cockroachdb:default.log.0
10502024-02-07T22:49:11.648Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_ce2d4fed-394a-468c-9614-f5ababd0bbac/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
10512024-02-07T22:49:13.135Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_ce2d4fed-394a-468c-9614-f5ababd0bbac/root/var/svc/log/oxide-zone-network-setup:default.log
10522024-02-07T22:49:13.135Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_ce2d4fed-394a-468c-9614-f5ababd0bbac/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1711 bytes)
10532024-02-07T22:49:14.239Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_ce2d4fed-394a-468c-9614-f5ababd0bbac/root/var/svc/log/oxide-zone-network-setup:default.log.0
10542024-02-07T22:49:14.239Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-agent:default.log (13684 bytes)
10552024-02-07T22:49:15.375Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-agent:default.log
10562024-02-07T22:49:15.375Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-agent:default.log.0 (9359 bytes)
10572024-02-07T22:49:16.498Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-agent:default.log.0
10582024-02-07T22:49:16.498Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
10592024-02-07T22:49:20.114Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-downstairs:default.log
10602024-02-07T22:49:20.114Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
10612024-02-07T22:49:20.668Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-downstairs:default.log.0
10622024-02-07T22:49:20.668Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-downstairs:downstairs-05856daa-212c-4df5-a2e7-53df60e99aaa.log (1090 bytes)
10632024-02-07T22:49:21.792Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-downstairs:downstairs-05856daa-212c-4df5-a2e7-53df60e99aaa.log
10642024-02-07T22:49:21.792Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-downstairs:downstairs-05856daa-212c-4df5-a2e7-53df60e99aaa.log.0 (4530 bytes)
10652024-02-07T22:49:22.886Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-downstairs:downstairs-05856daa-212c-4df5-a2e7-53df60e99aaa.log.0
10662024-02-07T22:49:22.886Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-downstairs:downstairs-56a75498-e470-4ca5-8c68-f9f5c3e85559.log (2933 bytes)
10672024-02-07T22:49:23.979Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-downstairs:downstairs-56a75498-e470-4ca5-8c68-f9f5c3e85559.log
10682024-02-07T22:49:23.979Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-downstairs:snapshot-05856daa-212c-4df5-a2e7-53df60e99aaa-c419998c-6c56-4366-8fb2-eb576bcaddbb.log (5154 bytes)
10692024-02-07T22:49:25.129Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_eea7d0e5-fe3b-43f3-a11c-2539423d7466/root/var/svc/log/oxide-crucible-downstairs:snapshot-05856daa-212c-4df5-a2e7-53df60e99aaa-c419998c-6c56-4366-8fb2-eb576bcaddbb.log
10702024-02-07T22:49:25.130Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_f506b660-35a0-4a14-a88b-86435a183cf8/root/var/svc/log/oxide-external_dns:default.log (11327 bytes)
10712024-02-07T22:49:25.223Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_f506b660-35a0-4a14-a88b-86435a183cf8/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 11327 -> 15105
10722024-02-07T22:49:26.314Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_f506b660-35a0-4a14-a88b-86435a183cf8/root/var/svc/log/oxide-external_dns:default.log
10732024-02-07T22:49:26.314Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_f506b660-35a0-4a14-a88b-86435a183cf8/root/var/svc/log/oxide-external_dns:default.log.0 (15615 bytes)
10742024-02-07T22:49:27.882Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_f506b660-35a0-4a14-a88b-86435a183cf8/root/var/svc/log/oxide-external_dns:default.log.0
10752024-02-07T22:49:27.882Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_142ae2a1-c8dd-4fce-b1a1-392fd63123a9/root/var/svc/log/oxide-internal_dns:default.log (11266 bytes)
10762024-02-07T22:49:27.980Zupload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_142ae2a1-c8dd-4fce-b1a1-392fd63123a9/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 11266 -> 15022
10772024-02-07T22:49:29.064Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_142ae2a1-c8dd-4fce-b1a1-392fd63123a9/root/var/svc/log/oxide-internal_dns:default.log
10782024-02-07T22:49:29.064Zuploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_142ae2a1-c8dd-4fce-b1a1-392fd63123a9/root/var/svc/log/oxide-internal_dns:default.log.0 (106127 bytes)
10792024-02-07T22:49:30.558Zuploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_142ae2a1-c8dd-4fce-b1a1-392fd63123a9/root/var/svc/log/oxide-internal_dns:default.log.0
10802024-02-07T22:49:30.558Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_1af3bc40-da27-466a-8df9-a9ac106fb16a/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
10812024-02-07T22:49:31.640Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_1af3bc40-da27-466a-8df9-a9ac106fb16a/root/var/svc/log/oxide-cockroachdb:default.log
10822024-02-07T22:49:31.640Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_1af3bc40-da27-466a-8df9-a9ac106fb16a/root/var/svc/log/oxide-cockroachdb:default.log.0 (4382 bytes)
10832024-02-07T22:49:33.215Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_1af3bc40-da27-466a-8df9-a9ac106fb16a/root/var/svc/log/oxide-cockroachdb:default.log.0
10842024-02-07T22:49:33.215Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_1af3bc40-da27-466a-8df9-a9ac106fb16a/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
10852024-02-07T22:49:34.292Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_1af3bc40-da27-466a-8df9-a9ac106fb16a/root/var/svc/log/oxide-zone-network-setup:default.log
10862024-02-07T22:49:34.293Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_1af3bc40-da27-466a-8df9-a9ac106fb16a/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1711 bytes)
10872024-02-07T22:49:35.647Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_1af3bc40-da27-466a-8df9-a9ac106fb16a/root/var/svc/log/oxide-zone-network-setup:default.log.0
10882024-02-07T22:49:35.647Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-agent:default.log (7757 bytes)
10892024-02-07T22:49:36.761Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-agent:default.log
10902024-02-07T22:49:36.762Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-agent:default.log.0 (9362 bytes)
10912024-02-07T22:49:38.350Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-agent:default.log.0
10922024-02-07T22:49:38.350Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
10932024-02-07T22:49:39.419Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-downstairs:default.log
10942024-02-07T22:49:39.420Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
10952024-02-07T22:49:40.525Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-downstairs:default.log.0
10962024-02-07T22:49:40.525Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-downstairs:downstairs-0f92844c-537d-4edb-a9a8-a4f9222fa286.log (1090 bytes)
10972024-02-07T22:49:42.455Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-downstairs:downstairs-0f92844c-537d-4edb-a9a8-a4f9222fa286.log
10982024-02-07T22:49:42.456Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-downstairs:downstairs-0f92844c-537d-4edb-a9a8-a4f9222fa286.log.0 (4526 bytes)
10992024-02-07T22:49:43.556Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-downstairs:downstairs-0f92844c-537d-4edb-a9a8-a4f9222fa286.log.0
11002024-02-07T22:49:43.556Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-downstairs:snapshot-0f92844c-537d-4edb-a9a8-a4f9222fa286-c419998c-6c56-4366-8fb2-eb576bcaddbb.log (5156 bytes)
11012024-02-07T22:49:49.808Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_1481d667-2d0e-4ea5-b496-38c2d5df32c7/root/var/svc/log/oxide-crucible-downstairs:snapshot-0f92844c-537d-4edb-a9a8-a4f9222fa286-c419998c-6c56-4366-8fb2-eb576bcaddbb.log
11022024-02-07T22:49:49.808Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_pantry_43bfd008-af90-4137-9154-1a92d241b5f8/root/var/svc/log/oxide-crucible-pantry:default.log (0 bytes)
11032024-02-07T22:49:51.376Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_pantry_43bfd008-af90-4137-9154-1a92d241b5f8/root/var/svc/log/oxide-crucible-pantry:default.log
11042024-02-07T22:49:51.377Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_pantry_43bfd008-af90-4137-9154-1a92d241b5f8/root/var/svc/log/oxide-crucible-pantry:default.log.0 (2337 bytes)
11052024-02-07T22:49:52.022Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_pantry_43bfd008-af90-4137-9154-1a92d241b5f8/root/var/svc/log/oxide-crucible-pantry:default.log.0
11062024-02-07T22:49:52.022Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_5764c971-0e8b-4e3d-9161-219d83575e34/root/var/svc/log/oxide-internal_dns:default.log (11262 bytes)
11072024-02-07T22:49:52.188Zupload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_5764c971-0e8b-4e3d-9161-219d83575e34/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 11262 -> 15017
11082024-02-07T22:49:53.308Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_5764c971-0e8b-4e3d-9161-219d83575e34/root/var/svc/log/oxide-internal_dns:default.log
11092024-02-07T22:49:53.308Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_5764c971-0e8b-4e3d-9161-219d83575e34/root/var/svc/log/oxide-internal_dns:default.log.0 (102142 bytes)
11102024-02-07T22:49:54.758Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_5764c971-0e8b-4e3d-9161-219d83575e34/root/var/svc/log/oxide-internal_dns:default.log.0
11112024-02-07T22:49:54.759Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_660d3f1f-7977-4be3-b624-3a6d8f918dea/root/var/svc/log/oxide-nexus:default.log (351198 bytes)
11122024-02-07T22:49:56.812Zupload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_660d3f1f-7977-4be3-b624-3a6d8f918dea/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 351198 -> 585335
11132024-02-07T22:49:57.885Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_660d3f1f-7977-4be3-b624-3a6d8f918dea/root/var/svc/log/oxide-nexus:default.log
11142024-02-07T22:49:57.885Zuploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_660d3f1f-7977-4be3-b624-3a6d8f918dea/root/var/svc/log/oxide-nexus:default.log.0 (1101378 bytes)
11152024-02-07T22:50:00.761Zuploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_660d3f1f-7977-4be3-b624-3a6d8f918dea/root/var/svc/log/oxide-nexus:default.log.0
11162024-02-07T22:50:00.762Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_db796704-3b39-404e-8a09-fe7a46a8459f/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
11172024-02-07T22:50:01.845Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_db796704-3b39-404e-8a09-fe7a46a8459f/root/var/svc/log/oxide-cockroachdb:default.log
11182024-02-07T22:50:01.845Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_db796704-3b39-404e-8a09-fe7a46a8459f/root/var/svc/log/oxide-cockroachdb:default.log.0 (4408 bytes)
11192024-02-07T22:50:02.990Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_db796704-3b39-404e-8a09-fe7a46a8459f/root/var/svc/log/oxide-cockroachdb:default.log.0
11202024-02-07T22:50:02.990Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_db796704-3b39-404e-8a09-fe7a46a8459f/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
11212024-02-07T22:50:08.706Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_db796704-3b39-404e-8a09-fe7a46a8459f/root/var/svc/log/oxide-zone-network-setup:default.log
11222024-02-07T22:50:08.706Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_db796704-3b39-404e-8a09-fe7a46a8459f/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1710 bytes)
11232024-02-07T22:50:09.802Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_db796704-3b39-404e-8a09-fe7a46a8459f/root/var/svc/log/oxide-zone-network-setup:default.log.0
11242024-02-07T22:50:09.802Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-agent:default.log (10679 bytes)
11252024-02-07T22:50:10.976Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-agent:default.log
11262024-02-07T22:50:10.977Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-agent:default.log.0 (14150 bytes)
11272024-02-07T22:50:12.810Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-agent:default.log.0
11282024-02-07T22:50:12.810Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11292024-02-07T22:50:13.901Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-downstairs:default.log
11302024-02-07T22:50:13.901Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11312024-02-07T22:50:15.028Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11322024-02-07T22:50:15.028Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-downstairs:downstairs-232d824d-fd99-4bb6-987e-38073f072292.log (0 bytes)
11332024-02-07T22:50:16.119Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-downstairs:downstairs-232d824d-fd99-4bb6-987e-38073f072292.log
11342024-02-07T22:50:16.120Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-downstairs:downstairs-232d824d-fd99-4bb6-987e-38073f072292.log.0 (3050 bytes)
11352024-02-07T22:50:17.251Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-downstairs:downstairs-232d824d-fd99-4bb6-987e-38073f072292.log.0
11362024-02-07T22:50:17.279Zuploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-downstairs:downstairs-3cb85022-f0bb-4d3e-8673-747299cd446b.log (5517 bytes)
11372024-02-07T22:50:18.391Zuploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_01ce0537-29b2-4971-adc8-ada60f183ab7/root/var/svc/log/oxide-crucible-downstairs:downstairs-3cb85022-f0bb-4d3e-8673-747299cd446b.log
11382024-02-07T22:50:18.391Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_c1a0177b-1d4e-4d44-a554-744deb526c03/root/var/svc/log/oxide-cockroachdb:default.log (0 bytes)
11392024-02-07T22:50:19.474Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_c1a0177b-1d4e-4d44-a554-744deb526c03/root/var/svc/log/oxide-cockroachdb:default.log
11402024-02-07T22:50:19.474Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_c1a0177b-1d4e-4d44-a554-744deb526c03/root/var/svc/log/oxide-cockroachdb:default.log.0 (4408 bytes)
11412024-02-07T22:50:20.592Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_c1a0177b-1d4e-4d44-a554-744deb526c03/root/var/svc/log/oxide-cockroachdb:default.log.0
11422024-02-07T22:50:20.593Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_c1a0177b-1d4e-4d44-a554-744deb526c03/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
11432024-02-07T22:50:24.086Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_c1a0177b-1d4e-4d44-a554-744deb526c03/root/var/svc/log/oxide-zone-network-setup:default.log
11442024-02-07T22:50:24.086Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_c1a0177b-1d4e-4d44-a554-744deb526c03/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1711 bytes)
11452024-02-07T22:50:24.973Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_c1a0177b-1d4e-4d44-a554-744deb526c03/root/var/svc/log/oxide-zone-network-setup:default.log.0
11462024-02-07T22:50:24.973Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_6efaaf4b-1d08-4410-9715-627796065d21/root/var/svc/log/oxide-crucible-agent:default.log (6359 bytes)
11472024-02-07T22:50:26.542Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_6efaaf4b-1d08-4410-9715-627796065d21/root/var/svc/log/oxide-crucible-agent:default.log
11482024-02-07T22:50:26.542Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_6efaaf4b-1d08-4410-9715-627796065d21/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
11492024-02-07T22:50:28.071Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_6efaaf4b-1d08-4410-9715-627796065d21/root/var/svc/log/oxide-crucible-agent:default.log.0
11502024-02-07T22:50:28.071Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_6efaaf4b-1d08-4410-9715-627796065d21/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11512024-02-07T22:50:29.565Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_6efaaf4b-1d08-4410-9715-627796065d21/root/var/svc/log/oxide-crucible-downstairs:default.log
11522024-02-07T22:50:29.565Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_6efaaf4b-1d08-4410-9715-627796065d21/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11532024-02-07T22:50:30.662Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_6efaaf4b-1d08-4410-9715-627796065d21/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11542024-02-07T22:50:30.698Zuploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_6efaaf4b-1d08-4410-9715-627796065d21/root/var/svc/log/oxide-crucible-downstairs:downstairs-62356106-9a64-4712-892a-c98983432b0c.log (2928 bytes)
11552024-02-07T22:50:36.340Zuploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_6efaaf4b-1d08-4410-9715-627796065d21/root/var/svc/log/oxide-crucible-downstairs:downstairs-62356106-9a64-4712-892a-c98983432b0c.log
11562024-02-07T22:50:36.340Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_0e0ccc06-6e15-4baa-a711-ea9939ce3597/root/var/svc/log/oxide-crucible-agent:default.log (10679 bytes)
11572024-02-07T22:50:37.513Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_0e0ccc06-6e15-4baa-a711-ea9939ce3597/root/var/svc/log/oxide-crucible-agent:default.log
11582024-02-07T22:50:37.517Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_0e0ccc06-6e15-4baa-a711-ea9939ce3597/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
11592024-02-07T22:50:38.627Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_0e0ccc06-6e15-4baa-a711-ea9939ce3597/root/var/svc/log/oxide-crucible-agent:default.log.0
11602024-02-07T22:50:38.653Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_0e0ccc06-6e15-4baa-a711-ea9939ce3597/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11612024-02-07T22:50:39.705Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_0e0ccc06-6e15-4baa-a711-ea9939ce3597/root/var/svc/log/oxide-crucible-downstairs:default.log
11622024-02-07T22:50:39.705Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_0e0ccc06-6e15-4baa-a711-ea9939ce3597/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11632024-02-07T22:50:40.827Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_0e0ccc06-6e15-4baa-a711-ea9939ce3597/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11642024-02-07T22:50:40.827Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_0e0ccc06-6e15-4baa-a711-ea9939ce3597/root/var/svc/log/oxide-crucible-downstairs:downstairs-8b2e146c-0b35-40db-aee8-ce32b0e5b3d8.log (5515 bytes)
11652024-02-07T22:50:41.975Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_0e0ccc06-6e15-4baa-a711-ea9939ce3597/root/var/svc/log/oxide-crucible-downstairs:downstairs-8b2e146c-0b35-40db-aee8-ce32b0e5b3d8.log
11662024-02-07T22:50:41.975Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_nexus_7512efda-9eab-44be-a06b-b127ca0bd7aa/root/var/svc/log/oxide-nexus:default.log (354100 bytes)
11672024-02-07T22:50:43.676Zupload warning: file "/pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_nexus_7512efda-9eab-44be-a06b-b127ca0bd7aa/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 354100 -> 620986
11682024-02-07T22:50:44.788Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_nexus_7512efda-9eab-44be-a06b-b127ca0bd7aa/root/var/svc/log/oxide-nexus:default.log
11692024-02-07T22:50:44.788Zuploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_nexus_7512efda-9eab-44be-a06b-b127ca0bd7aa/root/var/svc/log/oxide-nexus:default.log.0 (685410 bytes)
11702024-02-07T22:50:46.897Zuploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_nexus_7512efda-9eab-44be-a06b-b127ca0bd7aa/root/var/svc/log/oxide-nexus:default.log.0
11712024-02-07T22:50:46.913Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-agent:default.log (5930 bytes)
11722024-02-07T22:50:48.395Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-agent:default.log
11732024-02-07T22:50:48.396Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-agent:default.log.0 (13688 bytes)
11742024-02-07T22:50:49.751Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-agent:default.log.0
11752024-02-07T22:50:49.751Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11762024-02-07T22:50:50.840Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-downstairs:default.log
11772024-02-07T22:50:50.840Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11782024-02-07T22:50:51.925Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11792024-02-07T22:50:51.925Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-downstairs:downstairs-58d7300d-4987-4f2c-bf44-de928b1c6fb7.log (2932 bytes)
11802024-02-07T22:50:53.017Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-downstairs:downstairs-58d7300d-4987-4f2c-bf44-de928b1c6fb7.log
11812024-02-07T22:50:53.017Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-downstairs:downstairs-b067c645-f632-4bcc-99bd-d8195ba5b6d8.log (0 bytes)
11822024-02-07T22:50:56.282Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-downstairs:downstairs-b067c645-f632-4bcc-99bd-d8195ba5b6d8.log
11832024-02-07T22:50:56.282Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-downstairs:downstairs-b067c645-f632-4bcc-99bd-d8195ba5b6d8.log.0 (3051 bytes)
11842024-02-07T22:50:57.781Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2ce1c66b-948b-4120-8b98-ba6ffd124460/root/var/svc/log/oxide-crucible-downstairs:downstairs-b067c645-f632-4bcc-99bd-d8195ba5b6d8.log.0
11852024-02-07T22:50:57.782Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_pantry_97d02147-2ebd-44b1-a475-e4b25d1781c4/root/var/svc/log/oxide-crucible-pantry:default.log (0 bytes)
11862024-02-07T22:50:58.860Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_pantry_97d02147-2ebd-44b1-a475-e4b25d1781c4/root/var/svc/log/oxide-crucible-pantry:default.log
11872024-02-07T22:50:58.860Zuploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_pantry_97d02147-2ebd-44b1-a475-e4b25d1781c4/root/var/svc/log/oxide-crucible-pantry:default.log.0 (2338 bytes)
11882024-02-07T22:50:59.959Zuploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_pantry_97d02147-2ebd-44b1-a475-e4b25d1781c4/root/var/svc/log/oxide-crucible-pantry:default.log.0
11892024-02-07T22:50:59.986Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-agent:default.log (7756 bytes)
11902024-02-07T22:51:01.089Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-agent:default.log
11912024-02-07T22:51:01.089Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-agent:default.log.0 (19614 bytes)
11922024-02-07T22:51:02.299Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-agent:default.log.0
11932024-02-07T22:51:02.299Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
11942024-02-07T22:51:03.676Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:default.log
11952024-02-07T22:51:03.676Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
11962024-02-07T22:51:06.277Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:default.log.0
11972024-02-07T22:51:06.277Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:downstairs-40d9861c-fb1b-4ded-932f-aab2091ad717.log (0 bytes)
11982024-02-07T22:51:07.421Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:downstairs-40d9861c-fb1b-4ded-932f-aab2091ad717.log
11992024-02-07T22:51:07.422Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:downstairs-40d9861c-fb1b-4ded-932f-aab2091ad717.log.0 (3051 bytes)
12002024-02-07T22:51:08.961Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:downstairs-40d9861c-fb1b-4ded-932f-aab2091ad717.log.0
12012024-02-07T22:51:08.962Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:downstairs-62d7d11f-b007-432c-a631-2d570c318ef1.log (1091 bytes)
12022024-02-07T22:51:10.061Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:downstairs-62d7d11f-b007-432c-a631-2d570c318ef1.log
12032024-02-07T22:51:10.061Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:downstairs-62d7d11f-b007-432c-a631-2d570c318ef1.log.0 (4526 bytes)
12042024-02-07T22:51:11.163Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:downstairs-62d7d11f-b007-432c-a631-2d570c318ef1.log.0
12052024-02-07T22:51:11.164Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:snapshot-62d7d11f-b007-432c-a631-2d570c318ef1-c419998c-6c56-4366-8fb2-eb576bcaddbb.log (5155 bytes)
12062024-02-07T22:51:12.276Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_6741c1c4-4bc8-428c-bb0d-84b69658b3e4/root/var/svc/log/oxide-crucible-downstairs:snapshot-62d7d11f-b007-432c-a631-2d570c318ef1-c419998c-6c56-4366-8fb2-eb576bcaddbb.log
12072024-02-07T22:51:12.276Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_40121e29-293f-4f81-a234-6732f8641d5d/root/var/svc/log/oxide-crucible-pantry:default.log (542767 bytes)
12082024-02-07T22:51:15.697Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_40121e29-293f-4f81-a234-6732f8641d5d/root/var/svc/log/oxide-crucible-pantry:default.log
12092024-02-07T22:51:15.697Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_40121e29-293f-4f81-a234-6732f8641d5d/root/var/svc/log/oxide-crucible-pantry:default.log.0 (680992 bytes)
12102024-02-07T22:51:18.270Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_40121e29-293f-4f81-a234-6732f8641d5d/root/var/svc/log/oxide-crucible-pantry:default.log.0
12112024-02-07T22:51:18.270Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_1ae44761-6ce8-4fc9-9821-f03a8178d5a3/root/var/svc/log/oxide-oximeter:default.log (140417 bytes)
12122024-02-07T22:51:18.914Zupload warning: file "/pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_1ae44761-6ce8-4fc9-9821-f03a8178d5a3/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 140417 -> 294847
12132024-02-07T22:51:20.023Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_1ae44761-6ce8-4fc9-9821-f03a8178d5a3/root/var/svc/log/oxide-oximeter:default.log
12142024-02-07T22:51:20.023Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_1ae44761-6ce8-4fc9-9821-f03a8178d5a3/root/var/svc/log/oxide-oximeter:default.log.0 (142603 bytes)
12152024-02-07T22:51:21.455Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_1ae44761-6ce8-4fc9-9821-f03a8178d5a3/root/var/svc/log/oxide-oximeter:default.log.0
12162024-02-07T22:51:21.455Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_1ae44761-6ce8-4fc9-9821-f03a8178d5a3/root/var/svc/log/oxide-zone-network-setup:default.log (0 bytes)
12172024-02-07T22:51:22.550Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_1ae44761-6ce8-4fc9-9821-f03a8178d5a3/root/var/svc/log/oxide-zone-network-setup:default.log
12182024-02-07T22:51:22.550Zuploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_1ae44761-6ce8-4fc9-9821-f03a8178d5a3/root/var/svc/log/oxide-zone-network-setup:default.log.0 (1695 bytes)
12192024-02-07T22:51:23.676Zuploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_1ae44761-6ce8-4fc9-9821-f03a8178d5a3/root/var/svc/log/oxide-zone-network-setup:default.log.0
12202024-02-07T22:51:23.676Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_3080fed9-854b-45b1-94f5-31e519282a1d/root/var/svc/log/oxide-crucible-agent:default.log (10251 bytes)
12212024-02-07T22:51:29.039Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_3080fed9-854b-45b1-94f5-31e519282a1d/root/var/svc/log/oxide-crucible-agent:default.log
12222024-02-07T22:51:29.039Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_3080fed9-854b-45b1-94f5-31e519282a1d/root/var/svc/log/oxide-crucible-agent:default.log.0 (3436 bytes)
12232024-02-07T22:51:30.150Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_3080fed9-854b-45b1-94f5-31e519282a1d/root/var/svc/log/oxide-crucible-agent:default.log.0
12242024-02-07T22:51:30.229Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_3080fed9-854b-45b1-94f5-31e519282a1d/root/var/svc/log/oxide-crucible-downstairs:default.log (0 bytes)
12252024-02-07T22:51:31.230Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_3080fed9-854b-45b1-94f5-31e519282a1d/root/var/svc/log/oxide-crucible-downstairs:default.log
12262024-02-07T22:51:31.230Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_3080fed9-854b-45b1-94f5-31e519282a1d/root/var/svc/log/oxide-crucible-downstairs:default.log.0 (75 bytes)
12272024-02-07T22:51:32.360Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_3080fed9-854b-45b1-94f5-31e519282a1d/root/var/svc/log/oxide-crucible-downstairs:default.log.0
12282024-02-07T22:51:32.360Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_3080fed9-854b-45b1-94f5-31e519282a1d/root/var/svc/log/oxide-crucible-downstairs:downstairs-923d032b-e0a5-47f6-a45d-a9dcdd8c6e7d.log (5517 bytes)
12292024-02-07T22:51:33.490Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_3080fed9-854b-45b1-94f5-31e519282a1d/root/var/svc/log/oxide-crucible-downstairs:downstairs-923d032b-e0a5-47f6-a45d-a9dcdd8c6e7d.log
12302024-02-07T22:51:33.490Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_0e585f27-de0d-4a61-9e2e-3a2a59ed7868/root/var/svc/log/oxide-nexus:default.log (16243927 bytes)
12312024-02-07T22:51:36.684ZError: No response to keepalive - Terminating session
12322024-02-07T22:51:42.751Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_0e585f27-de0d-4a61-9e2e-3a2a59ed7868/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 16243927 -> 16641084
12332024-02-07T22:51:44.271Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_0e585f27-de0d-4a61-9e2e-3a2a59ed7868/root/var/svc/log/oxide-nexus:default.log
12342024-02-07T22:51:44.271Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_0e585f27-de0d-4a61-9e2e-3a2a59ed7868/root/var/svc/log/oxide-nexus:default.log.0 (19449011 bytes)
12352024-02-07T22:51:46.895ZError: No response de-activating SOL payload
12362024-02-07T22:51:48.115Z[SOL Session operational. Use ~? for help]
12372024-02-07T22:51:54.877Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_0e585f27-de0d-4a61-9e2e-3a2a59ed7868/root/var/svc/log/oxide-nexus:default.log.0
12382024-02-07T22:51:54.877Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/svc/log/oxide-ntp:default.log (0 bytes)
12392024-02-07T22:51:55.999Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/svc/log/oxide-ntp:default.log
12402024-02-07T22:51:55.999Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/svc/log/oxide-ntp:default.log.0 (1052 bytes)
12412024-02-07T22:51:57.392Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/svc/log/oxide-ntp:default.log.0
12422024-02-07T22:51:57.392Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/log/chrony/measurements.log (23153 bytes)
12432024-02-07T22:51:57.473Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/log/chrony/measurements.log" changed size mid upload: 23153 -> 31921
12442024-02-07T22:51:58.963Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/log/chrony/measurements.log
12452024-02-07T22:51:58.963Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/log/chrony/statistics.log (15678 bytes)
12462024-02-07T22:51:59.121Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/log/chrony/statistics.log" changed size mid upload: 15678 -> 23049
12472024-02-07T22:52:00.238Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/log/chrony/statistics.log
12482024-02-07T22:52:00.238Zuploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/log/chrony/tracking.log (2250 bytes)
12492024-02-07T22:52:00.315Zupload warning: file "/pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/log/chrony/tracking.log" changed size mid upload: 2250 -> 2778
12502024-02-07T22:52:01.409Zuploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_ntp_70719f18-685e-4b21-b6f4-b5254c103f60/root/var/log/chrony/tracking.log