01H17JK3X9NZ7MYVV8X18JGCJ2: helios / deploy

BasicConfig {
    output_rules: [
        "%/var/svc/log/oxide-sled-agent:default.log",
        "%/zone/oxz_*/root/var/svc/log/oxide-*.log",
        "%/zone/oxz_*/root/var/svc/log/system-illumos-*.log",
    ],
    rust_toolchain: None,
    target: Some(
        "lab-opte-0.23",
    ),
    access_repos: [],
    publish: [],
    skip_clone: true,
}

Buildomat Job: 01H17JN2QHREXB01CGQZP8916D

Tags:

Artefacts:

Output:

SEQ GLOBAL TIME DETAILS
12023-05-24T20:01:44.159Zjob dependencies complete; ready to run (waiting for 52 m 15 s)
22023-05-24T20:12:05.917Zjob assigned to worker 01H17P6MHMBA8XQFC45Q15Q2NY (queued for 10 m 21 s)
32023-05-24T20:12:05.950Zhttp://172.20.0.5:9969/ipxe/sock... ok
42023-05-24T20:12:06.020Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/kernel/amd64/unix... ok
52023-05-24T20:12:06.064Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/amd64/boot_archive... ok
62023-05-24T20:12:06.109Zhttp://172.20.0.5:9969/os/sock/platform/i86pc/amd64/boot_archive.hash... ok
72023-05-24T20:12:06.153Zhttp://172.20.0.5:9969/postboot/sock... ok
82023-05-24T20:12:06.198ZwcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U
92023-05-24T20:12:06.243ZOxide Helios Version stlouis-0-g7b17295d6d 64-bit (onu)
102023-05-24T20:12:06.287ZConfiguring devices.
112023-05-24T20:12:06.426Z#################################################
122023-05-24T20:12:06.505Z#### ############# ##
132023-05-24T20:12:06.561Z### ### ############ ##
142023-05-24T20:12:06.620Z## ### # ## ### ## Oxide Computer Company ##
152023-05-24T20:12:06.675Z## ## ## ### # ### ##
162023-05-24T20:12:06.723Z## # ### #### #### This Station Under ##
172023-05-24T20:12:06.767Z### ### #### # ### Computer Control ##
182023-05-24T20:12:06.820Z#### #### ### ## ##
192023-05-24T20:12:06.869Z##################################################
202023-05-24T20:12:06.918ZHostname: sock
212023-05-24T20:12:07.411ZLoading smf(7) serice descriptions:
222023-05-24T20:12:08.592Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap-fdf2ceb9b8861fae.gz
232023-05-24T20:12:10.084Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap-fdf2ceb9b8861fae.gz
242023-05-24T20:12:10.110Zdownloading input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
252023-05-24T20:12:29.857Zdownloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz
262023-05-24T20:12:52.923Zdownloading input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-daffba9032038905.gz
272023-05-24T20:13:15.039Zdownloaded input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-daffba9032038905.gz
282023-05-24T20:13:15.072Zdownloading input: /input/ci-tools/work/tufaceous.gz
292023-05-24T20:13:17.458Zdownloaded input: /input/ci-tools/work/tufaceous.gz
302023-05-24T20:13:17.506Zdownloading input: /input/package/work/package.tar.gz
312023-05-24T20:13:32.775Zdownloaded input: /input/package/work/package.tar.gz
322023-05-24T20:13:32.802Zdownloading input: /input/package/work/global-zone-packages.tar.gz
332023-05-24T20:13:53.640Zdownloaded input: /input/package/work/global-zone-packages.tar.gz
342023-05-24T20:13:53.886Zdownloading input: /input/package/work/trampoline-global-zone-packages.tar.gz
352023-05-24T20:14:05.511Zdownloaded input: /input/package/work/trampoline-global-zone-packages.tar.gz
362023-05-24T20:14:05.718Zdownloading input: /input/package/work/zones/clickhouse.tar.gz
372023-05-24T20:14:27.852Zdownloaded input: /input/package/work/zones/clickhouse.tar.gz
382023-05-24T20:14:27.882Zdownloading input: /input/package/work/zones/cockroachdb.tar.gz
392023-05-24T20:14:47.164Zdownloaded input: /input/package/work/zones/cockroachdb.tar.gz
402023-05-24T20:14:47.191Zdownloading input: /input/package/work/zones/crucible-pantry.tar.gz
412023-05-24T20:14:51.862Zdownloaded input: /input/package/work/zones/crucible-pantry.tar.gz
422023-05-24T20:14:51.888Zdownloading input: /input/package/work/zones/crucible.tar.gz
432023-05-24T20:14:59.233Zdownloaded input: /input/package/work/zones/crucible.tar.gz
442023-05-24T20:14:59.270Zdownloading input: /input/package/work/zones/external-dns.tar.gz
452023-05-24T20:15:05.034Zdownloaded input: /input/package/work/zones/external-dns.tar.gz
462023-05-24T20:15:05.071Zdownloading input: /input/package/work/zones/internal-dns.tar.gz
472023-05-24T20:15:09.432Zdownloaded input: /input/package/work/zones/internal-dns.tar.gz
482023-05-24T20:15:09.708Zdownloading input: /input/package/work/zones/ntp.tar.gz
492023-05-24T20:15:09.754Zdownloaded input: /input/package/work/zones/ntp.tar.gz
502023-05-24T20:15:09.785Zdownloading input: /input/package/work/zones/omicron-nexus.tar.gz
512023-05-24T20:15:22.083Zdownloaded input: /input/package/work/zones/omicron-nexus.tar.gz
522023-05-24T20:15:22.123Zdownloading input: /input/package/work/zones/oximeter-collector.tar.gz
532023-05-24T20:15:25.360Zdownloaded input: /input/package/work/zones/oximeter-collector.tar.gz
542023-05-24T20:15:25.387Zdownloading input: /input/package/work/zones/propolis-server.tar.gz
552023-05-24T20:15:37.133Zdownloaded input: /input/package/work/zones/propolis-server.tar.gz
562023-05-24T20:15:37.185Zdownloading input: /input/package/work/zones/switch-asic.tar.gz
572023-05-24T20:16:26.307Zdownloaded input: /input/package/work/zones/switch-asic.tar.gz
582023-05-24T20:16:26.339Zdownloading input: /input/package/work/zones/switch-softnpu.tar.gz
592023-05-24T20:16:57.547Zdownloaded input: /input/package/work/zones/switch-softnpu.tar.gz
 
602023-05-24T20:16:57.573Zstarting task 0: "setup"
612023-05-24T20:16:57.620Z++ uname -s
622023-05-24T20:16:57.647Z+ kern=SunOS
632023-05-24T20:16:57.681Z+ case "$kern" in
642023-05-24T20:16:57.708Z+ groupadd -g 12345 build
652023-05-24T20:16:57.739Z+ useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build
662023-05-24T20:16:59.618Z+ zfs create -o mountpoint=/work rpool/work
672023-05-24T20:16:59.653Z++ awk '$2 == "/home" { print $3 }' /etc/mnttab
682023-05-24T20:16:59.679Z+ home_fs=zfs
692023-05-24T20:16:59.706Z+ [[ zfs == autofs ]]
702023-05-24T20:16:59.732Z+ mkdir -p /home/build
712023-05-24T20:16:59.759Z+ chown build:build /home/build /work
722023-05-24T20:17:01.652Z+ chmod 0700 /home/build /work
732023-05-24T20:17:01.679Zprocess exited: duration 4053 ms, exit code 0
 
742023-05-24T20:17:01.827Zstarting task 1: "authentication"
752023-05-24T20:17:01.864Zprocess exited: duration 10 ms, exit code 0
 
762023-05-24T20:17:02.839Zstarting task 2: "build"
772023-05-24T20:17:02.871Z+ trap _exit_trap EXIT
782023-05-24T20:17:02.902Z+ pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3'
792023-05-24T20:17:03.065Zsecpolicy_fs_linkdir: 0x55 = 0x55
802023-05-24T20:17:03.091Zsecpolicy_fs_linkdir+1: 0xb8 = 0x48
812023-05-24T20:17:03.123Zsecpolicy_fs_linkdir+2: 0x1 = 0x89
822023-05-24T20:17:03.150Zsecpolicy_fs_linkdir+3: 0 = 0xe5
832023-05-24T20:17:03.176Zsecpolicy_fs_linkdir+4: 0 = 0xb8
842023-05-24T20:17:03.203Zsecpolicy_fs_linkdir+5: 0 = 0x1
852023-05-24T20:17:03.230Zsecpolicy_fs_linkdir+6: 0x48 = 0x0
862023-05-24T20:17:03.256Zsecpolicy_fs_linkdir+7: 0x89 = 0x0
872023-05-24T20:17:03.281Zsecpolicy_fs_linkdir+8: 0xe5 = 0x0
882023-05-24T20:17:03.308Zsecpolicy_fs_linkdir+9: 0x48 = 0x5d
892023-05-24T20:17:03.334Zsecpolicy_fs_linkdir+0xa: 0x83 = 0xc3
902023-05-24T20:17:03.363Z+ [[ -d /opt/oxide ]]
912023-05-24T20:17:03.397Z+ pfexec mv /opt/oxide /opt/oxide-underneath
922023-05-24T20:17:03.426Z+ pfexec mkdir /opt/oxide
932023-05-24T20:17:03.455Z+ pfexec mount -F tmpfs -O swap /opt/oxide
942023-05-24T20:17:03.481Z+ [[ -d /opt/oxide-underneath ]]
952023-05-24T20:17:03.506Z+ cd /opt/oxide-underneath
962023-05-24T20:17:03.540Z+ pfexec tar ceEp@/f - .
972023-05-24T20:17:03.591Z+ cd /opt/oxide
982023-05-24T20:17:03.618Z+ pfexec tar xveEp@/f -
992023-05-24T20:17:03.644Zx ., 0 bytes, 0 tape blocks
1002023-05-24T20:17:03.676Zx ./opte, 0 bytes, 0 tape blocks
1012023-05-24T20:17:04.147Zx ./opte/bin, 0 bytes, 0 tape blocks
1022023-05-24T20:17:04.274Zx ./opte/bin/opteadm, 11097184 bytes, 21675 tape blocks
1032023-05-24T20:17:04.317Z+ pfexec rm -rf /opt/oxide-underneath
1042023-05-24T20:17:04.521Z+ pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone
1052023-05-24T20:17:04.549Z+ pfexec mkdir /opt/oxide/work
1062023-05-24T20:17:04.577Z+ pfexec chown build:build /opt/oxide/work
1072023-05-24T20:17:04.609Z+ cd /opt/oxide/work
1082023-05-24T20:17:04.636Z+ ptime -m tar xvzf /input/package/work/package.tar.gz
1092023-05-24T20:17:04.725ZDecompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'...
1102023-05-24T20:17:04.752Zx out/maghemite.tar, 86512640 bytes, 168970 tape blocks
1112023-05-24T20:17:04.779Zx out/omicron-sled-agent.tar, 76398592 bytes, 149216 tape blocks
1122023-05-24T20:17:04.808Zx out/target/test, 49 bytes, 1 tape blocks
1132023-05-24T20:17:04.837Zx out/softnpu/libsidecar_lite.so, 16649944 bytes, 32520 tape blocks
1142023-05-24T20:17:04.867Zx out/softnpu/scadm, 24255976 bytes, 47375 tape blocks
1152023-05-24T20:17:04.894Zx out/softnpu/softnpu, 23583520 bytes, 46062 tape blocks
1162023-05-24T20:17:04.922Zx package-manifest.toml, 15360 bytes, 30 tape blocks
1172023-05-24T20:17:04.952Zx smf/sled-agent/non-gimlet/config.toml, 1993 bytes, 4 tape blocks
1182023-05-24T20:17:04.983Zx target/release/omicron-package, 36516560 bytes, 71322 tape blocks
1192023-05-24T20:17:05.012Zx tools/create_virtual_hardware.sh, 4570 bytes, 9 tape blocks
1202023-05-24T20:17:06.249Zx tools/scrimlet/create-softnpu-zone.sh, 619 bytes, 2 tape blocks
1212023-05-24T20:17:06.279Zx tools/scrimlet/destroy-softnpu-zone.sh, 195 bytes, 1 tape blocks
1222023-05-24T20:17:06.310Zx tools/scrimlet/softnpu-init.sh, 1372 bytes, 3 tape blocks
1232023-05-24T20:17:06.482Zx tools/scrimlet/softnpu-zone.txt, 330 bytes, 1 tape blocks
1242023-05-24T20:17:06.511Zx tools/scrimlet/softnpu.toml, 171 bytes, 1 tape blocks
1252023-05-24T20:17:06.539Z
1262023-05-24T20:17:06.564Zreal 1.581542495
1272023-05-24T20:17:06.590Zuser 1.426987551
1282023-05-24T20:17:06.623Zsys 0.275709949
1292023-05-24T20:17:06.649Ztrap 0.000068271
1302023-05-24T20:17:06.717Ztflt 0.000000000
1312023-05-24T20:17:06.776Zdflt 0.000000000
1322023-05-24T20:17:06.849Zkflt 0.000000000
1332023-05-24T20:17:06.885Zlock 0.000000000
1342023-05-24T20:17:06.911Zslp 1.425195828
1352023-05-24T20:17:06.936Zlat 0.032929638
1362023-05-24T20:17:06.962Zstop 0.000127754
1372023-05-24T20:17:06.988Z+ cp /input/package/work/zones/clickhouse.tar.gz /input/package/work/zones/cockroachdb.tar.gz /input/package/work/zones/crucible-pantry.tar.gz /input/package/work/zones/crucible.tar.gz /input/package/work/zones/external-dns.tar.gz /input/package/work/zones/internal-dns.tar.gz /input/package/work/zones/ntp.tar.gz /input/package/work/zones/omicron-nexus.tar.gz /input/package/work/zones/oximeter-collector.tar.gz /input/package/work/zones/propolis-server.tar.gz /input/package/work/zones/switch-asic.tar.gz /input/package/work/zones/switch-softnpu.tar.gz out/
1382023-05-24T20:17:07.014Z+ mkdir tests
1392023-05-24T20:17:07.044Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1402023-05-24T20:17:07.076Z+ ptime -m gunzip
1412023-05-24T20:17:07.447Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-fdf2ceb9b8861fae
1422023-05-24T20:17:07.472Z
1432023-05-24T20:17:08.185Zreal 0.141434603
1442023-05-24T20:17:08.461Zuser 0.130533121
1452023-05-24T20:17:09.682Zsys 0.010661915
1462023-05-24T20:17:09.911Ztrap 0.000014760
1472023-05-24T20:17:09.940Ztflt 0.000000000
1482023-05-24T20:17:10.117Zdflt 0.000000000
1492023-05-24T20:17:10.143Zkflt 0.000000000
1502023-05-24T20:17:10.168Zlock 0.000000000
1512023-05-24T20:17:10.193Zslp 0.000021000
1522023-05-24T20:17:10.219Zlat 0.000022450
1532023-05-24T20:17:10.244Zstop 0.000124453
1542023-05-24T20:17:10.270Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-fdf2ceb9b8861fae
1552023-05-24T20:17:10.296Z+ chmod a+x tests/bootstrap-fdf2ceb9b8861fae
1562023-05-24T20:17:10.322Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1572023-05-24T20:17:10.349Z+ ptime -m gunzip
1582023-05-24T20:17:10.785Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1592023-05-24T20:17:10.817Z
1602023-05-24T20:17:10.843Zreal 2.053386028
1612023-05-24T20:17:10.869Zuser 1.931387990
1622023-05-24T20:17:10.896Zsys 0.121608121
1632023-05-24T20:17:11.774Ztrap 0.000158762
1642023-05-24T20:17:11.834Ztflt 0.000000000
1652023-05-24T20:17:11.869Zdflt 0.000000000
1662023-05-24T20:17:11.926Zkflt 0.000000000
1672023-05-24T20:17:11.957Zlock 0.000000000
1682023-05-24T20:17:11.984Zslp 0.000019660
1692023-05-24T20:17:12.009Zlat 0.000021140
1702023-05-24T20:17:12.035Zstop 0.000128093
1712023-05-24T20:17:12.061Z++ basename /input/ci-tools/work/end-to-end-tests/bootstrap
1722023-05-24T20:17:12.090Z+ chmod a+x tests/bootstrap
1732023-05-24T20:17:12.117Z+ for p in /input/ci-tools/work/end-to-end-tests/*.gz
1742023-05-24T20:17:12.144Z+ ptime -m gunzip
1752023-05-24T20:17:12.175Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-daffba9032038905
1762023-05-24T20:17:12.201Z
1772023-05-24T20:17:12.226Zreal 2.354630360
1782023-05-24T20:17:12.253Zuser 2.205406067
1792023-05-24T20:17:12.282Zsys 0.148843893
1802023-05-24T20:17:12.308Ztrap 0.000146664
1812023-05-24T20:17:12.334Ztflt 0.000000000
1822023-05-24T20:17:12.360Zdflt 0.000000000
1832023-05-24T20:17:12.385Zkflt 0.000000000
1842023-05-24T20:17:12.412Zlock 0.000000000
1852023-05-24T20:17:12.438Zslp 0.000022650
1862023-05-24T20:17:12.463Zlat 0.000020940
1872023-05-24T20:17:12.492Zstop 0.000126983
1882023-05-24T20:17:12.520Z++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-daffba9032038905
1892023-05-24T20:17:12.570Z+ chmod a+x tests/end_to_end_tests-daffba9032038905
1902023-05-24T20:17:12.601Z+ ptime -m pfexec ./tools/create_virtual_hardware.sh
1912023-05-24T20:17:12.628Z+++ dirname ./tools/create_virtual_hardware.sh
1922023-05-24T20:17:12.673Z++ cd ./tools
1932023-05-24T20:17:12.703Z++ pwd
1942023-05-24T20:17:12.733Z+ SOURCE_DIR=/opt/oxide/work/tools
1952023-05-24T20:17:12.759Z+ OMICRON_TOP=/opt/oxide/work/tools/..
1962023-05-24T20:17:13.447Z+ MARKER=/etc/opt/oxide/NO_INSTALL
1972023-05-24T20:17:13.756Z+ [[ -f /etc/opt/oxide/NO_INSTALL ]]
1982023-05-24T20:17:13.783Z++ dladm show-phys -p -o LINK
1992023-05-24T20:17:13.809Z++ head -1
2002023-05-24T20:17:14.248Z+ PHYSICAL_LINK=igb0
2012023-05-24T20:17:14.275Z+ echo 'Using igb0 as physical link'
2022023-05-24T20:17:14.306ZUsing igb0 as physical link
2032023-05-24T20:17:14.335Z+ ensure_run_as_root
2042023-05-24T20:17:14.366Z++ id -u
2052023-05-24T20:17:14.396Z+ [[ 0 -ne 0 ]]
2062023-05-24T20:17:15.143Z+ ensure_zpools
2072023-05-24T20:17:15.170Z+ ZPOOL_TYPES=('oxp_' 'oxi_')
2082023-05-24T20:17:15.605Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
2092023-05-24T20:17:15.630Z+ readarray -t ZPOOLS
2102023-05-24T20:17:15.656Z++ grep '"oxp_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
2112023-05-24T20:17:15.683Z++ sed 's/[ ",]//g'
2122023-05-24T20:17:15.709Z+ for ZPOOL in "${ZPOOLS[@]}"
2132023-05-24T20:17:15.735Z+ echo 'Zpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]'
2142023-05-24T20:17:15.761ZZpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]
2152023-05-24T20:17:15.786Z+ VDEV_PATH=/opt/oxide/work/tools/../oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2162023-05-24T20:17:15.812Z+ [[ -f /opt/oxide/work/tools/../oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
2172023-05-24T20:17:15.837Z+ dd if=/dev/zero of=/opt/oxide/work/tools/../oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=10G
2182023-05-24T20:17:15.865Z0+0 records in
2192023-05-24T20:17:15.893Z0+0 records out
2202023-05-24T20:17:15.925Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
2212023-05-24T20:17:16.826Z+ success 'ZFS vdev /opt/oxide/work/tools/../oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
2222023-05-24T20:17:16.855Z+ set +x
2232023-05-24T20:17:16.889ZZFS vdev /opt/oxide/work/tools/../oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
2242023-05-24T20:17:16.914Z++ zpool list -o name
2252023-05-24T20:17:16.958Z++ grep oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
2262023-05-24T20:17:16.985Z+ [[ -z '' ]]
2272023-05-24T20:17:17.021Z+ zpool create -f oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b /opt/oxide/work/tools/../oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2282023-05-24T20:17:17.070Z+ success 'ZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
2292023-05-24T20:17:17.101Z+ set +x
2302023-05-24T20:17:17.133ZZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists
2312023-05-24T20:17:17.165Z+ for ZPOOL in "${ZPOOLS[@]}"
2322023-05-24T20:17:17.191Z+ echo 'Zpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
2332023-05-24T20:17:17.218Z+ VDEV_PATH=/opt/oxide/work/tools/../oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
2342023-05-24T20:17:17.246Z+ [[ -f /opt/oxide/work/tools/../oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
2352023-05-24T20:17:17.275ZZpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]
2362023-05-24T20:17:17.301Z+ dd if=/dev/zero of=/opt/oxide/work/tools/../oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=10G
2372023-05-24T20:17:17.326Z0+0 records in
2382023-05-24T20:17:17.351Z0+0 records out
2392023-05-24T20:17:17.518Z0 bytes (0 iB) transferred in 0.000005 secs (0 iB/sec)
2402023-05-24T20:17:17.543Z+ success 'ZFS vdev /opt/oxide/work/tools/../oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
2412023-05-24T20:17:17.568Z+ set +x
2422023-05-24T20:17:17.592ZZFS vdev /opt/oxide/work/tools/../oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
2432023-05-24T20:17:17.616Z++ zpool list -o name
2442023-05-24T20:17:17.641Z++ grep oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
2452023-05-24T20:17:17.666Z+ [[ -z '' ]]
2462023-05-24T20:17:17.691Z+ zpool create -f oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 /opt/oxide/work/tools/../oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
2472023-05-24T20:17:17.715Z+ success 'ZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
2482023-05-24T20:17:17.743Z+ set +x
2492023-05-24T20:17:17.768ZZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
2502023-05-24T20:17:18.485Z+ for ZPOOL in "${ZPOOLS[@]}"
2512023-05-24T20:17:18.591Z+ echo 'Zpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]'
2522023-05-24T20:17:18.682ZZpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]
2532023-05-24T20:17:18.731Z+ VDEV_PATH=/opt/oxide/work/tools/../oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
2542023-05-24T20:17:18.830Z+ [[ -f /opt/oxide/work/tools/../oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]]
2552023-05-24T20:17:18.894Z+ dd if=/dev/zero of=/opt/oxide/work/tools/../oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=10G
2562023-05-24T20:17:18.949Z0+0 records in
2572023-05-24T20:17:18.986Z0+0 records out
2582023-05-24T20:17:19.048Z0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec)
2592023-05-24T20:17:19.074Z+ success 'ZFS vdev /opt/oxide/work/tools/../oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists'
2602023-05-24T20:17:19.099Z+ set +x
2612023-05-24T20:17:19.135ZZFS vdev /opt/oxide/work/tools/../oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists
2622023-05-24T20:17:19.159Z++ zpool list -o name
2632023-05-24T20:17:19.184Z++ grep oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
2642023-05-24T20:17:19.209Z+ [[ -z '' ]]
2652023-05-24T20:17:19.237Z+ zpool create -f oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 /opt/oxide/work/tools/../oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev
2662023-05-24T20:17:19.270Z+ success 'ZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists'
2672023-05-24T20:17:19.296Z+ set +x
2682023-05-24T20:17:19.320ZZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists
2692023-05-24T20:17:19.345Z+ for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}"
2702023-05-24T20:17:19.369Z+ readarray -t ZPOOLS
2712023-05-24T20:17:19.393Z++ grep '"oxi_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml
2722023-05-24T20:17:19.417Z++ sed 's/[ ",]//g'
2732023-05-24T20:17:19.442Z+ for ZPOOL in "${ZPOOLS[@]}"
2742023-05-24T20:17:19.466Z+ echo 'Zpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]'
2752023-05-24T20:17:20.170ZZpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]
2762023-05-24T20:17:20.197Z+ VDEV_PATH=/opt/oxide/work/tools/../oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2772023-05-24T20:17:20.225Z+ [[ -f /opt/oxide/work/tools/../oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
2782023-05-24T20:17:20.253Z+ dd if=/dev/zero of=/opt/oxide/work/tools/../oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=10G
2792023-05-24T20:17:20.278Z0+0 records in
2802023-05-24T20:17:20.304Z0+0 records out
2812023-05-24T20:17:20.332Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
2822023-05-24T20:17:20.357Z+ success 'ZFS vdev /opt/oxide/work/tools/../oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
2832023-05-24T20:17:20.383Z+ set +x
2842023-05-24T20:17:20.827ZZFS vdev /opt/oxide/work/tools/../oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
2852023-05-24T20:17:20.852Z++ zpool list -o name
2862023-05-24T20:17:20.884Z++ grep oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b
2872023-05-24T20:17:20.915Z+ [[ -z '' ]]
2882023-05-24T20:17:20.941Z+ zpool create -f oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b /opt/oxide/work/tools/../oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2892023-05-24T20:17:20.966Z+ success 'ZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
2902023-05-24T20:17:20.994Z+ set +x
2912023-05-24T20:17:21.028ZZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists
2922023-05-24T20:17:21.052Z+ for ZPOOL in "${ZPOOLS[@]}"
2932023-05-24T20:17:21.077Z+ echo 'Zpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]'
2942023-05-24T20:17:21.102ZZpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]
2952023-05-24T20:17:21.127Z+ VDEV_PATH=/opt/oxide/work/tools/../oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
2962023-05-24T20:17:21.151Z+ [[ -f /opt/oxide/work/tools/../oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]]
2972023-05-24T20:17:21.832Z+ dd if=/dev/zero of=/opt/oxide/work/tools/../oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=10G
2982023-05-24T20:17:21.857Z0+0 records in
2992023-05-24T20:17:21.894Z0+0 records out
3002023-05-24T20:17:21.921Z0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec)
3012023-05-24T20:17:21.954Z+ success 'ZFS vdev /opt/oxide/work/tools/../oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists'
3022023-05-24T20:17:21.988Z+ set +x
3032023-05-24T20:17:22.012ZZFS vdev /opt/oxide/work/tools/../oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists
3042023-05-24T20:17:22.037Z++ zpool list -o name
3052023-05-24T20:17:22.062Z++ grep oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b
3062023-05-24T20:17:22.094Z+ [[ -z '' ]]
3072023-05-24T20:17:22.133Z+ zpool create -f oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b /opt/oxide/work/tools/../oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev
3082023-05-24T20:17:22.164Z+ success 'ZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists'
3092023-05-24T20:17:22.193Z+ set +x
3102023-05-24T20:17:22.219ZZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists
3112023-05-24T20:17:22.246Z+ ensure_simulated_chelsios igb0
3122023-05-24T20:17:22.277Z+ local PHYSICAL_LINK=igb0
3132023-05-24T20:17:22.305Z+ INDICES=("0" "1")
3142023-05-24T20:17:22.335Z+ for I in "${INDICES[@]}"
3152023-05-24T20:17:22.365Z++ get_simnet_name_if_exists net0
3162023-05-24T20:17:22.389Z++ dladm show-simnet -p -o LINK net0
3172023-05-24T20:17:22.414Z++ echo ''
3182023-05-24T20:17:22.439Z+ [[ -z '' ]]
3192023-05-24T20:17:22.470Z+ dladm create-simnet -t net0
3202023-05-24T20:17:22.497Z+ dladm create-simnet -t sc0_0
3212023-05-24T20:17:22.521Z+ dladm modify-simnet -t -p net0 sc0_0
3222023-05-24T20:17:22.546Z+ dladm set-linkprop -p mtu=1600 sc0_0
3232023-05-24T20:17:22.571Z+ dladm create-simnet -t sr0_0
3242023-05-24T20:17:22.603Z+ dladm create-simnet -t scr0_0
3252023-05-24T20:17:22.628Z+ dladm modify-simnet -t -p sr0_0 scr0_0
3262023-05-24T20:17:22.653Z+ success 'Simnet net0/sc0_0/sr0_0/scr0_0 exists'
3272023-05-24T20:17:22.677Z+ set +x
3282023-05-24T20:17:22.702ZSimnet net0/sc0_0/sr0_0/scr0_0 exists
3292023-05-24T20:17:22.726Z+ for I in "${INDICES[@]}"
3302023-05-24T20:17:22.751Z++ get_simnet_name_if_exists net1
3312023-05-24T20:17:22.776Z++ dladm show-simnet -p -o LINK net1
3322023-05-24T20:17:22.801Z++ echo ''
3332023-05-24T20:17:22.826Z+ [[ -z '' ]]
3342023-05-24T20:17:23.498Z+ dladm create-simnet -t net1
3352023-05-24T20:17:23.560Z+ dladm create-simnet -t sc1_0
3362023-05-24T20:17:23.626Z+ dladm modify-simnet -t -p net1 sc1_0
3372023-05-24T20:17:23.693Z+ dladm set-linkprop -p mtu=1600 sc1_0
3382023-05-24T20:17:23.830Z+ dladm create-simnet -t sr0_1
3392023-05-24T20:17:23.919Z+ dladm create-simnet -t scr0_1
3402023-05-24T20:17:23.944Z+ dladm modify-simnet -t -p sr0_1 scr0_1
3412023-05-24T20:17:23.970Z+ success 'Simnet net1/sc1_0/sr0_1/scr0_1 exists'
3422023-05-24T20:17:23.996Z+ set +x
3432023-05-24T20:17:24.022ZSimnet net1/sc1_0/sr0_1/scr0_1 exists
3442023-05-24T20:17:24.048Z++ get_vnic_name_if_exists sc0_1
3452023-05-24T20:17:24.075Z++ dladm show-vnic -p -o LINK sc0_1
3462023-05-24T20:17:24.101Z++ echo ''
3472023-05-24T20:17:24.127Z+ [[ -z '' ]]
3482023-05-24T20:17:24.159Z+ dladm create-vnic -t sc0_1 -l igb0 -m a8:e1:de:01:70:1d
3492023-05-24T20:17:24.185Z+ success 'Vnic sc0_1 exists'
3502023-05-24T20:17:24.213Z+ set +x
3512023-05-24T20:17:24.240ZVnic sc0_1 exists
3522023-05-24T20:17:24.265Z+ ensure_softnpu_zone
3532023-05-24T20:17:24.291Z+ zoneadm list
3542023-05-24T20:17:24.317Z+ grep -q softnpu
3552023-05-24T20:17:24.343Z+ mkdir -p /softnpu-zone
3562023-05-24T20:17:24.368Z+ mkdir -p /opt/oxide/softnpu/stuff
3572023-05-24T20:17:24.397Z+ cp tools/scrimlet/softnpu.toml /opt/oxide/softnpu/stuff/
3582023-05-24T20:17:24.433Z+ cp tools/scrimlet/softnpu-init.sh /opt/oxide/softnpu/stuff/
3592023-05-24T20:17:24.461Z+ cp out/softnpu/libsidecar_lite.so /opt/oxide/softnpu/stuff/
3602023-05-24T20:17:24.486Z+ cp out/softnpu/softnpu /opt/oxide/softnpu/stuff/
3612023-05-24T20:17:25.166Z+ cp out/softnpu/scadm /opt/oxide/softnpu/stuff/
3622023-05-24T20:17:25.193Z+ zfs create -p -o mountpoint=/softnpu-zone rpool/softnpu-zone
3632023-05-24T20:17:25.219Z+ zonecfg -z softnpu -f tools/scrimlet/softnpu-zone.txt
3642023-05-24T20:17:25.244Z+ zoneadm -z softnpu install
3652023-05-24T20:17:25.270Z/softnpu-zone must not be group readable.
3662023-05-24T20:17:25.295Z/softnpu-zone must not be group executable.
3672023-05-24T20:17:25.322Z/softnpu-zone must not be world readable.
3682023-05-24T20:17:25.348Z/softnpu-zone must not be world executable.
3692023-05-24T20:17:25.373Z/softnpu-zone: changing permissions to 0700.
3702023-05-24T20:17:25.399ZINFO: omicron: installing zone softnpu @ "/softnpu-zone"...
3712023-05-24T20:17:25.425ZINFO: omicron: replicating /usr tree...
3722023-05-24T20:17:25.454ZINFO: omicron: replicating /lib tree...
3732023-05-24T20:17:25.485ZINFO: omicron: replicating /sbin tree...
3742023-05-24T20:17:25.511ZINFO: omicron: pruning SMF manifests...
3752023-05-24T20:17:25.537ZINFO: omicron: pruning global-only files...
3762023-05-24T20:17:25.563ZINFO: omicron: unpacking baseline archive...
3772023-05-24T20:17:25.588ZINFO: omicron: install complete, probably!
3782023-05-24T20:17:25.614Z+ zoneadm -z softnpu boot
3792023-05-24T20:17:25.639Z+ success 'softnpu zone exists'
3802023-05-24T20:17:25.664Z+ set +x
3812023-05-24T20:17:25.690Zsoftnpu zone exists
3822023-05-24T20:17:25.716Z+ enable_softnpu
3832023-05-24T20:17:25.742Z+ zlogin softnpu uname -a
3842023-05-24T20:17:25.768ZSunOS softnpu 5.11 stlouis-0-g7b17295d6d i86pc i386 i86pc
3852023-05-24T20:17:25.796Z+ zlogin softnpu pgrep softnpu
3862023-05-24T20:17:25.822Z+ success 'softnpu started'
3872023-05-24T20:17:25.848Z+ set +x
3882023-05-24T20:17:25.873Zsoftnpu started
3892023-05-24T20:17:25.899Z+ zlogin softnpu /stuff/softnpu /stuff/softnpu.toml
3902023-05-24T20:17:25.927Z
3912023-05-24T20:17:25.953Zreal 1.557055105
3922023-05-24T20:17:25.980Zuser 0.288500949
3932023-05-24T20:17:26.010Zsys 1.158944770
3942023-05-24T20:17:26.039Ztrap 0.000021874
3952023-05-24T20:17:26.064Ztflt 0.000000000
3962023-05-24T20:17:26.090Zdflt 0.000004610
3972023-05-24T20:17:26.115Zkflt 0.000000000
3982023-05-24T20:17:26.140Zlock 0.000000000
3992023-05-24T20:17:26.820Zslp 2.855105643
4002023-05-24T20:17:26.847Zlat 0.000882478
4012023-05-24T20:17:26.872Zstop 0.000460771
4022023-05-24T20:17:26.913Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml
4032023-05-24T20:17:26.943ZConfig {
4042023-05-24T20:17:26.974Z p4_program: "/stuff/libsidecar_lite.so",
4052023-05-24T20:17:27.006Z ports: [
4062023-05-24T20:17:27.032Z Port {
4072023-05-24T20:17:27.057Z sidecar: "sc0_0",
4082023-05-24T20:17:27.084Z scrimlet: "sr0_0",
4092023-05-24T20:17:27.109Z mtu: 1600,
4102023-05-24T20:17:27.135Z },
4112023-05-24T20:17:27.161Z Port {
4122023-05-24T20:17:27.186Z sidecar: "sc0_1",
4132023-05-24T20:17:27.212Z scrimlet: "sr0_1",
4142023-05-24T20:17:27.238Z mtu: 1500,
4152023-05-24T20:17:27.263Z },
4162023-05-24T20:17:27.289Z ],
4172023-05-24T20:17:27.318Z}
4182023-05-24T20:17:27.344Z++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml
4192023-05-24T20:17:27.370Z+ SILO_NAME=recovery
4202023-05-24T20:17:27.395Z++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml
4212023-05-24T20:17:27.421Z+ EXTERNAL_DNS_DOMAIN=oxide.test
4222023-05-24T20:17:27.448Z+ rm -f pkg/config-rss.toml
4232023-05-24T20:17:27.474Z+ TLS_NAME=recovery.sys.oxide.test
4242023-05-24T20:17:27.503Z+ openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin
4252023-05-24T20:17:27.528ZGenerating a RSA private key
4262023-05-24T20:17:27.554Z....................................................................++++
4272023-05-24T20:17:27.579Z.................++++
4282023-05-24T20:17:27.604Zwriting new private key to 'pkg/initial-tls-key.pem'
4292023-05-24T20:17:27.630Z-----
4302023-05-24T20:17:27.655Z+ tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
4312023-05-24T20:17:27.680Za pkg/initial-tls-cert.pem 2K
4322023-05-24T20:17:27.705Za pkg/initial-tls-key.pem 4K
4332023-05-24T20:17:27.731Z+ rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem
4342023-05-24T20:17:27.756Z+ rmdir pkg
4352023-05-24T20:17:27.781Z+ E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem
4362023-05-24T20:17:27.806Z+ pfexec mkdir -p /usr/oxide
4372023-05-24T20:17:28.476Z+ pfexec rm -f /usr/oxide/tcpproxy
4382023-05-24T20:17:28.506Z+ pfexec curl -sSfL -o /usr/oxide/tcpproxy http://catacomb.eng.oxide.computer:12346/tcpproxy
4392023-05-24T20:17:28.532Z+ pfexec chmod +x /usr/oxide/tcpproxy
4402023-05-24T20:17:28.561Z+ pfexec rm -f /var/svc/manifest/site/tcpproxy.xml
4412023-05-24T20:17:28.587Z+ pfexec curl -sSfL -o /var/svc/manifest/site/tcpproxy.xml http://catacomb.eng.oxide.computer:12346/tcpproxy.xml
4422023-05-24T20:17:28.619Z+ pfexec svccfg import /var/svc/manifest/site/tcpproxy.xml
4432023-05-24T20:17:28.645Z+ OMICRON_NO_UNINSTALL=1
4442023-05-24T20:17:28.682Z+ ptime -m pfexec ./target/release/omicron-package -t test install
4452023-05-24T20:17:28.709ZMay 24 20:18:13.523 DEBG target[test]: Target({"image": "standard", "machine": "non-gimlet", "switch": "softnpu"})
4462023-05-24T20:17:28.742ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/clickhouse.tar.gz, src: out/clickhouse.tar.gz
4472023-05-24T20:17:28.784ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/crucible_pantry.tar.gz, src: out/crucible-pantry.tar.gz
4482023-05-24T20:17:28.810ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/external_dns.tar.gz, src: out/external-dns.tar.gz
4492023-05-24T20:17:28.836ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/crucible.tar.gz, src: out/crucible.tar.gz
4502023-05-24T20:17:28.861ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/cockroachdb.tar.gz, src: out/cockroachdb.tar.gz
4512023-05-24T20:17:28.898ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/oximeter.tar.gz, src: out/oximeter-collector.tar.gz
4522023-05-24T20:17:28.993ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/ntp.tar.gz, src: out/ntp.tar.gz
4532023-05-24T20:17:29.020ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/sled-agent.tar, src: out/omicron-sled-agent.tar
4542023-05-24T20:17:29.045ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/propolis-server.tar.gz, src: out/propolis-server.tar.gz
4552023-05-24T20:17:29.071ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/mg-ddm.tar, src: out/maghemite.tar
4562023-05-24T20:17:29.097ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/nexus.tar.gz, src: out/omicron-nexus.tar.gz
4572023-05-24T20:17:29.124ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/switch.tar.gz, src: out/switch-softnpu.tar.gz
4582023-05-24T20:17:29.152ZMay 24 20:18:13.526 INFO Installing service, dst: /opt/oxide/internal_dns.tar.gz, src: out/internal-dns.tar.gz
4592023-05-24T20:17:29.179ZMay 24 20:18:13.702 INFO Unpacking service tarball, service_path: /opt/oxide/mg-ddm, tar_path: /opt/oxide/mg-ddm.tar
4602023-05-24T20:17:29.205ZMay 24 20:18:13.765 INFO Unpacking service tarball, service_path: /opt/oxide/sled-agent, tar_path: /opt/oxide/sled-agent.tar
4612023-05-24T20:17:29.234ZMay 24 20:18:13.807 INFO Installing boostrap service from /opt/oxide/sled-agent/pkg/manifest.xml
4622023-05-24T20:17:29.264Z
4632023-05-24T20:17:29.292Zreal 0.431029252
4642023-05-24T20:17:29.317Zuser 0.076913472
4652023-05-24T20:17:29.344Zsys 1.315881045
4662023-05-24T20:17:29.370Ztrap 0.000018459
4672023-05-24T20:17:29.395Ztflt 0.000015460
4682023-05-24T20:17:29.421Zdflt 0.000511604
4692023-05-24T20:17:29.452Zkflt 0.000000000
4702023-05-24T20:17:29.732Zlock 39.291014370
4712023-05-24T20:17:29.945Zslp 0.658447255
4722023-05-24T20:17:29.971Zlat 0.002654452
4732023-05-24T20:17:30.157Zstop 0.188251736
4742023-05-24T20:17:30.186Z+ IP_POOL_START=192.168.1.50
4752023-05-24T20:17:30.226Z+ IP_POOL_END=192.168.1.90
4762023-05-24T20:17:30.259Z++ dladm show-vnic sc0_1 -p -o macaddress
4772023-05-24T20:17:30.291Z++ sed -E 's/[ :]/&0/g; s/0([^:]{2}(:|$))/\1/g'
4782023-05-24T20:17:30.323Z+ SOFTNPU_MAC=a8:e1:de:01:70:1d
4792023-05-24T20:17:30.358Z+ pfexec ./out/softnpu/scadm --server /opt/oxide/softnpu/stuff/server --client /opt/oxide/softnpu/stuff/client standalone add-proxy-arp 192.168.1.50 192.168.1.90 a8:e1:de:01:70:1d
4802023-05-24T20:17:30.393Z+ tar xf out/omicron-sled-agent.tar pkg/config-rss.toml
4812023-05-24T20:17:30.425Z++ sed -n 's/first = "\(.*\)"/\1/p' pkg/config-rss.toml
4822023-05-24T20:17:30.458Z+ SERVICE_IP_POOL_START=192.168.1.20
4832023-05-24T20:17:30.490Z++ sed -n 's/last = "\(.*\)"/\1/p' pkg/config-rss.toml
4842023-05-24T20:17:30.523Z+ SERVICE_IP_POOL_END=192.168.1.22
4852023-05-24T20:17:30.552Z+ rm -r pkg
4862023-05-24T20:17:30.579Z+ pfexec ./out/softnpu/scadm --server /opt/oxide/softnpu/stuff/server --client /opt/oxide/softnpu/stuff/client standalone add-proxy-arp 192.168.1.20 192.168.1.22 a8:e1:de:01:70:1d
4872023-05-24T20:17:30.607Z+ pfexec ./out/softnpu/scadm --server /opt/oxide/softnpu/stuff/server --client /opt/oxide/softnpu/stuff/client standalone dump-state
4882023-05-24T20:17:30.639Zlocal v6:
4892023-05-24T20:17:30.665Zlocal v4:
4902023-05-24T20:17:30.690Zrouter v6:
4912023-05-24T20:17:30.716Zrouter v4:
4922023-05-24T20:17:30.742Zresolver v4:
4932023-05-24T20:17:30.768Zresolver v6:
4942023-05-24T20:17:30.794Znat v4:
4952023-05-24T20:17:30.825Znat_v6:
4962023-05-24T20:17:30.856Zport_mac:
4972023-05-24T20:17:30.884Zicmp_v6:
4982023-05-24T20:17:30.910Zicmp_v4:
4992023-05-24T20:17:30.937Zproxy arp:
5002023-05-24T20:17:30.963Z192.168.1.50/192.168.1.90: a8:e1:de:01:70:1d
5012023-05-24T20:17:30.988Z192.168.1.20/192.168.1.22: a8:e1:de:01:70:1d
5022023-05-24T20:17:31.014Z+ retry=0
5032023-05-24T20:17:31.047Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
5042023-05-24T20:17:31.077Z+ [[ 0 -gt 30 ]]
5052023-05-24T20:17:31.106Z+ sleep 1
5062023-05-24T20:17:31.132Z+ retry=1
5072023-05-24T20:17:31.818Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
5082023-05-24T20:17:31.864Z+ [[ 1 -gt 30 ]]
5092023-05-24T20:17:31.890Z+ sleep 1
5102023-05-24T20:17:31.916Z+ retry=2
5112023-05-24T20:17:31.950Z+ curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/'
5122023-05-24T20:17:56.504Z+ export GATEWAY_IP=192.168.1.199
5132023-05-24T20:17:56.530Z+ GATEWAY_IP=192.168.1.199
5142023-05-24T20:17:56.561Z++ dladm show-phys -m -p -o ADDRESS
5152023-05-24T20:17:56.596Z++ head -n 1
5162023-05-24T20:17:56.624Z+ export GATEWAY_MAC=18:c0:4d:d:a0:2a
5172023-05-24T20:17:56.664Z+ GATEWAY_MAC=18:c0:4d:d:a0:2a
5182023-05-24T20:17:56.696Z+ pfexec ipadm create-addr -T static -a 192.168.1.199/24 igb0/sidehatch
5192023-05-24T20:17:57.003Z+ ./tools/scrimlet/softnpu-init.sh
5202023-05-24T20:17:57.030Z+ GATEWAY_IP=192.168.1.199
5212023-05-24T20:17:57.056Z+ echo 'Using 192.168.1.199 as gateway ip'
5222023-05-24T20:17:57.101ZUsing 192.168.1.199 as gateway ip
5232023-05-24T20:17:57.138Z+ GATEWAY_MAC=18:c0:4d:d:a0:2a
5242023-05-24T20:17:57.166Z+ echo 'Using 18:c0:4d:d:a0:2a as gateway mac'
5252023-05-24T20:17:57.193ZUsing 18:c0:4d:d:a0:2a as gateway mac
5262023-05-24T20:17:57.222Z+ z_swadm link create rear0 --speed 100G --fec RS
5272023-05-24T20:17:57.252Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm link create rear0 --speed 100G --fec RS
5282023-05-24T20:17:57.282ZCreated link rear0/0
5292023-05-24T20:17:57.309Z+ z_swadm link create qsfp0 --speed 100G --fec RS
5302023-05-24T20:17:57.336Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm link create qsfp0 --speed 100G --fec RS
5312023-05-24T20:17:58.185ZCreated link qsfp0/0
5322023-05-24T20:17:58.212Z+ z_swadm addr add rear0/0 fe80::aae1:deff:fe01:701c
5332023-05-24T20:17:58.237Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm addr add rear0/0 fe80::aae1:deff:fe01:701c
5342023-05-24T20:17:58.264Z+ z_swadm addr add qsfp0/0 fe80::aae1:deff:fe01:701d
5352023-05-24T20:17:58.289Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm addr add qsfp0/0 fe80::aae1:deff:fe01:701d
5362023-05-24T20:17:58.314Z+ z_swadm addr add rear0/0 fd00:99::1
5372023-05-24T20:17:58.345Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm addr add rear0/0 fd00:99::1
5382023-05-24T20:17:58.375Z+ z_swadm route add fd00:1122:3344:0101::/64 rear0/0 fe80::aae1:deff:fe00:1
5392023-05-24T20:17:58.399Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm route add fd00:1122:3344:0101::/64 rear0/0 fe80::aae1:deff:fe00:1
5402023-05-24T20:17:58.424Z+ z_swadm arp add fe80::aae1:deff:fe00:1 a8:e1:de:00:00:01
5412023-05-24T20:17:58.448Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm arp add fe80::aae1:deff:fe00:1 a8:e1:de:00:00:01
5422023-05-24T20:17:58.473Z+ z_swadm arp add 192.168.1.199 18:c0:4d:d:a0:2a
5432023-05-24T20:17:58.501Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm arp add 192.168.1.199 18:c0:4d:d:a0:2a
5442023-05-24T20:17:58.526Z+ z_swadm route add 0.0.0.0/0 qsfp0/0 192.168.1.199
5452023-05-24T20:17:58.552Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm route add 0.0.0.0/0 qsfp0/0 192.168.1.199
5462023-05-24T20:17:58.577Z+ z_swadm link ls
5472023-05-24T20:17:58.602Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm link ls
5482023-05-24T20:17:58.626ZPort/Link Media Speed FEC Enabled State MAC
5492023-05-24T20:17:58.651Zrear0/0 Copper 100G RS false unknown a8:40:25:3b:bc:6d
5502023-05-24T20:17:58.676Zqsfp0/0 Copper 100G RS false unknown a8:40:25:3b:bc:8d
5512023-05-24T20:17:58.700Z+ z_swadm addr list
5522023-05-24T20:17:58.725Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm addr list
5532023-05-24T20:17:58.750ZLink IPv4 IPv6
5542023-05-24T20:17:58.775Zrear0/0 fd00:99::1
5552023-05-24T20:17:58.800Z fe80::aae1:deff:fe01:701c
5562023-05-24T20:17:58.825Zqsfp0/0 fe80::aae1:deff:fe01:701d
5572023-05-24T20:17:58.852Z+ z_swadm route list
5582023-05-24T20:17:58.877Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm route list
5592023-05-24T20:17:58.901ZSubnet Port Link Gateway
5602023-05-24T20:17:58.925Z0.0.0.0/0 qsfp0 0 192.168.1.199
5612023-05-24T20:17:58.950Zfd00:1122:3344:101::/64 rear0 0 fe80::aae1:deff:fe00:1
5622023-05-24T20:17:58.975Z+ z_swadm arp list
5632023-05-24T20:17:59.000Z+ pfexec zlogin oxz_switch /opt/oxide/dendrite/bin/swadm arp list
5642023-05-24T20:17:59.025Zhost mac age
5652023-05-24T20:17:59.050Z192.168.1.199 18:c0:4d:0d:a0:2a 0s
5662023-05-24T20:17:59.075Zfe80::aae1:deff:fe00:1 a8:e1:de:00:00:01 0s
5672023-05-24T20:17:59.100Z+ export RUST_BACKTRACE=1
5682023-05-24T20:17:59.125Z+ RUST_BACKTRACE=1
5692023-05-24T20:17:59.149Z+ ./tests/bootstrap
5702023-05-24T20:17:59.174Z2023-05-24 20:18:58.704258578 UTC: attempting to log into API
 
5712023-05-24T20:18:03.754Zsock console login: May 24 20:19:04 sok ip: ip: joining multicasts failed (4) on vopte0 - will use link layer broadcasts for multicast
 
5722023-05-24T20:18:12.476Z2023-05-24 20:19:13.706222171 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5732023-05-24T20:18:13.509Z2023-05-24 20:19:14.707064978 UTC: attempting to log into API
5742023-05-24T20:18:28.957Z2023-05-24 20:19:29.709186087 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5752023-05-24T20:18:29.482Z2023-05-24 20:19:30.710149896 UTC: attempting to log into API
5762023-05-24T20:18:44.607Z2023-05-24 20:19:45.711396450 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5772023-05-24T20:18:45.483Z2023-05-24 20:19:46.712281108 UTC: attempting to log into API
5782023-05-24T20:19:00.487Z2023-05-24 20:20:01.713373978 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5792023-05-24T20:19:01.742Z2023-05-24 20:20:02.714273757 UTC: attempting to log into API
5802023-05-24T20:19:16.485Z2023-05-24 20:20:17.716173489 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5812023-05-24T20:19:18.267Z2023-05-24 20:20:18.717122098 UTC: attempting to log into API
5822023-05-24T20:19:32.495Z2023-05-24 20:20:33.718360002 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5832023-05-24T20:19:33.546Z2023-05-24 20:20:34.719213539 UTC: attempting to log into API
5842023-05-24T20:19:48.820Z2023-05-24 20:20:49.720420761 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5852023-05-24T20:19:49.489Z2023-05-24 20:20:50.721299429 UTC: attempting to log into API
5862023-05-24T20:20:04.489Z2023-05-24 20:21:05.722513602 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5872023-05-24T20:20:05.607Z2023-05-24 20:21:06.723459571 UTC: attempting to log into API
5882023-05-24T20:20:21.002Z2023-05-24 20:21:21.724847539 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5892023-05-24T20:20:21.507Z2023-05-24 20:21:22.725761408 UTC: attempting to log into API
5902023-05-24T20:20:36.874Z2023-05-24 20:21:37.727937367 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5912023-05-24T20:20:37.518Z2023-05-24 20:21:38.729925214 UTC: attempting to log into API
5922023-05-24T20:20:52.499Z2023-05-24 20:21:53.732016031 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5932023-05-24T20:20:53.758Z2023-05-24 20:21:54.733992788 UTC: attempting to log into API
5942023-05-24T20:21:08.634Z2023-05-24 20:22:09.735201402 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5952023-05-24T20:21:09.685Z2023-05-24 20:22:10.736171772 UTC: attempting to log into API
5962023-05-24T20:21:24.506Z2023-05-24 20:22:25.737697353 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5972023-05-24T20:21:25.740Z2023-05-24 20:22:26.738599012 UTC: attempting to log into API
5982023-05-24T20:21:40.553Z2023-05-24 20:22:41.739917807 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
5992023-05-24T20:21:41.508Z2023-05-24 20:22:42.741773591 UTC: attempting to log into API
6002023-05-24T20:21:57.084Z2023-05-24 20:22:57.744072903 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
6012023-05-24T20:21:57.512Z2023-05-24 20:22:58.745979779 UTC: attempting to log into API
6022023-05-24T20:22:12.513Z2023-05-24 20:23:13.747306814 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
6032023-05-24T20:22:13.574Z2023-05-24 20:23:14.748195882 UTC: attempting to log into API
6042023-05-24T20:22:28.515Z2023-05-24 20:23:29.749276991 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
6052023-05-24T20:22:30.001Z2023-05-24 20:23:30.750224061 UTC: attempting to log into API
6062023-05-24T20:22:44.797Z2023-05-24 20:23:45.751233418 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
6072023-05-24T20:22:45.518Z2023-05-24 20:23:46.752206288 UTC: attempting to log into API
6082023-05-24T20:23:00.519Z2023-05-24 20:24:01.753578416 UTC: login failed: error sending request for url (http://recovery.sys.oxide.test/login/recovery/local): error trying to connect: operation timed out
6092023-05-24T20:23:01.524ZError: logging in
6102023-05-24T20:23:01.549Z
6112023-05-24T20:23:01.575ZCaused by:
6122023-05-24T20:23:02.185Z timed out after 304.050284839s
6132023-05-24T20:23:02.210Z
6142023-05-24T20:23:02.236ZStack backtrace:
6152023-05-24T20:23:02.262Z
6162023-05-24T20:23:02.288Z+ _exit_trap
6172023-05-24T20:23:02.314Z+ local status=1
6182023-05-24T20:23:02.981Z+ [[ 1 -eq 0 ]]
6192023-05-24T20:23:03.087Z+ set +o errexit
6202023-05-24T20:23:03.113Z+ set -o xtrace
6212023-05-24T20:23:03.139Z+ banner evidence
6222023-05-24T20:23:03.165Z
6232023-05-24T20:23:03.190Z ###### # # # ##### ###### # # #### ######
6242023-05-24T20:23:03.217Z # # # # # # # ## # # # #
6252023-05-24T20:23:03.243Z ##### # # # # # ##### # # # # #####
6262023-05-24T20:23:03.268Z # # # # # # # # # # # #
6272023-05-24T20:23:03.296Z # # # # # # # # ## # # #
6282023-05-24T20:23:03.322Z ###### ## # ##### ###### # # #### ######
6292023-05-24T20:23:03.348Z
6302023-05-24T20:23:03.373Z+ zoneadm list -civ
6312023-05-24T20:23:03.399Z ID NAME STATUS PATH BRAND IP
6322023-05-24T20:23:03.424Z 0 global running / ipkg shared
6332023-05-24T20:23:03.450Z 1 softnpu running /softnpu-zone omicron1 excl
6342023-05-24T20:23:03.476Z 2 oxz_internal_dns running /zone/oxz_internal_dns omicron1 excl
6352023-05-24T20:23:03.514Z 4 oxz_switch running /zone/oxz_switch omicron1 excl
6362023-05-24T20:23:03.539Z 5 oxz_ntp running /zone/oxz_ntp omicron1 excl
6372023-05-24T20:23:03.566Z 12 oxz_cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 running /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone/oxz_cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 omicron1 excl
6382023-05-24T20:23:03.592Z 13 oxz_clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 running /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone/oxz_clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 omicron1 excl
6392023-05-24T20:23:03.617Z 29 oxz_crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 running /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone/oxz_crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 omicron1 excl
6402023-05-24T20:23:03.643Z 34 oxz_crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b running /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/zone/oxz_crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b omicron1 excl
6412023-05-24T20:23:03.669Z 37 oxz_crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 running /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone/oxz_crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 omicron1 excl
6422023-05-24T20:23:03.695Z 39 oxz_oximeter running /zone/oxz_oximeter omicron1 excl
6432023-05-24T20:23:03.721Z 40 oxz_crucible_pantry running /zone/oxz_crucible_pantry omicron1 excl
6442023-05-24T20:23:03.746Z+ pfexec dladm show-phys -m
6452023-05-24T20:23:03.772ZLINK SLOT ADDRESS INUSE CLIENT
6462023-05-24T20:23:03.798Zigb0 primary 18:c0:4d:d:a0:2a yes igb0
6472023-05-24T20:23:03.823Zigb1 primary 18:c0:4d:d:a0:2b no --
6482023-05-24T20:23:03.849Ztfpkt0 primary 2:0:0:0:0:0 no --
6492023-05-24T20:23:03.875Z+ pfexec dladm show-link
6502023-05-24T20:23:03.904ZLINK CLASS MTU STATE BRIDGE OVER
6512023-05-24T20:23:03.929Zigb0 phys 1500 up -- --
6522023-05-24T20:23:03.955Zigb1 phys 1500 unknown -- --
6532023-05-24T20:23:04.632Ztfpkt0 phys 1500 down -- --
6542023-05-24T20:23:04.658Znet0 simnet 9000 up -- sc0_0
6552023-05-24T20:23:04.684Zsc0_0 simnet 1600 up -- net0
6562023-05-24T20:23:04.710Zsr0_0 simnet 9000 up -- scr0_0
6572023-05-24T20:23:04.735Zscr0_0 simnet 9000 up -- sr0_0
6582023-05-24T20:23:04.761Znet1 simnet 9000 up -- sc1_0
6592023-05-24T20:23:04.787Zsc1_0 simnet 1600 up -- net1
6602023-05-24T20:23:04.812Zsr0_1 simnet 9000 up -- scr0_1
6612023-05-24T20:23:04.838Zscr0_1 simnet 9000 up -- sr0_1
6622023-05-24T20:23:04.864Zsc0_1 vnic 1500 up -- igb0
6632023-05-24T20:23:04.891Zbootstrap_stub0 etherstub 9000 up -- --
6642023-05-24T20:23:04.917Zbootstrap0 vnic 9000 up -- bootstrap_stub0
6652023-05-24T20:23:04.944Zunderlay_stub0 etherstub 9000 up -- --
6662023-05-24T20:23:04.970Zunderlay0 vnic 9000 up -- underlay_stub0
6672023-05-24T20:23:04.996ZoxControlService1 vnic 9000 up -- underlay_stub0
6682023-05-24T20:23:05.024Zopte0 misc 1500 up -- --
6692023-05-24T20:23:05.049Zvopte0 vnic 1500 up -- opte0
6702023-05-24T20:23:05.075ZoxBootstrap2 vnic 9000 up -- bootstrap_stub0
6712023-05-24T20:23:05.101ZoxControlService3 vnic 9000 up -- underlay_stub0
6722023-05-24T20:23:05.342ZoxControlService4 vnic 9000 up -- underlay_stub0
6732023-05-24T20:23:05.371ZoxControlService21 vnic 9000 up -- underlay_stub0
6742023-05-24T20:23:05.405ZLINK CLASS MTU STATE BRIDGE OVER
6752023-05-24T20:23:05.430ZoxControlService22 vnic 9000 up -- underlay_stub0
6762023-05-24T20:23:05.456ZoxControlService48 vnic 9000 up -- underlay_stub0
6772023-05-24T20:23:05.481ZoxControlService64 vnic 9000 up -- underlay_stub0
6782023-05-24T20:23:05.507ZoxControlService75 vnic 9000 up -- underlay_stub0
6792023-05-24T20:23:05.532ZoxControlService77 vnic 9000 up -- underlay_stub0
6802023-05-24T20:23:05.558ZoxControlService78 vnic 9000 up -- underlay_stub0
6812023-05-24T20:23:05.583Z+ pfexec dladm show-vnic
6822023-05-24T20:23:06.283ZLINK OVER SPEED MACADDRESS MACADDRTYPE VID
6832023-05-24T20:23:06.309Zsc0_1 igb0 1000 a8:e1:de:1:70:1d fixed 0
6842023-05-24T20:23:06.343Zbootstrap0 bootstrap_stub0 0 2:8:20:88:e7:15 random 0
6852023-05-24T20:23:06.369Zunderlay0 underlay_stub0 0 2:8:20:d7:81:57 random 0
6862023-05-24T20:23:06.394ZoxControlService1 underlay_stub0 0 2:8:20:79:f7:25 random 0
6872023-05-24T20:23:06.419Zvopte0 opte0 0 a8:40:25:ff:d5:51 fixed 0
6882023-05-24T20:23:06.444ZoxBootstrap2 bootstrap_stub0 0 2:8:20:47:1e:d9 random 0
6892023-05-24T20:23:06.469ZoxControlService3 underlay_stub0 0 2:8:20:ac:b3:d7 random 0
6902023-05-24T20:23:06.494ZoxControlService4 underlay_stub0 0 2:8:20:ce:66:4f random 0
6912023-05-24T20:23:06.519ZoxControlService21 underlay_stub0 0 2:8:20:a0:aa:2d random 0
6922023-05-24T20:23:06.544ZoxControlService22 underlay_stub0 0 2:8:20:0:a0:ad random 0
6932023-05-24T20:23:06.570ZoxControlService48 underlay_stub0 0 2:8:20:5b:d6:1b random 0
6942023-05-24T20:23:06.595ZoxControlService64 underlay_stub0 0 2:8:20:b2:cb:d5 random 0
6952023-05-24T20:23:06.620ZoxControlService75 underlay_stub0 0 2:8:20:e7:c0:8a random 0
6962023-05-24T20:23:06.645ZoxControlService77 underlay_stub0 0 2:8:20:5e:cb:76 random 0
6972023-05-24T20:23:06.670ZoxControlService78 underlay_stub0 0 2:8:20:7b:a5:4f random 0
6982023-05-24T20:23:06.695Z+ pfexec ipadm
6992023-05-24T20:23:06.720ZADDROBJ TYPE STATE ADDR
7002023-05-24T20:23:06.745Zlo0/v4 static ok 127.0.0.1/8
7012023-05-24T20:23:06.771Zigb0/dhcp dhcp ok 10.151.2.100/24
7022023-05-24T20:23:06.803Zigb0/sidehatch static ok 192.168.1.199/24
7032023-05-24T20:23:06.828Zlo0/v6 static ok ::1/128
7042023-05-24T20:23:06.854Znet0/ll addrconf ok fe80::1cea:83ff:fe87:4134%net0/10
7052023-05-24T20:23:06.879Znet1/ll addrconf ok fe80::289b:f5ff:fe5c:6a64%net1/10
7062023-05-24T20:23:06.911Zbootstrap0/ll addrconf ok fe80::8:20ff:fe88:e715%bootstrap0/10
7072023-05-24T20:23:06.944Zbootstrap0/bootstrap6 static ok fdb0:18c0:4d0d:a02a::1/64
7082023-05-24T20:23:06.970Zunderlay0/ll addrconf ok fe80::8:20ff:fed7:8157%underlay0/10
7092023-05-24T20:23:07.012Zunderlay0/sled6 static ok fd00:1122:3344:101::1/64
7102023-05-24T20:23:07.044Zunderlay0/internaldns static ok fd00:1122:3344:1::2/64
7112023-05-24T20:23:07.078Z+ pfexec netstat -rncva
7122023-05-24T20:23:07.112Z
7132023-05-24T20:23:07.137ZIRE Table: IPv4
7142023-05-24T20:23:07.162Z Destination Mask Gateway Device MTU Ref Flg Out In/Fwd
7152023-05-24T20:23:07.188Z-------------------- --------------- -------------------- ------ ----- --- --- ------ ------
7162023-05-24T20:23:07.212Z0.0.0.0 255.255.255.255 -- igb0 1500 1 UHb 0 0
7172023-05-24T20:23:07.238Zdefault 0.0.0.0 10.151.2.1 igb0 1500 3 UG 473172 48
7182023-05-24T20:23:07.263Z10.0.0.0 255.255.255.255 -- igb0 1500 1 UHb 0 0
7192023-05-24T20:23:07.926Z10.151.2.0 255.255.255.255 -- igb0 1500 1 UHb 0 0
7202023-05-24T20:23:07.954Z10.151.2.0/24 255.255.255.0 10.151.2.100 igb0 1500 3 U 0 0
7212023-05-24T20:23:07.984Z10.151.2.1/32 255.255.255.255 10.151.2.100 igb0 1500 3 UC 0 0
7222023-05-24T20:23:08.009Z10.151.2.100 255.255.255.255 -- igb0 1500 2 UHL 0 915546
7232023-05-24T20:23:08.039Z10.151.2.255 255.255.255.255 -- igb0 1500 1 UHb 0 0
7242023-05-24T20:23:08.067Z10.255.255.255 255.255.255.255 -- igb0 1500 1 UHb 0 0
7252023-05-24T20:23:08.187Z127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 70 70
7262023-05-24T20:23:08.213Z192.168.1.0 255.255.255.255 -- igb0 1500 1 UHb 0 0
7272023-05-24T20:23:08.238Z192.168.1.0/24 255.255.255.0 192.168.1.199 igb0 1500 3 U 57 0
7282023-05-24T20:23:08.263Z192.168.1.20/32 255.255.255.255 192.168.1.199 igb0 1500 2 UC 57 0
7292023-05-24T20:23:08.289Z192.168.1.199 255.255.255.255 -- igb0 1500 2 UHL 0 0
7302023-05-24T20:23:08.314Z192.168.1.255 255.255.255.255 -- igb0 1500 1 UHb 0 0
7312023-05-24T20:23:08.340Z255.255.255.255 255.255.255.255 -- igb0 1500 1 UHb 0 0
7322023-05-24T20:23:08.365Z
7332023-05-24T20:23:08.390ZIRE Table: IPv6
7342023-05-24T20:23:08.416Z Destination/Mask Gateway If MTU Ref Flags Out In/Fwd
7352023-05-24T20:23:08.442Z--------------------------- --------------------------- ----- ----- --- ----- ------ ------
7362023-05-24T20:23:08.467Z::1 ::1 lo0 8252 2 UH 84 84
7372023-05-24T20:23:08.493Zfdb0:18c0:4d0d:a02a::1 -- bootstrap0 9000 2 UHL 7 7
7382023-05-24T20:23:08.518Zfe80::289b:f5ff:fe5c:6a64 -- net1 9000 2 UHL 0 0
7392023-05-24T20:23:08.544Zfe80::8:20ff:fe88:e715 -- bootstrap0 9000 2 UHL 0 0
7402023-05-24T20:23:08.570Zfe80::8:20ff:fed7:8157 -- underlay0 9000 2 UHL 0 0
7412023-05-24T20:23:08.596Zfd00:1122:3344:101::5 fd00:1122:3344:101::1 underlay0 9000 2 UC 6 6
7422023-05-24T20:23:08.622Zfe80::1cea:83ff:fe87:4134 -- net0 9000 2 UHL 0 0
7432023-05-24T20:23:08.648Zfd00:1122:3344:101::4 fd00:1122:3344:101::1 underlay0 9000 6 UC 809 0
7442023-05-24T20:23:08.673Zfd00:1122:3344:101::1 -- underlay0 9000 4 UHL 498 518
7452023-05-24T20:23:08.705Zfd00:1122:3344:1::1 fd00:1122:3344:1::2 underlay0 9000 2 UC 405 42
7462023-05-24T20:23:08.739Zfd00:1122:3344:1::2 -- underlay0 9000 2 UHL 0 365
7472023-05-24T20:23:08.774Zfd00:1122:3344:101::2 fd00:1122:3344:101::1 underlay0 9000 2 UC 75 36
7482023-05-24T20:23:08.820Zfdb0:18c0:4d0d:a02a::/64 fdb0:18c0:4d0d:a02a::1 bootstrap0 9000 2 U 0 0
7492023-05-24T20:23:08.852Zfd00:1122:3344:1::/64 fd00:1122:3344:1::2 underlay0 9000 3 U 405 42
7502023-05-24T20:23:08.884Zfd00:1122:3344:101::/64 fd00:1122:3344:101::1 underlay0 9000 5 U 890 42
7512023-05-24T20:23:08.918Zfe80::/10 fe80::8:20ff:fed7:8157 underlay0 9000 2 U 0 0
7522023-05-24T20:23:09.410Zfe80::/10 fe80::8:20ff:fe88:e715 bootstrap0 9000 2 U 0 0
7532023-05-24T20:23:09.445Zfe80::/10 fe80::289b:f5ff:fe5c:6a64 net1 9000 2 U 0 0
7542023-05-24T20:23:09.486Zfe80::/10 fe80::1cea:83ff:fe87:4134 net0 9000 2 U 0 0
7552023-05-24T20:23:09.684Z+ pfexec netstat -anu
7562023-05-24T20:23:09.709Z
7572023-05-24T20:23:09.735ZUDP: IPv4
7582023-05-24T20:23:09.760Z Local Address Remote Address User Pid Command State
7592023-05-24T20:23:09.785Z-------------------- -------------------- -------- ------ -------------- ----------
7602023-05-24T20:23:09.811Z *.* root 78 in.mpathd Unbound
7612023-05-24T20:23:09.837Z *.* root 78 in.mpathd Unbound
7622023-05-24T20:23:09.862Z *.68 root 211 dhcpagent Idle
7632023-05-24T20:23:09.888Z *.546 root 211 dhcpagent Idle
7642023-05-24T20:23:09.921Z10.151.2.100.68 root 211 dhcpagent Idle
7652023-05-24T20:23:09.947Z *.111 daemon 730 rpcbind Idle
7662023-05-24T20:23:09.972Z *.* daemon 730 rpcbind Unbound
7672023-05-24T20:23:09.998Z *.57332 daemon 730 rpcbind Idle
7682023-05-24T20:23:10.023Z *.111 daemon 730 rpcbind Idle
7692023-05-24T20:23:10.049Z *.* daemon 730 rpcbind Unbound
7702023-05-24T20:23:10.079Z *.45320 daemon 730 rpcbind Idle
7712023-05-24T20:23:10.196Z *.* root 741 in.ndpd Unbound
7722023-05-24T20:23:10.222Z
7732023-05-24T20:23:10.248ZUDP: IPv6
7742023-05-24T20:23:10.274Z Local Address Remote Address User Pid Command State If
7752023-05-24T20:23:10.300Z--------------------------------- --------------------------------- -------- ------ -------------- ---------- -----
7762023-05-24T20:23:10.331Z *.* root 78 in.mpathd Unbound
7772023-05-24T20:23:10.357Z *.546 root 211 dhcpagent Idle
7782023-05-24T20:23:10.391Z *.111 daemon 730 rpcbind Idle
7792023-05-24T20:23:10.441Z *.* daemon 730 rpcbind Unbound
7802023-05-24T20:23:10.475Z *.57332 daemon 730 rpcbind Idle
7812023-05-24T20:23:10.503Z *.* root 741 in.ndpd Unbound
7822023-05-24T20:23:10.531Zff02::dd.3549 root 1287 ddmd Idle net0
7832023-05-24T20:23:10.565Zff02::dd.3549 root 1287 ddmd Idle net1
7842023-05-24T20:23:10.600Zfe80::1cea:83ff:fe87:4134.3549 root 1287 ddmd Idle net0
7852023-05-24T20:23:10.626Zfe80::289b:f5ff:fe5c:6a64.3549 root 1287 ddmd Idle net1
7862023-05-24T20:23:10.652Z
7872023-05-24T20:23:11.333ZTCP: IPv4
7882023-05-24T20:23:11.360Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State
7892023-05-24T20:23:11.386Z-------------------- -------------------- -------- ------ -------------- ------ ------ ------ ------ -----------
7902023-05-24T20:23:11.412Z127.0.0.1.5999 *.* root 78 in.mpathd 0 0 128000 0 LISTEN
7912023-05-24T20:23:11.438Z127.0.0.1.4999 *.* root 211 dhcpagent 0 0 128000 0 LISTEN
7922023-05-24T20:23:11.472Z *.111 *.* daemon 730 rpcbind 0 0 128000 0 LISTEN
7932023-05-24T20:23:11.498Z *.* *.* daemon 730 rpcbind 0 0 128000 0 IDLE
7942023-05-24T20:23:11.524Z *.111 *.* daemon 730 rpcbind 0 0 128000 0 LISTEN
7952023-05-24T20:23:11.554Z *.* *.* daemon 730 rpcbind 0 0 128000 0 IDLE
7962023-05-24T20:23:11.580Z *.54321 *.* root 1038 tcpproxy 0 0 128000 0 LISTEN
7972023-05-24T20:23:11.607Z10.151.2.100.37052 54.214.71.208.443 root 813 agent 256296 371 128872 0 ESTABLISHED
7982023-05-24T20:23:11.637Z
7992023-05-24T20:23:11.663ZTCP: IPv6
8002023-05-24T20:23:11.689Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State If
8012023-05-24T20:23:11.715Z--------------------------------- --------------------------------- -------- ------ -------------- ------ ------ ------ ------ ----------- -----
8022023-05-24T20:23:11.741Z::1.5999 *.* root 78 in.mpathd 0 0 128000 0 LISTEN
8032023-05-24T20:23:11.767Z *.111 *.* daemon 730 rpcbind 0 0 128000 0 LISTEN
8042023-05-24T20:23:11.794Z *.* *.* daemon 730 rpcbind 0 0 128000 0 IDLE
8052023-05-24T20:23:11.821Z *.54321 *.* root 1038 tcpproxy 0 0 128000 0 LISTEN
8062023-05-24T20:23:11.847Z::1.8000 *.* root 1287 ddmd 0 0 128000 0 LISTEN
8072023-05-24T20:23:11.873Zfdb0:18c0:4d0d:a02a::1.80 *.* root 1097 sled-agent 0 0 128000 0 LISTEN
8082023-05-24T20:23:11.899Zfdb0:18c0:4d0d:a02a::1.12346 *.* root 1097 sled-agent 0 0 128000 0 LISTEN
8092023-05-24T20:23:11.926Zfd00:1122:3344:101::1.12345 *.* root 1097 sled-agent 0 0 128000 0 LISTEN
8102023-05-24T20:23:11.952Zfd00:1122:3344:101::1.62083 fd00:1122:3344:101::4.12221 root 1097 sled-agent 0 0 134100 0 SYN_SENT
8112023-05-24T20:23:11.978Zfd00:1122:3344:101::1.51451 fd00:1122:3344:101::1.12345 root 1097 sled-agent 65440 0 139264 0 ESTABLISHED
8122023-05-24T20:23:12.004Zfd00:1122:3344:101::1.12345 fd00:1122:3344:101::1.51451 root 1097 sled-agent 139060 0 130880 0 ESTABLISHED
8132023-05-24T20:23:12.047Zfd00:1122:3344:101::1.60474 fd00:1122:3344:101::4.12221 root 1097 sled-agent 0 0 134100 0 SYN_SENT
8142023-05-24T20:23:12.077Zfd00:1122:3344:101::1.62242 fd00:1122:3344:101::4.12221 root 1097 sled-agent 0 0 134100 0 SYN_SENT
8152023-05-24T20:23:12.105Zfd00:1122:3344:101::1.61082 fd00:1122:3344:101::4.12221 root 1097 sled-agent 0 0 134100 0 SYN_SENT
8162023-05-24T20:23:12.132Z
8172023-05-24T20:23:12.162ZActive UNIX domain sockets
8182023-05-24T20:23:12.190ZAddress Type User Pid Command Local Address Remote Address
8192023-05-24T20:23:12.223Z---------------- ---------- -------- ------ -------------- --------------------------------------- ---------------------------------------
8202023-05-24T20:23:12.254Zfffffe8613c9e160 stream-ord root 27961 zoneadmd /var/run/zones/oxz_crucible_pantry.cons
8212023-05-24T20:23:12.281Zfffffe8613c9ec40 stream-ord root 27259 zoneadmd /var/run/zones/oxz_oximeter.console_soc
8222023-05-24T20:23:12.313Zfffffe85f06db0e8 stream-ord root 25933 zoneadmd /var/run/zones/oxz_crucible_oxp_e4b4dc8
8232023-05-24T20:23:12.964Zfffffe85f06dbbc8 stream-ord root 8995 zoneadmd /var/run/zones/oxz_cockroachdb_oxp_f4b4
8242023-05-24T20:23:12.993Zfffffe85ed8ab0e0 stream-ord root 9707 zoneadmd /var/run/zones/oxz_clickhouse_oxp_f4b4d
8252023-05-24T20:23:13.020Zfffffe85971c2b80 stream-ord root 4181 zoneadmd /var/run/zones/oxz_ntp.console_sock
8262023-05-24T20:23:13.049Zfffffe85c730a078 stream-ord root 3236 zoneadmd /var/run/zones/oxz_switch.console_sock
8272023-05-24T20:23:13.093Zfffffe85c34c2b48 stream-ord root 1734 zoneadmd /var/run/zones/oxz_internal_dns.console
8282023-05-24T20:23:13.126Zfffffe85a4a387a0 stream-ord root 23887 zoneadmd /var/run/zones/oxz_crucible_oxp_d462a7f
8292023-05-24T20:23:13.157Zfffffe85a8afa058 stream-ord root 1287 ddmd
8302023-05-24T20:23:13.194Zfffffe85a8afa058 stream-ord root 1287 ddmd
8312023-05-24T20:23:13.312Zfffffe85a8afa3f8 stream-ord root 1287 ddmd
8322023-05-24T20:23:13.344Zfffffe85a8af0b20 stream-ord root 1097 sled-agent
8332023-05-24T20:23:13.376Zfffffe85a8af0b20 stream-ord root 1097 sled-agent
8342023-05-24T20:23:13.413Zfffffe85a8d24038 stream-ord root 1097 sled-agent
8352023-05-24T20:23:13.449Zfffffe8586fc83c0 stream-ord root 813 agent
8362023-05-24T20:23:13.486Zfffffe8586fc83c0 stream-ord root 813 agent
8372023-05-24T20:23:13.523Zfffffe8586fc8760 stream-ord root 813 agent
8382023-05-24T20:23:13.553Zfffffe8586fc8b00 stream-ord root 20471 zoneadmd /var/run/zones/oxz_crucible_oxp_f4b4dc8
8392023-05-24T20:23:13.579Zfffffe8586e67018 stream-ord root 947 zoneadmd /var/run/zones/softnpu.console_sock
8402023-05-24T20:23:13.605Zfffffe8586da9750 dgram root 741 in.ndpd /var/run/in.ndpd_mib
8412023-05-24T20:23:13.640Zfffffe8586da9af0 stream-ord root 741 in.ndpd /var/run/in.ndpd_ipadm
8422023-05-24T20:23:13.666Zfffffe85845123a8 stream-ord root 768 inetd /var/run/.inetd.uds
8432023-05-24T20:23:13.692Z+ pfexec arp -an
8442023-05-24T20:23:13.718ZNet to Media Table: IPv4
8452023-05-24T20:23:13.745ZDevice IP Address Mask Flags Phys Addr
8462023-05-24T20:23:13.771Z------ -------------------- --------------- -------- ---------------
8472023-05-24T20:23:13.797Zigb0 192.168.1.20 255.255.255.255 a8:e1:de:01:70:1d
8482023-05-24T20:23:13.823Zigb0 10.151.2.1 255.255.255.255 aa:00:04:00:ca:fe
8492023-05-24T20:23:13.849Zigb0 192.168.1.199 255.255.255.255 SPLA 18:c0:4d:0d:a0:2a
8502023-05-24T20:23:13.876Zigb0 10.151.2.100 255.255.255.255 SPLA 18:c0:4d:0d:a0:2a
8512023-05-24T20:23:13.902Z+ pfexec ./out/softnpu/scadm --server /opt/oxide/softnpu/stuff/server --client /opt/oxide/softnpu/stuff/client standalone dump-state
8522023-05-24T20:23:13.928Zlocal v6:
8532023-05-24T20:23:14.217Zfd00:99::1
8542023-05-24T20:23:14.400Zfe80::aae1:deff:fe01:701c
8552023-05-24T20:23:14.438Zfe80::aae1:deff:fe01:701d
8562023-05-24T20:23:14.469Zlocal v4:
8572023-05-24T20:23:14.496Zrouter v6:
8582023-05-24T20:23:14.671Zfd00:1122:3344:101::/64 -> fe80::aae1:deff:fe00:1 (1)
8592023-05-24T20:23:14.701Zrouter v4:
8602023-05-24T20:23:14.727Z0.0.0.0/0 -> 192.168.1.199 (2)
8612023-05-24T20:23:14.753Zresolver v4:
8622023-05-24T20:23:14.779Z192.168.1.199 -> 18:c0:4d:0d:a0:2a
8632023-05-24T20:23:14.805Zresolver v6:
8642023-05-24T20:23:14.831Zfe80::aae1:deff:fe00:1 -> a8:e1:de:00:00:01
8652023-05-24T20:23:14.857Znat v4:
8662023-05-24T20:23:14.883Z192.168.1.22 0/16383 -> fd00:1122:3344:101::1 100/a8:40:25:ff:d5:51
8672023-05-24T20:23:14.909Znat_v6:
8682023-05-24T20:23:14.936Zport_mac:
8692023-05-24T20:23:14.965Z2: a8:40:25:3b:bc:8d
8702023-05-24T20:23:15.086Z1: a8:40:25:3b:bc:6d
8712023-05-24T20:23:15.122Zicmp_v6:
8722023-05-24T20:23:15.148Zicmp_v4:
8732023-05-24T20:23:15.175Zproxy arp:
8742023-05-24T20:23:15.202Z192.168.1.50/192.168.1.90: a8:e1:de:01:70:1d
8752023-05-24T20:23:15.228Z192.168.1.20/192.168.1.22: a8:e1:de:01:70:1d
8762023-05-24T20:23:15.254Z+ pfexec /opt/oxide/opte/bin/opteadm list-ports
8772023-05-24T20:23:15.281ZLINK MAC ADDRESS IPv4 ADDRESS EXTERNAL IPv4 IPv6 ADDRESS EXTERNAL IPv6 STATE
8782023-05-24T20:23:15.307Zopte0 A8:40:25:FF:D5:51 172.30.2.5 None None None running
8792023-05-24T20:23:15.333Z++ pfexec /opt/oxide/opte/bin/opteadm list-ports
8802023-05-24T20:23:15.359Z++ tail +2
8812023-05-24T20:23:15.385Z++ awk '{ print $1; }'
8822023-05-24T20:23:15.412Z+ PORTS=opte0
8832023-05-24T20:23:15.438Z+ for p in $PORTS
8842023-05-24T20:23:15.465Z++ pfexec /opt/oxide/opte/bin/opteadm list-layers -p opte0
8852023-05-24T20:23:15.491Z++ tail +2
8862023-05-24T20:23:15.517Z++ awk '{ print $1; }'
8872023-05-24T20:23:15.543Z+ LAYERS='gateway
8882023-05-24T20:23:15.569Zfirewall
8892023-05-24T20:23:15.595Zrouter
8902023-05-24T20:23:15.622Znat
8912023-05-24T20:23:15.648Zoverlay'
8922023-05-24T20:23:16.327Z+ for l in $LAYERS
8932023-05-24T20:23:16.353Z+ pfexec /opt/oxide/opte/bin/opteadm dump-layer -p opte0 gateway
8942023-05-24T20:23:16.392ZLayer gateway
8952023-05-24T20:23:16.418Z======================================================================
8962023-05-24T20:23:16.445ZInbound Flows
8972023-05-24T20:23:16.472Z----------------------------------------------------------------------
8982023-05-24T20:23:16.499ZPROTO SRC IP SPORT DST IP DPORT HITS ACTION
8992023-05-24T20:23:16.525Z
9002023-05-24T20:23:16.551ZOutbound Flows
9012023-05-24T20:23:16.577Z----------------------------------------------------------------------
9022023-05-24T20:23:16.603ZPROTO SRC IP SPORT DST IP DPORT HITS ACTION
9032023-05-24T20:23:16.632Z
9042023-05-24T20:23:16.658ZInbound Rules
9052023-05-24T20:23:16.684Z----------------------------------------------------------------------
9062023-05-24T20:23:16.710ZID PRI HITS PREDICATES ACTION
9072023-05-24T20:23:16.736Z0 1000 0 inner.ip.dst=172.30.2.5 "Static: ether.src=A8:40:25:FF:77:77"
9082023-05-24T20:23:16.762Z inner.ether.dst=A8:40:25:FF:D5:51
9092023-05-24T20:23:16.788Z
9102023-05-24T20:23:16.815ZDEF -- 0 -- "deny"
9112023-05-24T20:23:16.840Z
9122023-05-24T20:23:16.866ZOutbound Rules
9132023-05-24T20:23:16.893Z----------------------------------------------------------------------
9142023-05-24T20:23:16.918ZID PRI HITS PREDICATES ACTION
9152023-05-24T20:23:16.944Z3 1 0 inner.ether.src=A8:40:25:FF:D5:51 "Hairpin: ICMPv4 Echo Reply (A8:40:25:FF:77:77,172.30.2.1) => (A8:40:25:FF:D5:51,172.30.2.5)"
9162023-05-24T20:23:16.970Z inner.ether.dst=A8:40:25:FF:77:77
9172023-05-24T20:23:16.997Z inner.ip.src=172.30.2.5
9182023-05-24T20:23:17.023Z inner.ip.dst=172.30.2.1
9192023-05-24T20:23:17.049Z inner.ip.proto=ICMP
9202023-05-24T20:23:17.075Z icmp.msg_type=echo request
9212023-05-24T20:23:17.101Z
9222023-05-24T20:23:17.127Z2 1 1 inner.ether.dst=FF:FF:FF:FF:FF:FF "Hairpin: DHCPv4 ACK: 172.30.2.5"
9232023-05-24T20:23:17.153Z inner.ether.src=A8:40:25:FF:D5:51
9242023-05-24T20:23:17.179Z inner.ip.src=0.0.0.0
9252023-05-24T20:23:17.206Z inner.ip.dst=255.255.255.255
9262023-05-24T20:23:17.232Z inner.ip.proto=UDP
9272023-05-24T20:23:17.258Z inner.ulp.dst=67
9282023-05-24T20:23:17.283Z inner.ulp.src=68
9292023-05-24T20:23:17.310Z dhcp.msg_type=Request
9302023-05-24T20:23:17.934Z
9312023-05-24T20:23:17.961Z1 1 1 inner.ether.dst=FF:FF:FF:FF:FF:FF "Hairpin: DHCPv4 OFFER: 172.30.2.5"
9322023-05-24T20:23:17.987Z inner.ether.src=A8:40:25:FF:D5:51
9332023-05-24T20:23:18.013Z inner.ip.src=0.0.0.0
9342023-05-24T20:23:18.039Z inner.ip.dst=255.255.255.255
9352023-05-24T20:23:18.065Z inner.ip.proto=UDP
9362023-05-24T20:23:18.091Z inner.ulp.dst=67
9372023-05-24T20:23:18.117Z inner.ulp.src=68
9382023-05-24T20:23:18.143Z dhcp.msg_type=Discover
9392023-05-24T20:23:18.169Z
9402023-05-24T20:23:18.195Z0 1 14 inner.ether.ether_type=ARP "Handle Packet"
9412023-05-24T20:23:18.221Z inner.ether.dst=FF:FF:FF:FF:FF:FF
9422023-05-24T20:23:18.247Z inner.ether.src=A8:40:25:FF:D5:51
9432023-05-24T20:23:18.273Z
9442023-05-24T20:23:18.299Z4 1000 24 inner.ip.src=172.30.2.5 "Meta: vpc-meta"
9452023-05-24T20:23:18.386Z inner.ether.src=A8:40:25:FF:D5:51
9462023-05-24T20:23:18.412Z
9472023-05-24T20:23:18.438ZDEF -- 18 -- "deny"
9482023-05-24T20:23:18.465Z
9492023-05-24T20:23:18.491Z+ for l in $LAYERS
9502023-05-24T20:23:18.518Z+ pfexec /opt/oxide/opte/bin/opteadm dump-layer -p opte0 firewall
9512023-05-24T20:23:18.545ZLayer firewall
9522023-05-24T20:23:18.574Z======================================================================
9532023-05-24T20:23:18.618ZInbound Flows
9542023-05-24T20:23:18.644Z----------------------------------------------------------------------
9552023-05-24T20:23:18.670ZPROTO SRC IP SPORT DST IP DPORT HITS ACTION
9562023-05-24T20:23:18.696ZUDP 1.1.1.1 53 172.30.2.5 57669 0 no-op
9572023-05-24T20:23:18.722ZUDP 1.1.1.1 53 172.30.2.5 62157 0 no-op
9582023-05-24T20:23:18.748ZUDP 9.9.9.9 53 172.30.2.5 41992 0 no-op
9592023-05-24T20:23:18.774ZUDP 9.9.9.9 53 172.30.2.5 52435 0 no-op
9602023-05-24T20:23:18.800Z
9612023-05-24T20:23:18.828ZOutbound Flows
9622023-05-24T20:23:18.857Z----------------------------------------------------------------------
9632023-05-24T20:23:18.887ZPROTO SRC IP SPORT DST IP DPORT HITS ACTION
9642023-05-24T20:23:18.913ZUDP 172.30.2.5 41992 9.9.9.9 53 0 no-op
9652023-05-24T20:23:19.567ZUDP 172.30.2.5 52435 9.9.9.9 53 0 no-op
 
9662023-05-24T20:23:19.591ZMay 24 20:24:19 sock ip: ip: joining multicasts failed (4) on vopte1 - will use link layer broadcasts for multicast
 
9672023-05-24T20:23:19.597ZUDP 172.30.2.5 57669 1.1.1.1 53 0 no-op
9682023-05-24T20:23:19.622ZUDP 172.30.2.5 62157 1.1.1.1 53 0 no-op
9692023-05-24T20:23:19.648Z
9702023-05-24T20:23:19.761ZInbound Rules
9712023-05-24T20:23:19.787Z----------------------------------------------------------------------
9722023-05-24T20:23:19.812ZID PRI HITS PREDICATES ACTION
9732023-05-24T20:23:19.837Z0 100 0 * "Stateful Allow"
9742023-05-24T20:23:19.863ZDEF -- 0 -- "deny"
9752023-05-24T20:23:19.888Z
9762023-05-24T20:23:19.913ZOutbound Rules
9772023-05-24T20:23:19.938Z----------------------------------------------------------------------
9782023-05-24T20:23:19.964ZID PRI HITS PREDICATES ACTION
9792023-05-24T20:23:19.990ZDEF -- 24 -- "stateful allow"
9802023-05-24T20:23:20.015Z
9812023-05-24T20:23:20.041Z+ for l in $LAYERS
9822023-05-24T20:23:20.067Z+ pfexec /opt/oxide/opte/bin/opteadm dump-layer -p opte0 router
9832023-05-24T20:23:20.093ZLayer router
9842023-05-24T20:23:20.129Z======================================================================
9852023-05-24T20:23:20.154ZInbound Flows
9862023-05-24T20:23:20.180Z----------------------------------------------------------------------
9872023-05-24T20:23:20.205ZPROTO SRC IP SPORT DST IP DPORT HITS ACTION
9882023-05-24T20:23:20.231Z
9892023-05-24T20:23:20.258ZOutbound Flows
9902023-05-24T20:23:20.283Z----------------------------------------------------------------------
9912023-05-24T20:23:20.308ZPROTO SRC IP SPORT DST IP DPORT HITS ACTION
9922023-05-24T20:23:20.335Z
9932023-05-24T20:23:20.362ZInbound Rules
9942023-05-24T20:23:20.387Z----------------------------------------------------------------------
9952023-05-24T20:23:20.412ZID PRI HITS PREDICATES ACTION
9962023-05-24T20:23:20.445ZDEF -- 0 -- "allow"
9972023-05-24T20:23:20.471Z
9982023-05-24T20:23:20.496ZOutbound Rules
9992023-05-24T20:23:20.527Z----------------------------------------------------------------------
10002023-05-24T20:23:20.552ZID PRI HITS PREDICATES ACTION
10012023-05-24T20:23:21.221Z0 18 0 inner.ip.dst=172.30.2.0/24 "Meta: Target = Subnet: 172.30.2.0/24"
10022023-05-24T20:23:21.247Z1 42 24 inner.ip.dst=0.0.0.0/0 "Meta: Target = IG"
10032023-05-24T20:23:21.273ZDEF -- 0 -- "deny"
10042023-05-24T20:23:21.298Z
10052023-05-24T20:23:21.324Z+ for l in $LAYERS
10062023-05-24T20:23:21.350Z+ pfexec /opt/oxide/opte/bin/opteadm dump-layer -p opte0 nat
10072023-05-24T20:23:21.376ZLayer nat
10082023-05-24T20:23:21.406Z======================================================================
10092023-05-24T20:23:21.434ZInbound Flows
10102023-05-24T20:23:21.459Z----------------------------------------------------------------------
10112023-05-24T20:23:21.485ZPROTO SRC IP SPORT DST IP DPORT HITS ACTION
10122023-05-24T20:23:21.510ZUDP 1.1.1.1 53 192.168.1.22 16378 0 SNAT
10132023-05-24T20:23:21.535ZUDP 1.1.1.1 53 192.168.1.22 16382 0 SNAT
10142023-05-24T20:23:21.561ZUDP 9.9.9.9 53 192.168.1.22 16379 0 SNAT
10152023-05-24T20:23:21.586ZUDP 9.9.9.9 53 192.168.1.22 16383 0 SNAT
10162023-05-24T20:23:21.612Z
10172023-05-24T20:23:21.637ZOutbound Flows
10182023-05-24T20:23:21.662Z----------------------------------------------------------------------
10192023-05-24T20:23:21.688ZPROTO SRC IP SPORT DST IP DPORT HITS ACTION
10202023-05-24T20:23:21.713ZUDP 172.30.2.5 41992 9.9.9.9 53 0 SNAT
10212023-05-24T20:23:21.738ZUDP 172.30.2.5 52435 9.9.9.9 53 0 SNAT
10222023-05-24T20:23:21.768ZUDP 172.30.2.5 57669 1.1.1.1 53 0 SNAT
10232023-05-24T20:23:21.793ZUDP 172.30.2.5 62157 1.1.1.1 53 0 SNAT
10242023-05-24T20:23:21.819Z
10252023-05-24T20:23:21.855ZInbound Rules
10262023-05-24T20:23:21.881Z----------------------------------------------------------------------
10272023-05-24T20:23:21.907ZID PRI HITS PREDICATES ACTION
10282023-05-24T20:23:21.932ZDEF -- 0 -- "allow"
10292023-05-24T20:23:21.958Z
10302023-05-24T20:23:21.983ZOutbound Rules
10312023-05-24T20:23:22.014Z----------------------------------------------------------------------
10322023-05-24T20:23:22.047ZID PRI HITS PREDICATES ACTION
10332023-05-24T20:23:22.072Z0 100 24 inner.ether.ether_type=IPv4 "Stateful: 192.168.1.22:0-16383"
10342023-05-24T20:23:22.097Z meta: router-target=ig
10352023-05-24T20:23:22.123Z
10362023-05-24T20:23:22.153ZDEF -- 0 -- "allow"
10372023-05-24T20:23:22.186Z
10382023-05-24T20:23:22.212Z+ for l in $LAYERS
10392023-05-24T20:23:22.861Z+ pfexec /opt/oxide/opte/bin/opteadm dump-layer -p opte0 overlay
10402023-05-24T20:23:22.886ZLayer overlay
10412023-05-24T20:23:22.912Z======================================================================
10422023-05-24T20:23:22.945ZInbound Flows
10432023-05-24T20:23:22.970Z----------------------------------------------------------------------
10442023-05-24T20:23:23.018ZPROTO SRC IP SPORT DST IP DPORT HITS ACTION
10452023-05-24T20:23:23.045Z
10462023-05-24T20:23:23.072ZOutbound Flows
10472023-05-24T20:23:23.098Z----------------------------------------------------------------------
10482023-05-24T20:23:23.123ZPROTO SRC IP SPORT DST IP DPORT HITS ACTION
10492023-05-24T20:23:23.148Z
10502023-05-24T20:23:23.173ZInbound Rules
10512023-05-24T20:23:23.198Z----------------------------------------------------------------------
10522023-05-24T20:23:23.224ZID PRI HITS PREDICATES ACTION
10532023-05-24T20:23:23.249Z0 1 0 * "Static: Decap"
10542023-05-24T20:23:23.274ZDEF -- 0 -- "deny"
10552023-05-24T20:23:23.300Z
10562023-05-24T20:23:23.325ZOutbound Rules
10572023-05-24T20:23:23.351Z----------------------------------------------------------------------
10582023-05-24T20:23:23.376ZID PRI HITS PREDICATES ACTION
10592023-05-24T20:23:23.460Z0 1 24 * "Static: Encap"
10602023-05-24T20:23:23.486ZDEF -- 0 -- "deny"
10612023-05-24T20:23:23.512Z
10622023-05-24T20:23:23.537Z+ pfexec zfs list
10632023-05-24T20:23:23.562ZNAME USED AVAIL REFER MOUNTPOINT
10642023-05-24T20:23:23.602Zoxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b 371K 9.20G 24K /oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b
10652023-05-24T20:23:23.635Zoxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster 24K 9.20G 24K /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster
10662023-05-24T20:23:23.660Zoxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/config 65K 9.20G 65K /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/config
10672023-05-24T20:23:23.685Zoxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/crash 24K 9.20G 24K /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/crash
10682023-05-24T20:23:23.711Zoxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b/install 24K 9.20G 24K /pool/int/a462a7f7-b628-40fe-80ff-4e4189e2d62b/install
10692023-05-24T20:23:23.827Zoxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b 359K 9.20G 24K /oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b
10702023-05-24T20:23:23.853Zoxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster 24K 9.20G 24K /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/cluster
10712023-05-24T20:23:24.504Zoxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b/config 65K 9.20G 65K /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/config
10722023-05-24T20:23:24.531Zoxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b/crash 24K 9.20G 24K /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/crash
10732023-05-24T20:23:24.561Zoxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b/install 24K 9.20G 24K /pool/int/b462a7f7-b628-40fe-80ff-4e4189e2d62b/install
10742023-05-24T20:23:24.589Zoxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b 246M 8.96G 24K /oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
10752023-05-24T20:23:24.616Zoxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b/crucible 48K 8.96G 24K /data
10762023-05-24T20:23:24.642Zoxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b/crucible/regions 24K 8.96G 24K /data/regions
10772023-05-24T20:23:24.668Zoxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b/zone 246M 8.96G 34K /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/zone
10782023-05-24T20:23:24.694Zoxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b/zone/oxz_crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b 246M 8.96G 246M /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/zone/oxz_crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
10792023-05-24T20:23:24.723Zoxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 247M 8.96G 24K /oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
10802023-05-24T20:23:24.758Zoxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible 48K 8.96G 24K /data
10812023-05-24T20:23:24.785Zoxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible/regions 24K 8.96G 24K /data/regions
10822023-05-24T20:23:24.811Zoxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone 246M 8.96G 34K /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone
10832023-05-24T20:23:24.838Zoxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone/oxz_crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 246M 8.96G 246M /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone/oxz_crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
10842023-05-24T20:23:24.863Zoxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 2.23G 6.97G 24K /oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
10852023-05-24T20:23:24.889Zoxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/clickhouse 32K 6.97G 32K /data
10862023-05-24T20:23:24.916Zoxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/cockroachdb 1.03G 6.97G 1.03G /data
10872023-05-24T20:23:24.942Zoxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible 48K 6.97G 24K /data
10882023-05-24T20:23:24.968Zoxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crucible/regions 24K 6.97G 24K /data/regions
10892023-05-24T20:23:24.998Zoxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone 1.20G 6.97G 36K /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone
10902023-05-24T20:23:25.024Zoxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone/oxz_clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 478M 6.97G 478M /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone/oxz_clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
10912023-05-24T20:23:25.051Zoxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone/oxz_cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 505M 6.97G 505M /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone/oxz_cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
10922023-05-24T20:23:25.077Zoxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone/oxz_crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 246M 6.97G 246M /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/zone/oxz_crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
10932023-05-24T20:23:25.104Zrpool 1.78G 5.49G 24K /rpool
10942023-05-24T20:23:25.133Zrpool/home 24.5K 5.49G 24.5K /home
10952023-05-24T20:23:25.159Zrpool/input 1.15G 5.49G 1.15G /input
10962023-05-24T20:23:25.185Zrpool/softnpu-zone 62.9M 5.49G 62.9M /softnpu-zone
10972023-05-24T20:23:25.211Zrpool/work 24K 5.49G 24K /work
10982023-05-24T20:23:25.238Zrpool/zone 575M 5.49G 27K /zone
10992023-05-24T20:23:25.264Zrpool/zone/oxz_crucible_pantry 90.8M 5.49G 90.8M /zone/oxz_crucible_pantry
11002023-05-24T20:23:25.290Zrpool/zone/oxz_internal_dns 95.6M 5.49G 95.6M /zone/oxz_internal_dns
11012023-05-24T20:23:25.316Zrpool/zone/oxz_ntp 69.9M 5.49G 69.9M /zone/oxz_ntp
11022023-05-24T20:23:25.342Zrpool/zone/oxz_oximeter 87.9M 5.49G 87.9M /zone/oxz_oximeter
11032023-05-24T20:23:25.369Zrpool/zone/oxz_switch 231M 5.49G 231M /zone/oxz_switch
11042023-05-24T20:23:25.395Z+ pfexec zpool list
11052023-05-24T20:23:25.427ZNAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
11062023-05-24T20:23:25.453Zoxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b 9.50G 371K 9.50G - - 0% 0% 1.00x ONLINE -
11072023-05-24T20:23:25.487Zoxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b 9.50G 359K 9.50G - - 0% 0% 1.00x ONLINE -
11082023-05-24T20:23:26.123Zoxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b 9.50G 247M 9.26G - - 0% 2% 1.00x ONLINE -
11092023-05-24T20:23:26.157Zoxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 9.50G 249M 9.26G - - 0% 2% 1.00x ONLINE -
11102023-05-24T20:23:26.184Zoxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 9.50G 2.23G 7.27G - - 0% 23% 1.00x ONLINE -
11112023-05-24T20:23:26.210Zrpool 7.50G 1.88G 5.62G - - 0% 25% 1.00x ONLINE -
11122023-05-24T20:23:26.239Z+ pfexec fmdump -eVp
11132023-05-24T20:23:26.272ZTIME CLASS
11142023-05-24T20:23:26.298Z+ pfexec ptree -z global
11152023-05-24T20:23:26.330Z8 /lib/svc/bin/svc.startd
11162023-05-24T20:23:26.363Z 652 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -
11172023-05-24T20:23:26.390Z 664 /usr/lib/saf/sac -t 300
11182023-05-24T20:23:26.416Z 668 /usr/lib/saf/ttymon
11192023-05-24T20:23:26.442Z 812 /usr/bin/ctrun -l child -o noorphan,regent /opt/buildomat/lib/agent ru
11202023-05-24T20:23:26.468Z 813 /opt/buildomat/lib/agent run
11212023-05-24T20:23:26.495Z 826 /bin/bash /tmp/GXVSbcXz9UqwCzbQ.sh
11222023-05-24T20:23:26.520Z 28051 ptree -z global
11232023-05-24T20:23:26.546Z 1038 /usr/oxide/tcpproxy -b :: -l 54321 -h 172.20.0.5 -r 12346
11242023-05-24T20:23:26.572Z10 /lib/svc/bin/svc.configd
11252023-05-24T20:23:26.597Z47 /lib/inet/ipmgmtd
11262023-05-24T20:23:26.624Z49 /sbin/dlmgmtd
11272023-05-24T20:23:26.649Z53 /lib/inet/netcfgd
11282023-05-24T20:23:26.678Z78 /lib/inet/in.mpathd
11292023-05-24T20:23:26.705Z98 /usr/lib/pfexecd
11302023-05-24T20:23:26.731Z165 /usr/lib/zones/zonestatd
11312023-05-24T20:23:26.757Z185 /usr/lib/sysevent/syseventd
11322023-05-24T20:23:26.783Z211 /sbin/dhcpagent
11332023-05-24T20:23:26.808Z213 devfsadmd
11342023-05-24T20:23:26.834Z579 /usr/lib/picl/picld
11352023-05-24T20:23:26.860Z611 /usr/sbin/nscd
11362023-05-24T20:23:26.886Z651 /usr/lib/utmpd
11372023-05-24T20:23:26.912Z676 /usr/sbin/syslogd
11382023-05-24T20:23:26.946Z693 /usr/sbin/cron
11392023-05-24T20:23:26.971Z730 /usr/sbin/rpcbind
11402023-05-24T20:23:26.997Z741 /usr/lib/inet/in.ndpd
11412023-05-24T20:23:27.023Z743 /usr/lib/fm/fmd/fmd
11422023-05-24T20:23:27.049Z768 /usr/lib/inet/inetd start
11432023-05-24T20:23:27.074Z794 /usr/perl5/bin/perl /usr/lib/intrd
11442023-05-24T20:23:27.107Z947 zoneadmd -z softnpu
11452023-05-24T20:23:27.753Z980 zlogin softnpu /stuff/softnpu /stuff/softnpu.toml
11462023-05-24T20:23:27.779Z1095 ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /
11472023-05-24T20:23:27.805Z 1097 /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.
11482023-05-24T20:23:27.831Z1285 ctrun -l child -o noorphan,regent /opt/oxide/mg-ddm/pkg/ddm_method_scrip
11492023-05-24T20:23:27.857Z 1287 /opt/oxide/mg-ddm/ddmd --admin-port 8000 --admin-addr ::1 --kind serve
11502023-05-24T20:23:27.883Z1734 zoneadmd -z oxz_internal_dns
11512023-05-24T20:23:27.908Z3236 zoneadmd -z oxz_switch
11522023-05-24T20:23:27.934Z4181 zoneadmd -z oxz_ntp
11532023-05-24T20:23:27.962Z8995 zoneadmd -z oxz_cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
11542023-05-24T20:23:27.987Z9707 zoneadmd -z oxz_clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
11552023-05-24T20:23:28.013Z20471 zoneadmd -z oxz_crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
11562023-05-24T20:23:28.039Z23887 zoneadmd -z oxz_crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
11572023-05-24T20:23:28.066Z25933 zoneadmd -z oxz_crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
11582023-05-24T20:23:28.092Z27259 zoneadmd -z oxz_oximeter
11592023-05-24T20:23:28.118Z27961 zoneadmd -z oxz_crucible_pantry
11602023-05-24T20:23:28.145Z+ pfexec svcs -xv
11612023-05-24T20:23:28.172Zsvc:/system/hotplug:default (hotplug daemon)
11622023-05-24T20:23:28.198Z State: disabled since May 24, 2023 at 08:12:43 PM UTC
11632023-05-24T20:23:28.224ZReason: Disabled by an administrator.
11642023-05-24T20:23:28.250Z See: http://illumos.org/msg/SMF-8000-05
11652023-05-24T20:23:28.276Z See: man -M /usr/share/man -s 8 hotplugd
11662023-05-24T20:23:28.309ZImpact: 1 dependent service is not running:
11672023-05-24T20:23:28.354Z svc:/system/t6init:default
11682023-05-24T20:23:28.388Z
11692023-05-24T20:23:28.504Zsvc:/system/ipcc:default (Configure the system to support IPCC)
11702023-05-24T20:23:28.591Z State: disabled since May 24, 2023 at 08:12:45 PM UTC
11712023-05-24T20:23:28.624ZReason: Disabled by an administrator.
11722023-05-24T20:23:28.651Z See: http://illumos.org/msg/SMF-8000-05
11732023-05-24T20:23:28.678Z See: /var/svc/log/system-ipcc:default.log
11742023-05-24T20:23:28.703ZImpact: 1 dependent service is not running:
11752023-05-24T20:23:28.729Z svc:/system/t6init:default
11762023-05-24T20:23:29.375Z++ zoneadm list -n
11772023-05-24T20:23:29.401Z+ for z in $(zoneadm list -n)
11782023-05-24T20:23:29.429Z+ banner softnpu
11792023-05-24T20:23:29.455Z
11802023-05-24T20:23:29.481Z #### #### ###### ##### # # ##### # #
11812023-05-24T20:23:29.506Z # # # # # ## # # # # #
11822023-05-24T20:23:29.535Z #### # # ##### # # # # # # # #
11832023-05-24T20:23:29.562Z # # # # # # # # ##### # #
11842023-05-24T20:23:29.589Z # # # # # # # ## # # #
11852023-05-24T20:23:29.615Z #### #### # # # # # ####
11862023-05-24T20:23:29.640Z
11872023-05-24T20:23:29.666Z+ pfexec svcs -xv -z softnpu
11882023-05-24T20:23:29.692Zsvc:/network/dns/client:default (DNS resolver)
11892023-05-24T20:23:29.717Z Zone: softnpu
11902023-05-24T20:23:29.753Z State: offline since May 24, 2023 at 08:18:13 PM UTC
11912023-05-24T20:23:29.779ZReason: Unknown.
11922023-05-24T20:23:29.805Z See: http://illumos.org/msg/SMF-8000-AR
11932023-05-24T20:23:29.831Z See: man -M /usr/share/man -s 3RESOLV resolver
11942023-05-24T20:23:29.858ZImpact: This service is not running.
11952023-05-24T20:23:29.885Z+ pfexec ptree -z softnpu
11962023-05-24T20:23:29.911Z950 zsched
11972023-05-24T20:23:29.937Z 967 /sbin/init
11982023-05-24T20:23:29.962Z982 /stuff/softnpu /stuff/softnpu.toml
11992023-05-24T20:23:29.988Z992 /lib/svc/bin/svc.startd
12002023-05-24T20:23:30.017Z 1629 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -
12012023-05-24T20:23:30.043Z994 /lib/svc/bin/svc.configd
12022023-05-24T20:23:30.069Z1050 /lib/inet/ipmgmtd
12032023-05-24T20:23:30.095Z1053 /lib/inet/netcfgd
12042023-05-24T20:23:30.121Z1372 /usr/lib/pfexecd
12052023-05-24T20:23:30.154Z1388 /usr/lib/fm/fmd/fmd
12062023-05-24T20:23:30.181Z1578 /usr/sbin/nscd
12072023-05-24T20:23:30.208Z1628 /usr/lib/utmpd
12082023-05-24T20:23:30.235Z1643 /usr/sbin/cron
12092023-05-24T20:23:30.269Z1651 /usr/sbin/syslogd
12102023-05-24T20:23:30.297Z1699 /usr/lib/inet/in.ndpd
12112023-05-24T20:23:30.323Z+ pfexec zlogin softnpu ipadm
12122023-05-24T20:23:30.349ZADDROBJ TYPE STATE ADDR
12132023-05-24T20:23:31.273Zlo0/v4 static ok 127.0.0.1/8
12142023-05-24T20:23:31.300Zlo0/v6 static ok ::1/128
12152023-05-24T20:23:31.430Z+ pfexec zlogin softnpu netstat -rncva
12162023-05-24T20:23:31.534Z
12172023-05-24T20:23:31.591ZIRE Table: IPv4
12182023-05-24T20:23:31.656Z Destination Mask Gateway Device MTU Ref Flg Out In/Fwd
12192023-05-24T20:23:31.683Z-------------------- --------------- -------------------- ------ ----- --- --- ------ ------
12202023-05-24T20:23:31.710Z127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 0 0
12212023-05-24T20:23:31.737Z
12222023-05-24T20:23:31.763ZIRE Table: IPv6
12232023-05-24T20:23:31.788Z Destination/Mask Gateway If MTU Ref Flags Out In/Fwd
12242023-05-24T20:23:31.818Z--------------------------- --------------------------- ----- ----- --- ----- ------ ------
12252023-05-24T20:23:31.845Z::1 ::1 lo0 8252 2 UH 0 0
12262023-05-24T20:23:31.871Z+ pfexec zlogin softnpu netstat -anu
12272023-05-24T20:23:31.898Z
12282023-05-24T20:23:31.924ZUDP: IPv4
12292023-05-24T20:23:31.950Z Local Address Remote Address User Pid Command State
12302023-05-24T20:23:31.976Z-------------------- -------------------- -------- ------ -------------- ----------
12312023-05-24T20:23:32.002Z *.* root 1699 in.ndpd Unbound
12322023-05-24T20:23:32.639Z
12332023-05-24T20:23:32.666ZUDP: IPv6
12342023-05-24T20:23:32.692Z Local Address Remote Address User Pid Command State If
12352023-05-24T20:23:32.718Z--------------------------------- --------------------------------- -------- ------ -------------- ---------- -----
12362023-05-24T20:23:32.744Z *.* root 1699 in.ndpd Unbound
12372023-05-24T20:23:32.770Z
12382023-05-24T20:23:32.796ZActive UNIX domain sockets
12392023-05-24T20:23:32.822ZAddress Type User Pid Command Local Address Remote Address
12402023-05-24T20:23:32.848Z---------------- ---------- -------- ------ -------------- --------------------------------------- ---------------------------------------
 
12412023-05-24T20:23:32.865ZMay 24 20:24:33 sok ip: ip: joining multicasts failed (4) on vopte2 - will use link layer broadcasts for multicast
 
12422023-05-24T20:23:32.874Zfffffe85a8cd4b10 stream-ord root 1699 in.ndpd /var/run/in.ndpd_ipadm
12432023-05-24T20:23:32.900Zfffffe85a0e3f028 dgram root 1699 in.ndpd /var/run/in.ndpd_mib
12442023-05-24T20:23:32.925Zfffffe85a0e3fb08 dgram root 982 softnpu /stuff/server
12452023-05-24T20:23:32.951Zfffffe8586fc8020 stream-ord root 982 softnpu
12462023-05-24T20:23:32.977Zfffffe8586fc8020 stream-ord root 982 softnpu
12472023-05-24T20:23:33.004Zfffffe8584512008 stream-ord root 982 softnpu
12482023-05-24T20:23:33.030Z+ pfexec zlogin softnpu arp -an
12492023-05-24T20:23:33.056Z+ for z in $(zoneadm list -n)
12502023-05-24T20:23:33.082Z+ banner internal_dns
12512023-05-24T20:23:33.108Z
12522023-05-24T20:23:33.134Z # # # ##### ###### ##### # # ## # #####
12532023-05-24T20:23:33.160Z # ## # # # # # ## # # # # # #
12542023-05-24T20:23:33.186Z # # # # # ##### # # # # # # # # # #
12552023-05-24T20:23:33.212Z # # # # # # ##### # # # ###### # # #
12562023-05-24T20:23:33.248Z # # ## # # # # # ## # # # # #
12572023-05-24T20:23:33.274Z # # # # ###### # # # # # # ###### ####### #####
12582023-05-24T20:23:33.300Z
12592023-05-24T20:23:33.326Z+ pfexec svcs -xv -z oxz_internal_dns
12602023-05-24T20:23:33.352Zsvc:/network/dns/client:default (DNS resolver)
12612023-05-24T20:23:33.378Z Zone: oxz_internal_dns
12622023-05-24T20:23:33.404Z State: offline since May 24, 2023 at 08:18:18 PM UTC
12632023-05-24T20:23:33.430ZReason: Unknown.
12642023-05-24T20:23:33.456Z See: http://illumos.org/msg/SMF-8000-AR
12652023-05-24T20:23:33.482Z See: man -M /usr/share/man -s 3RESOLV resolver
12662023-05-24T20:23:33.508ZImpact: This service is not running.
12672023-05-24T20:23:33.534Z+ pfexec ptree -z oxz_internal_dns
12682023-05-24T20:23:33.560Z1737 zsched
12692023-05-24T20:23:34.275Z 1753 /sbin/init
12702023-05-24T20:23:34.383Z1765 /lib/svc/bin/svc.startd
12712023-05-24T20:23:34.409Z 2743 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -
12722023-05-24T20:23:34.435Z1768 /lib/svc/bin/svc.configd
12732023-05-24T20:23:34.462Z1874 /lib/inet/ipmgmtd
12742023-05-24T20:23:34.489Z1878 /lib/inet/netcfgd
12752023-05-24T20:23:34.514Z2315 /usr/lib/pfexecd
12762023-05-24T20:23:34.559Z2348 /usr/lib/fm/fmd/fmd
12772023-05-24T20:23:34.586Z2605 /usr/sbin/nscd
12782023-05-24T20:23:34.612Z2742 /usr/lib/utmpd
12792023-05-24T20:23:34.650Z2758 /usr/sbin/cron
12802023-05-24T20:23:34.676Z2763 /usr/sbin/syslogd
12812023-05-24T20:23:34.701Z2816 /usr/lib/inet/in.ndpd
12822023-05-24T20:23:34.727Z3042 ctrun -l child -o noorphan,regent /opt/oxide/dns-server/bin/dns-server -
12832023-05-24T20:23:34.756Z 3045 /opt/oxide/dns-server/bin/dns-server --config-file /var/svc/manifest/s
12842023-05-24T20:23:34.782Z3089 /sbin/dhcpagent
12852023-05-24T20:23:34.808Z+ pfexec zlogin oxz_internal_dns ipadm
12862023-05-24T20:23:34.834ZADDROBJ TYPE STATE ADDR
12872023-05-24T20:23:34.860Zlo0/v4 static ok 127.0.0.1/8
12882023-05-24T20:23:34.885Zlo0/v6 static ok ::1/128
12892023-05-24T20:23:34.911ZoxControlService1/ll addrconf ok fe80::8:20ff:fe79:f725%oxControlService1/10
12902023-05-24T20:23:34.937ZoxControlService1/omicron6 static ok fd00:1122:3344:1::1/64
12912023-05-24T20:23:34.963Z+ pfexec zlogin oxz_internal_dns netstat -rncva
12922023-05-24T20:23:34.989Z
12932023-05-24T20:23:35.016ZIRE Table: IPv4
12942023-05-24T20:23:35.042Z Destination Mask Gateway Device MTU Ref Flg Out In/Fwd
12952023-05-24T20:23:35.068Z-------------------- --------------- -------------------- ------ ----- --- --- ------ ------
12962023-05-24T20:23:35.095Z127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 19 19
12972023-05-24T20:23:35.121Z
12982023-05-24T20:23:35.147ZIRE Table: IPv6
12992023-05-24T20:23:35.173Z Destination/Mask Gateway If MTU Ref Flags Out In/Fwd
13002023-05-24T20:23:35.198Z--------------------------- --------------------------- ----- ----- --- ----- ------ ------
13012023-05-24T20:23:35.225Z::1 ::1 lo0 8252 2 UH 0 0
13022023-05-24T20:23:35.911Zfd00:1122:3344:1::1 -- oxControlService1 9000 2 UHL 0 447
13032023-05-24T20:23:35.937Zfe80::8:20ff:fe79:f725 -- oxControlService1 9000 2 UHL 0 0
13042023-05-24T20:23:35.972Zfd00:1122:3344:1::2 fd00:1122:3344:1::1 oxControlService1 9000 4 UC 364 0
13052023-05-24T20:23:35.998Zfd00:1122:3344:1::/64 fd00:1122:3344:1::1 oxControlService1 9000 3 U 364 0
13062023-05-24T20:23:36.025Zfe80::/10 fe80::8:20ff:fe79:f725 oxControlService1 9000 2 U 0 0
13072023-05-24T20:23:36.051Zdefault fd00:1122:3344:1::2 0 2 UG 42 0
13082023-05-24T20:23:36.077Z+ pfexec zlogin oxz_internal_dns netstat -anu
13092023-05-24T20:23:36.103Z
13102023-05-24T20:23:36.129ZUDP: IPv4
13112023-05-24T20:23:36.156Z Local Address Remote Address User Pid Command State
13122023-05-24T20:23:36.187Z-------------------- -------------------- -------- ------ -------------- ----------
13132023-05-24T20:23:36.213Z *.* root 2816 in.ndpd Unbound
13142023-05-24T20:23:36.239Z *.68 root 3089 dhcpagent Idle
13152023-05-24T20:23:36.264Z *.546 root 3089 dhcpagent Idle
13162023-05-24T20:23:36.290Z
13172023-05-24T20:23:36.316ZUDP: IPv6
13182023-05-24T20:23:36.342Z Local Address Remote Address User Pid Command State If
13192023-05-24T20:23:36.368Z--------------------------------- --------------------------------- -------- ------ -------------- ---------- -----
13202023-05-24T20:23:36.394Z *.* root 2816 in.ndpd Unbound
13212023-05-24T20:23:36.420Zfd00:1122:3344:1::1.53 root 3045 dns-server Idle
13222023-05-24T20:23:36.452Z *.546 root 3089 dhcpagent Idle
13232023-05-24T20:23:36.479Z
13242023-05-24T20:23:36.506ZTCP: IPv4
13252023-05-24T20:23:36.540Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State
13262023-05-24T20:23:36.574Z-------------------- -------------------- -------- ------ -------------- ------ ------ ------ ------ -----------
13272023-05-24T20:23:36.600Z127.0.0.1.4999 *.* root 3089 dhcpagent 0 0 128000 0 LISTEN
13282023-05-24T20:23:36.628Z
13292023-05-24T20:23:36.654ZTCP: IPv6
13302023-05-24T20:23:36.682Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State If
13312023-05-24T20:23:36.715Z--------------------------------- --------------------------------- -------- ------ -------------- ------ ------ ------ ------ ----------- -----
13322023-05-24T20:23:36.741Zfd00:1122:3344:1::1.5353 *.* root 3045 dns-server 0 0 128000 0 LISTEN
13332023-05-24T20:23:36.767Z
13342023-05-24T20:23:36.793ZActive UNIX domain sockets
13352023-05-24T20:23:36.819ZAddress Type User Pid Command Local Address Remote Address
13362023-05-24T20:23:36.845Z---------------- ---------- -------- ------ -------------- --------------------------------------- ---------------------------------------
13372023-05-24T20:23:36.871Zfffffe85c704a070 stream-ord root 3045 dns-server
13382023-05-24T20:23:36.897Zfffffe85c704a070 stream-ord root 3045 dns-server
13392023-05-24T20:23:37.542Zfffffe85c704a410 stream-ord root 3045 dns-server
13402023-05-24T20:23:37.568Zfffffe85c34c2408 dgram root 2816 in.ndpd /var/run/in.ndpd_mib
13412023-05-24T20:23:37.594Zfffffe8586da9010 stream-ord root 2816 in.ndpd /var/run/in.ndpd_ipadm
13422023-05-24T20:23:37.624Z+ pfexec zlogin oxz_internal_dns arp -an
13432023-05-24T20:23:37.657Z+ for z in $(zoneadm list -n)
13442023-05-24T20:23:37.689Z+ banner switch
13452023-05-24T20:23:37.722Z
13462023-05-24T20:23:37.749Z #### # # # ##### #### # #
13472023-05-24T20:23:37.781Z # # # # # # # # #
13482023-05-24T20:23:37.808Z #### # # # # # ######
13492023-05-24T20:23:37.834Z # # ## # # # # # #
13502023-05-24T20:23:37.859Z # # ## ## # # # # # #
13512023-05-24T20:23:37.885Z #### # # # # #### # #
13522023-05-24T20:23:37.911Z
13532023-05-24T20:23:37.937Z+ pfexec svcs -xv -z oxz_switch
13542023-05-24T20:23:37.963Zsvc:/network/dns/client:default (DNS resolver)
13552023-05-24T20:23:37.988Z Zone: oxz_switch
13562023-05-24T20:23:38.014Z State: offline since May 24, 2023 at 08:18:36 PM UTC
13572023-05-24T20:23:38.040ZReason: Unknown.
13582023-05-24T20:23:38.066Z See: http://illumos.org/msg/SMF-8000-AR
13592023-05-24T20:23:38.092Z See: man -M /usr/share/man -s 3RESOLV resolver
13602023-05-24T20:23:38.118ZImpact: This service is not running.
13612023-05-24T20:23:38.144Z+ pfexec ptree -z oxz_switch
13622023-05-24T20:23:38.170Z3239 zsched
13632023-05-24T20:23:38.196Z 3256 /sbin/init
13642023-05-24T20:23:38.221Z3263 /lib/svc/bin/svc.startd
13652023-05-24T20:23:38.247Z 3760 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -
13662023-05-24T20:23:38.275Z3265 /lib/svc/bin/svc.configd
13672023-05-24T20:23:38.300Z3312 /lib/inet/netcfgd
13682023-05-24T20:23:38.418Z3316 /lib/inet/ipmgmtd
13692023-05-24T20:23:38.444Z3568 /usr/lib/pfexecd
13702023-05-24T20:23:38.478Z3595 /usr/lib/fm/fmd/fmd
13712023-05-24T20:23:38.504Z3707 /usr/sbin/nscd
13722023-05-24T20:23:38.530Z3758 /usr/lib/utmpd
13732023-05-24T20:23:39.177Z3777 /usr/sbin/cron
13742023-05-24T20:23:39.204Z3783 /usr/sbin/syslogd
13752023-05-24T20:23:39.231Z3994 /usr/lib/inet/in.ripngd -s
13762023-05-24T20:23:39.258Z4082 /opt/oxide/dendrite/bin/dpd run
13772023-05-24T20:23:39.288Z4108 /opt/oxide/mgs/bin/mgs run --id-and-address-from-smf /var/svc/manifest/s
13782023-05-24T20:23:39.316Z4138 ctrun -l child -o noorphan,regent /opt/oxide/wicketd/bin/wicketd run /va
13792023-05-24T20:23:39.342Z 4139 /opt/oxide/wicketd/bin/wicketd run /var/svc/manifest/site/wicketd/conf
13802023-05-24T20:23:39.369Z4161 ctrun -l child -o noorphan,regent /opt/oxide/mg-ddm/pkg/ddm_method_scrip
13812023-05-24T20:23:39.465Z 4162 /opt/oxide/mg-ddm/bin/ddmd --admin-port 8000 --admin-addr :: --kind tr
13822023-05-24T20:23:39.491Z4577 /sbin/dhcpagent
13832023-05-24T20:23:39.517Z4700 /usr/lib/inet/in.ndpd
13842023-05-24T20:23:39.543Z+ pfexec zlogin oxz_switch ipadm
13852023-05-24T20:23:39.578ZADDROBJ TYPE STATE ADDR
13862023-05-24T20:23:39.604Zlo0/v4 static ok 127.0.0.1/8
13872023-05-24T20:23:39.630Zlo0/v6 static ok ::1/128
13882023-05-24T20:23:39.671ZoxControlService3/ll addrconf ok fe80::8:20ff:feac:b3d7%oxControlService3/10
13892023-05-24T20:23:39.697ZoxControlService3/omicron6 static ok fd00:1122:3344:101::2/64
13902023-05-24T20:23:39.723ZoxBootstrap2/ll addrconf ok fe80::8:20ff:fe47:1ed9%oxBootstrap2/10
13912023-05-24T20:23:39.749ZoxBootstrap2/bootstrap6 static ok fdb0:18c0:4d0d:a02a::2/64
13922023-05-24T20:23:39.774Z+ pfexec zlogin oxz_switch netstat -rncva
13932023-05-24T20:23:39.801Z
13942023-05-24T20:23:39.826ZIRE Table: IPv4
13952023-05-24T20:23:39.852Z Destination Mask Gateway Device MTU Ref Flg Out In/Fwd
13962023-05-24T20:23:39.878Z-------------------- --------------- -------------------- ------ ----- --- --- ------ ------
13972023-05-24T20:23:39.905Z127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 32 32
13982023-05-24T20:23:39.931Z
13992023-05-24T20:23:39.956ZIRE Table: IPv6
14002023-05-24T20:23:39.982Z Destination/Mask Gateway If MTU Ref Flags Out In/Fwd
14012023-05-24T20:23:40.008Z--------------------------- --------------------------- ----- ----- --- ----- ------ ------
14022023-05-24T20:23:40.035Z::1 ::1 lo0 8252 4 UH 94 94
14032023-05-24T20:23:40.061Zfdb0:18c0:4d0d:a02a::2 -- oxBootstrap2 9000 2 UHL 0 0
14042023-05-24T20:23:40.086Zfe80::8:20ff:fe47:1ed9 -- oxBootstrap2 9000 2 UHL 0 0
14052023-05-24T20:23:40.113Zfd00:1122:3344:101::4 fd00:1122:3344:101::2 oxControlService3 9000 3 UC 88 0
14062023-05-24T20:23:40.139Zfd00:1122:3344:101::1 fd00:1122:3344:101::2 oxControlService3 9000 3 UC 20 0
14072023-05-24T20:23:40.165Zfe80::8:20ff:feac:b3d7 -- oxControlService3 9000 2 UHL 0 0
14082023-05-24T20:23:40.610Zfd00:1122:3344:101::2 -- oxControlService3 9000 2 UHL 87 183
14092023-05-24T20:23:40.800Zfdb0:18c0:4d0d:a02a::/64 fdb0:18c0:4d0d:a02a::2 oxBootstrap2 9000 2 U 0 0
14102023-05-24T20:23:40.827Zfd00:1122:3344:101::/64 fd00:1122:3344:101::2 oxControlService3 9000 4 U 108 0
14112023-05-24T20:23:40.853Zfdb0::/16 fe80::8:20ff:fe88:e715 oxBootstrap2 9000 1 UG 0 0
14122023-05-24T20:23:40.879Zfe80::/10 fe80::8:20ff:feac:b3d7 oxControlService3 9000 2 U 0 0
14132023-05-24T20:23:40.906Zfe80::/10 fe80::8:20ff:fe47:1ed9 oxBootstrap2 9000 2 U 0 0
14142023-05-24T20:23:40.932Zdefault fd00:1122:3344:101::1 0 2 UG 36 0
14152023-05-24T20:23:40.959Z+ pfexec zlogin oxz_switch netstat -anu
14162023-05-24T20:23:40.987Z
14172023-05-24T20:23:41.013ZUDP: IPv4
14182023-05-24T20:23:41.039Z Local Address Remote Address User Pid Command State
14192023-05-24T20:23:41.065Z-------------------- -------------------- -------- ------ -------------- ----------
14202023-05-24T20:23:41.092Z *.* root 4700 in.ndpd Unbound
14212023-05-24T20:23:41.120Z *.* root 3994 in.ripngd Unbound
14222023-05-24T20:23:41.146Z *.521 root 3994 in.ripngd Idle
14232023-05-24T20:23:41.172Z *.22222 root 4108 mgs Idle
14242023-05-24T20:23:41.198Z *.521 root 3994 in.ripngd Idle
14252023-05-24T20:23:41.224Z *.68 root 4577 dhcpagent Idle
14262023-05-24T20:23:41.250Z *.546 root 4577 dhcpagent Idle
14272023-05-24T20:23:41.276Z
14282023-05-24T20:23:41.302ZUDP: IPv6
14292023-05-24T20:23:41.328Z Local Address Remote Address User Pid Command State If
14302023-05-24T20:23:41.354Z--------------------------------- --------------------------------- -------- ------ -------------- ---------- -----
14312023-05-24T20:23:41.381Z *.* root 4700 in.ndpd Unbound
14322023-05-24T20:23:41.407Z *.* root 3994 in.ripngd Unbound
14332023-05-24T20:23:41.433Z *.521 root 3994 in.ripngd Idle oxBootstrap2
14342023-05-24T20:23:41.459Z *.22222 root 4108 mgs Idle
14352023-05-24T20:23:41.485Z *.521 root 3994 in.ripngd Idle oxControlService3
14362023-05-24T20:23:41.518Z *.546 root 4577 dhcpagent Idle
14372023-05-24T20:23:41.544Z
14382023-05-24T20:23:41.577ZTCP: IPv4
14392023-05-24T20:23:41.603Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State
14402023-05-24T20:23:41.629Z-------------------- -------------------- -------- ------ -------------- ------ ------ ------ ------ -----------
14412023-05-24T20:23:41.663Z *.8000 *.* root 4162 ddmd 0 0 128000 0 LISTEN
14422023-05-24T20:23:41.690Z127.0.0.1.4999 *.* root 4577 dhcpagent 0 0 128000 0 LISTEN
14432023-05-24T20:23:41.719Z
14442023-05-24T20:23:41.745ZTCP: IPv6
14452023-05-24T20:23:41.772Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State If
14462023-05-24T20:23:42.448Z--------------------------------- --------------------------------- -------- ------ -------------- ------ ------ ------ ------ ----------- -----
14472023-05-24T20:23:42.475Zfd00:1122:3344:101::2.12224 *.* root 4082 dpd 0 0 128000 0 LISTEN
14482023-05-24T20:23:42.501Z::1.12224 *.* root 4082 dpd 0 0 128000 0 LISTEN
14492023-05-24T20:23:42.528Z::1.12225 *.* root 4108 mgs 0 0 128000 0 LISTEN
14502023-05-24T20:23:42.560Zfd00:1122:3344:101::2.12225 *.* root 4108 mgs 0 0 128000 0 LISTEN
14512023-05-24T20:23:42.587Z::1.43830 ::1.12225 root 4139 wicketd 65440 0 139264 0 ESTABLISHED
14522023-05-24T20:23:42.614Z::1.12225 ::1.43830 root 4108 mgs 139060 0 130880 0 ESTABLISHED
14532023-05-24T20:23:42.639Z::1.12226 *.* root 4139 wicketd 0 0 128000 0 LISTEN
14542023-05-24T20:23:42.666Zfdb0:18c0:4d0d:a02a::2.12227 *.* root 4139 wicketd 0 0 128000 0 LISTEN
14552023-05-24T20:23:42.692Z *.8000 *.* root 4162 ddmd 0 0 128000 0 LISTEN
14562023-05-24T20:23:42.718Zfd00:1122:3344:101::2.42490 fd00:1122:3344:101::4.12221 root 4082 dpd 0 0 134100 0 SYN_SENT
14572023-05-24T20:23:42.744Z
14582023-05-24T20:23:42.770ZActive UNIX domain sockets
14592023-05-24T20:23:42.797ZAddress Type User Pid Command Local Address Remote Address
14602023-05-24T20:23:42.823Z---------------- ---------- -------- ------ -------------- --------------------------------------- ---------------------------------------
14612023-05-24T20:23:42.849Zfffffe85971c2440 stream-ord root 4162 ddmd
14622023-05-24T20:23:42.875Zfffffe85971c2440 stream-ord root 4162 ddmd
14632023-05-24T20:23:42.901Zfffffe85971c27e0 stream-ord root 4162 ddmd
14642023-05-24T20:23:42.928Zfffffe859c014098 stream-ord root 4139 wicketd
14652023-05-24T20:23:42.954Zfffffe859c014098 stream-ord root 4139 wicketd
14662023-05-24T20:23:42.981Zfffffe859c014438 stream-ord root 4139 wicketd
14672023-05-24T20:23:43.096Zfffffe859c0147d8 stream-ord root 4108 mgs
14682023-05-24T20:23:43.122Zfffffe859c014b78 stream-ord root 4108 mgs
14692023-05-24T20:23:43.148Zfffffe859a135090 stream-ord root 4108 mgs
14702023-05-24T20:23:43.175Zfffffe859a135090 stream-ord root 4108 mgs
14712023-05-24T20:23:43.200Zfffffe859a135430 stream-ord root 4108 mgs
14722023-05-24T20:23:43.226Zfffffe859a135b70 stream-ord root 4082 dpd
14732023-05-24T20:23:43.252Zfffffe8599949088 stream-ord root 4082 dpd
14742023-05-24T20:23:43.279Zfffffe8599949428 stream-ord root 4082 dpd
14752023-05-24T20:23:43.305Zfffffe8599949428 stream-ord root 4082 dpd
14762023-05-24T20:23:43.333Zfffffe85999497c8 stream-ord root 4082 dpd
14772023-05-24T20:23:43.360Zfffffe8598432080 dgram root 4700 in.ndpd /var/run/in.ndpd_mib
14782023-05-24T20:23:43.385Zfffffe8584512748 stream-ord root 4700 in.ndpd /var/run/in.ndpd_ipadm
14792023-05-24T20:23:43.411Z+ pfexec zlogin oxz_switch arp -an
14802023-05-24T20:23:44.070Z+ for z in $(zoneadm list -n)
14812023-05-24T20:23:44.096Z+ banner ntp
14822023-05-24T20:23:44.123Z
14832023-05-24T20:23:44.149Z # # ##### #####
14842023-05-24T20:23:44.174Z ## # # # #
14852023-05-24T20:23:44.211Z # # # # # #
14862023-05-24T20:23:44.237Z # # # # #####
14872023-05-24T20:23:44.262Z # ## # #
14882023-05-24T20:23:44.288Z # # # #
14892023-05-24T20:23:44.321Z
14902023-05-24T20:23:44.347Z+ pfexec svcs -xv -z oxz_ntp
14912023-05-24T20:23:44.373Z+ pfexec ptree -z oxz_ntp
14922023-05-24T20:23:44.399Z4184 zsched
14932023-05-24T20:23:44.425Z 4200 /sbin/init
14942023-05-24T20:23:44.451Z4207 /lib/svc/bin/svc.startd
14952023-05-24T20:23:44.538Z 4767 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -
14962023-05-24T20:23:44.564Z4209 /lib/svc/bin/svc.configd
14972023-05-24T20:23:44.590Z4256 /lib/inet/netcfgd
14982023-05-24T20:23:44.618Z4260 /lib/inet/ipmgmtd
14992023-05-24T20:23:44.644Z4535 /usr/lib/pfexecd
15002023-05-24T20:23:44.681Z4547 /usr/lib/fm/fmd/fmd
15012023-05-24T20:23:44.716Z4765 /usr/lib/utmpd
15022023-05-24T20:23:44.750Z4781 /usr/sbin/cron
15032023-05-24T20:23:44.776Z4788 /usr/sbin/syslogd
15042023-05-24T20:23:44.838Z4837 /usr/lib/inet/in.ndpd
15052023-05-24T20:23:44.866Z4928 /sbin/dhcpagent
15062023-05-24T20:23:44.892Z4958 /usr/sbin/nscd
15072023-05-24T20:23:44.917Z4989 /usr/sbin/chronyd -f /etc/inet/chrony.conf
15082023-05-24T20:23:44.943Z 4990 /usr/sbin/chronyd -f /etc/inet/chrony.conf
15092023-05-24T20:23:44.968Z+ pfexec zlogin oxz_ntp ipadm
15102023-05-24T20:23:44.999ZADDROBJ TYPE STATE ADDR
15112023-05-24T20:23:45.040Zlo0/v4 static ok 127.0.0.1/8
15122023-05-24T20:23:45.514Zvopte0/public dhcp ok 172.30.2.5/32
15132023-05-24T20:23:45.540Zlo0/v6 static ok ::1/128
15142023-05-24T20:23:45.714ZoxControlService4/ll addrconf ok fe80::8:20ff:fece:664f%oxControlService4/10
15152023-05-24T20:23:45.753ZoxControlService4/omicron6 static ok fd00:1122:3344:101::c/64
15162023-05-24T20:23:45.779Z+ pfexec zlogin oxz_ntp netstat -rncva
15172023-05-24T20:23:45.805Z
15182023-05-24T20:23:45.831ZIRE Table: IPv4
15192023-05-24T20:23:45.858Z Destination Mask Gateway Device MTU Ref Flg Out In/Fwd
15202023-05-24T20:23:45.884Z-------------------- --------------- -------------------- ------ ----- --- --- ------ ------
15212023-05-24T20:23:45.916Z0.0.0.0 255.255.255.255 -- vopte0 1500 1 UHb 0 0
15222023-05-24T20:23:45.944Zdefault 0.0.0.0 172.30.2.1 0 2 UG 48 0
15232023-05-24T20:23:45.973Z127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 44 44
15242023-05-24T20:23:46.000Z172.30.0.0 255.255.255.255 -- vopte0 1500 1 UHb 0 0
15252023-05-24T20:23:46.032Z172.30.2.1 255.255.255.255 172.30.2.5 vopte0 1500 2 UH 48 0
15262023-05-24T20:23:46.063Z172.30.2.5 255.255.255.255 -- vopte0 1500 2 UHL 0 0
15272023-05-24T20:23:46.095Z172.30.2.5 255.255.255.255 172.30.2.5 vopte0 1500 2 UH 0 0
15282023-05-24T20:23:46.125Z172.30.255.255 255.255.255.255 -- vopte0 1500 1 UHb 0 0
15292023-05-24T20:23:46.151Z255.255.255.255 255.255.255.255 -- vopte0 1500 1 UHb 0 0
15302023-05-24T20:23:46.177Z
15312023-05-24T20:23:46.203ZIRE Table: IPv6
15322023-05-24T20:23:46.230Z Destination/Mask Gateway If MTU Ref Flags Out In/Fwd
15332023-05-24T20:23:46.256Z--------------------------- --------------------------- ----- ----- --- ----- ------ ------
15342023-05-24T20:23:46.282Z::1 ::1 lo0 8252 2 UH 12 12
15352023-05-24T20:23:46.308Zfd00:1122:3344:101::c -- oxControlService4 9000 2 UHL 0 0
15362023-05-24T20:23:46.334Zfe80::8:20ff:fece:664f -- oxControlService4 9000 2 UHL 0 0
15372023-05-24T20:23:46.360Zfd00:1122:3344:101::/64 fd00:1122:3344:101::c oxControlService4 9000 2 U 0 0
15382023-05-24T20:23:46.386Zfe80::/10 fe80::8:20ff:fece:664f oxControlService4 9000 2 U 0 0
15392023-05-24T20:23:46.413Zdefault fd00:1122:3344:101::1 0 1 UG 0 0
15402023-05-24T20:23:46.439Z+ pfexec zlogin oxz_ntp netstat -anu
15412023-05-24T20:23:46.465Z
15422023-05-24T20:23:46.490ZUDP: IPv4
15432023-05-24T20:23:46.516Z Local Address Remote Address User Pid Command State
15442023-05-24T20:23:47.341Z-------------------- -------------------- -------- ------ -------------- ----------
15452023-05-24T20:23:47.374Z *.* root 4837 in.ndpd Unbound
15462023-05-24T20:23:47.403Z *.68 root 4928 dhcpagent Idle
15472023-05-24T20:23:47.430Z *.546 root 4928 dhcpagent Idle
15482023-05-24T20:23:47.469Z172.30.2.5.68 root 4928 dhcpagent Idle
15492023-05-24T20:23:47.496Z127.0.0.1.323 chrony 4989 chronyd Idle
15502023-05-24T20:23:47.523Z *.123 chrony 4989 chronyd Idle
15512023-05-24T20:23:47.553Z
15522023-05-24T20:23:47.579ZUDP: IPv6
15532023-05-24T20:23:47.605Z Local Address Remote Address User Pid Command State If
15542023-05-24T20:23:47.631Z--------------------------------- --------------------------------- -------- ------ -------------- ---------- -----
15552023-05-24T20:23:47.660Z *.* root 4837 in.ndpd Unbound
15562023-05-24T20:23:47.692Z *.546 root 4928 dhcpagent Idle
15572023-05-24T20:23:47.736Z::1.323 chrony 4989 chronyd Idle
15582023-05-24T20:23:47.763Z *.123 chrony 4989 chronyd Idle
15592023-05-24T20:23:47.789Z
15602023-05-24T20:23:48.240ZTCP: IPv4
15612023-05-24T20:23:48.269Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State
15622023-05-24T20:23:48.302Z-------------------- -------------------- -------- ------ -------------- ------ ------ ------ ------ -----------
15632023-05-24T20:23:48.330Z127.0.0.1.4999 *.* root 4928 dhcpagent 0 0 128000 0 LISTEN
15642023-05-24T20:23:49.005Z
15652023-05-24T20:23:49.038ZActive UNIX domain sockets
15662023-05-24T20:23:49.153ZAddress Type User Pid Command Local Address Remote Address
15672023-05-24T20:23:49.181Z---------------- ---------- -------- ------ -------------- --------------------------------------- ---------------------------------------
15682023-05-24T20:23:49.220Zfffffe859f66c7f8 stream-ord root 4837 in.ndpd /var/run/in.ndpd_ipadm
15692023-05-24T20:23:49.250Zfffffe859f66cb98 dgram root 4837 in.ndpd /var/run/in.ndpd_mib
15702023-05-24T20:23:49.285Zfffffe85cd0190b0 dgram chrony 4989 chronyd /var/run/chrony/chronyd.sock
15712023-05-24T20:23:49.319Zfffffe85cd019450 stream-ord
15722023-05-24T20:23:49.345Zfffffe85cd0197f0 stream-ord
15732023-05-24T20:23:49.369Z+ pfexec zlogin oxz_ntp arp -an
15742023-05-24T20:23:49.395ZNet to Media Table: IPv4
15752023-05-24T20:23:49.419ZDevice IP Address Mask Flags Phys Addr
15762023-05-24T20:23:49.444Z------ -------------------- --------------- -------- ---------------
15772023-05-24T20:23:49.468Zvopte0 172.30.2.5 255.255.255.255 SPLA a8:40:25:ff:d5:51
15782023-05-24T20:23:49.493Z+ for z in $(zoneadm list -n)
15792023-05-24T20:23:49.517Z+ banner cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
15802023-05-24T20:23:49.600Z
15812023-05-24T20:23:49.624Z #### #### #### # # ##### #### ## #### # # #####
15822023-05-24T20:23:49.649Z # # # # # # # # # # # # # # # # # # # #
15832023-05-24T20:23:49.679Z # # # # #### # # # # # # # ###### # #
15842023-05-24T20:23:49.727Z # # # # # # ##### # # ###### # # # # #
15852023-05-24T20:23:49.757Z # # # # # # # # # # # # # # # # # # # #
15862023-05-24T20:23:49.794Z #### #### #### # # # # #### # # #### # # #####
15872023-05-24T20:23:49.819Z
15882023-05-24T20:23:49.844Z+ pfexec svcs -xv -z oxz_cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
15892023-05-24T20:23:49.868Zsvc:/network/dns/client:default (DNS resolver)
15902023-05-24T20:23:49.893Z Zone: oxz_cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
15912023-05-24T20:23:49.919Z State: offline since May 24, 2023 at 08:20:00 PM UTC
15922023-05-24T20:23:49.943ZReason: Unknown.
15932023-05-24T20:23:49.968Z See: http://illumos.org/msg/SMF-8000-AR
15942023-05-24T20:23:49.992Z See: man -M /usr/share/man -s 3RESOLV resolver
15952023-05-24T20:23:50.675ZImpact: This service is not running.
15962023-05-24T20:23:50.723Z+ pfexec ptree -z oxz_cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
15972023-05-24T20:23:50.785Z8998 zsched
15982023-05-24T20:23:50.814Z 9014 /sbin/init
15992023-05-24T20:23:50.849Z9021 /lib/svc/bin/svc.startd
16002023-05-24T20:23:50.879Z 9516 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -
16012023-05-24T20:23:50.906Z9023 /lib/svc/bin/svc.configd
16022023-05-24T20:23:50.933Z9070 /lib/inet/ipmgmtd
16032023-05-24T20:23:50.960Z9071 /lib/inet/netcfgd
16042023-05-24T20:23:50.990Z9348 /usr/lib/pfexecd
16052023-05-24T20:23:51.017Z9365 /usr/lib/fm/fmd/fmd
16062023-05-24T20:23:51.048Z9463 /usr/sbin/nscd
16072023-05-24T20:23:51.076Z9515 /usr/lib/utmpd
16082023-05-24T20:23:51.109Z9551 /usr/sbin/cron
16092023-05-24T20:23:51.137Z9552 /usr/sbin/syslogd
16102023-05-24T20:23:51.162Z9661 /opt/oxide/cockroachdb/bin/cockroach start-single-node --insecure --list
16112023-05-24T20:23:51.187Z9690 /sbin/dhcpagent
16122023-05-24T20:23:51.211Z9703 /usr/lib/inet/in.ndpd
16132023-05-24T20:23:51.236Z+ pfexec zlogin oxz_cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 ipadm
16142023-05-24T20:23:51.261ZADDROBJ TYPE STATE ADDR
16152023-05-24T20:23:51.286Zlo0/v4 static ok 127.0.0.1/8
16162023-05-24T20:23:51.310Zlo0/v6 static ok ::1/128
16172023-05-24T20:23:51.337ZoxControlService21/ll addrconf ok fe80::8:20ff:fea0:aa2d%oxControlService21/10
16182023-05-24T20:23:51.362ZoxControlService21/omicron6 static ok fd00:1122:3344:101::6/64
16192023-05-24T20:23:51.386Z+ pfexec zlogin oxz_cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 netstat -rncva
16202023-05-24T20:23:51.418Z
16212023-05-24T20:23:51.443ZIRE Table: IPv4
16222023-05-24T20:23:51.476Z Destination Mask Gateway Device MTU Ref Flg Out In/Fwd
16232023-05-24T20:23:51.510Z-------------------- --------------- -------------------- ------ ----- --- --- ------ ------
16242023-05-24T20:23:51.535Z127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 18 18
16252023-05-24T20:23:51.567Z
16262023-05-24T20:23:51.593ZIRE Table: IPv6
16272023-05-24T20:23:51.617Z Destination/Mask Gateway If MTU Ref Flags Out In/Fwd
16282023-05-24T20:23:51.642Z--------------------------- --------------------------- ----- ----- --- ----- ------ ------
16292023-05-24T20:23:51.667Z::1 ::1 lo0 8252 2 UH 2 2
16302023-05-24T20:23:52.337Zfe80::8:20ff:fea0:aa2d -- oxControlService21 9000 2 UHL 0 0
16312023-05-24T20:23:52.363Zfd00:1122:3344:101::6 -- oxControlService21 9000 4 UHL 21 21
16322023-05-24T20:23:52.391Zfd00:1122:3344:101::/64 fd00:1122:3344:101::6 oxControlService21 9000 2 U 0 0
16332023-05-24T20:23:52.418Zfe80::/10 fe80::8:20ff:fea0:aa2d oxControlService21 9000 2 U 0 0
16342023-05-24T20:23:52.446Zdefault fd00:1122:3344:101::1 0 1 UG 0 0
16352023-05-24T20:23:52.492Z+ pfexec zlogin oxz_cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 netstat -anu
16362023-05-24T20:23:52.532Z
16372023-05-24T20:23:52.557ZUDP: IPv4
16382023-05-24T20:23:52.582Z Local Address Remote Address User Pid Command State
16392023-05-24T20:23:52.608Z-------------------- -------------------- -------- ------ -------------- ----------
16402023-05-24T20:23:52.632Z *.* root 9703 in.ndpd Unbound
16412023-05-24T20:23:52.657Z *.68 root 9690 dhcpagent Idle
16422023-05-24T20:23:52.682Z *.546 root 9690 dhcpagent Idle
16432023-05-24T20:23:52.707Z
16442023-05-24T20:23:52.732ZUDP: IPv6
16452023-05-24T20:23:52.757Z Local Address Remote Address User Pid Command State If
16462023-05-24T20:23:52.782Z--------------------------------- --------------------------------- -------- ------ -------------- ---------- -----
16472023-05-24T20:23:52.807Z *.* root 9703 in.ndpd Unbound
16482023-05-24T20:23:52.838Z *.546 root 9690 dhcpagent Idle
16492023-05-24T20:23:52.866Z
16502023-05-24T20:23:52.893ZTCP: IPv4
16512023-05-24T20:23:52.917Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State
16522023-05-24T20:23:52.942Z-------------------- -------------------- -------- ------ -------------- ------ ------ ------ ------ -----------
16532023-05-24T20:23:52.967Z127.0.0.1.4999 *.* root 9690 dhcpagent 0 0 128000 0 LISTEN
16542023-05-24T20:23:53.000Z
16552023-05-24T20:23:53.025ZTCP: IPv6
16562023-05-24T20:23:53.058Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State If
16572023-05-24T20:23:53.090Z--------------------------------- --------------------------------- -------- ------ -------------- ------ ------ ------ ------ ----------- -----
16582023-05-24T20:23:53.115Zfd00:1122:3344:101::6.8080 *.* root 9661 cockroach 0 0 128000 0 LISTEN
16592023-05-24T20:23:53.140Zfd00:1122:3344:101::6.32221 *.* root 9661 cockroach 0 0 128000 0 LISTEN
16602023-05-24T20:23:53.165Zfd00:1122:3344:101::6.45592 fd00:1122:3344:101::6.32221 root 9661 cockroach 65440 0 139264 0 ESTABLISHED
16612023-05-24T20:23:53.190Zfd00:1122:3344:101::6.32221 fd00:1122:3344:101::6.45592 root 9661 cockroach 139060 0 130880 0 ESTABLISHED
16622023-05-24T20:23:53.214Z
16632023-05-24T20:23:53.239ZActive UNIX domain sockets
16642023-05-24T20:23:53.265ZAddress Type User Pid Command Local Address Remote Address
16652023-05-24T20:23:53.984Z---------------- ---------- -------- ------ -------------- --------------------------------------- ---------------------------------------
16662023-05-24T20:23:54.010Zfffffe85f06db488 dgram root 9703 in.ndpd /var/run/in.ndpd_mib
16672023-05-24T20:23:54.036Zfffffe85f06db828 stream-ord root 9703 in.ndpd /var/run/in.ndpd_ipadm
16682023-05-24T20:23:54.063Z+ pfexec zlogin oxz_cockroachdb_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 arp -an
16692023-05-24T20:23:54.089Z+ for z in $(zoneadm list -n)
16702023-05-24T20:23:54.116Z+ banner clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
16712023-05-24T20:23:54.142Z
16722023-05-24T20:23:54.169Z #### # # #### # # # # #### # # #### ######
16732023-05-24T20:23:54.198Z # # # # # # # # # # # # # # # #
16742023-05-24T20:23:54.226Z # # # # #### ###### # # # # #### #####
16752023-05-24T20:23:54.258Z # # # # # # # # # # # # # #
16762023-05-24T20:23:54.284Z # # # # # # # # # # # # # # # # #
16772023-05-24T20:23:54.310Z #### ###### # #### # # # # #### #### #### ######
16782023-05-24T20:23:54.337Z
16792023-05-24T20:23:54.364Z+ pfexec svcs -xv -z oxz_clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
16802023-05-24T20:23:54.390Zsvc:/network/dns/client:default (DNS resolver)
16812023-05-24T20:23:54.416Z Zone: oxz_clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
16822023-05-24T20:23:54.442Z State: offline since May 24, 2023 at 08:20:15 PM UTC
16832023-05-24T20:23:54.479ZReason: Unknown.
16842023-05-24T20:23:54.505Z See: http://illumos.org/msg/SMF-8000-AR
16852023-05-24T20:23:54.531Z See: man -M /usr/share/man -s 3RESOLV resolver
16862023-05-24T20:23:54.556ZImpact: This service is not running.
16872023-05-24T20:23:54.582Z+ pfexec ptree -z oxz_clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
16882023-05-24T20:23:54.671Z9710 zsched
16892023-05-24T20:23:54.697Z 9726 /sbin/init
16902023-05-24T20:23:54.725Z9733 /lib/svc/bin/svc.startd
16912023-05-24T20:23:54.759Z 10235 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -
16922023-05-24T20:23:54.787Z9735 /lib/svc/bin/svc.configd
16932023-05-24T20:23:54.813Z9781 /lib/inet/ipmgmtd
16942023-05-24T20:23:54.850Z9784 /lib/inet/netcfgd
16952023-05-24T20:23:54.878Z10039 /usr/lib/pfexecd
16962023-05-24T20:23:54.904Z10072 /usr/lib/fm/fmd/fmd
16972023-05-24T20:23:54.930Z10177 /usr/sbin/nscd
16982023-05-24T20:23:54.956Z10233 /usr/lib/utmpd
16992023-05-24T20:23:55.687Z10261 /usr/sbin/cron
17002023-05-24T20:23:55.715Z10271 /usr/sbin/syslogd
17012023-05-24T20:23:55.743Z10369 /opt/oxide/clickhouse/clickhouse server --log-file /var/tmp/clickhouse-s
17022023-05-24T20:23:55.769Z 10370 /opt/oxide/clickhouse/clickhouse server --log-file /var/tmp/clickhouse
17032023-05-24T20:23:55.797Z11085 /sbin/dhcpagent
17042023-05-24T20:23:55.823Z11119 /usr/lib/inet/in.ndpd
17052023-05-24T20:23:55.849Z+ pfexec zlogin oxz_clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 ipadm
17062023-05-24T20:23:55.875ZADDROBJ TYPE STATE ADDR
17072023-05-24T20:23:55.901Zlo0/v4 static ok 127.0.0.1/8
17082023-05-24T20:23:55.933Zlo0/v6 static ok ::1/128
17092023-05-24T20:23:55.959ZoxControlService22/ll addrconf ok fe80::8:20ff:fe00:a0ad%oxControlService22/10
17102023-05-24T20:23:55.987ZoxControlService22/omicron6 static ok fd00:1122:3344:101::7/64
17112023-05-24T20:23:56.013Z+ pfexec zlogin oxz_clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 netstat -rncva
17122023-05-24T20:23:56.043Z
17132023-05-24T20:23:56.069ZIRE Table: IPv4
17142023-05-24T20:23:56.095Z Destination Mask Gateway Device MTU Ref Flg Out In/Fwd
17152023-05-24T20:23:56.121Z-------------------- --------------- -------------------- ------ ----- --- --- ------ ------
17162023-05-24T20:23:56.147Z127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 18 18
17172023-05-24T20:23:56.173Z
17182023-05-24T20:23:56.199ZIRE Table: IPv6
17192023-05-24T20:23:56.225Z Destination/Mask Gateway If MTU Ref Flags Out In/Fwd
17202023-05-24T20:23:56.251Z--------------------------- --------------------------- ----- ----- --- ----- ------ ------
17212023-05-24T20:23:56.277Z::1 ::1 lo0 8252 2 UH 0 0
17222023-05-24T20:23:56.303Zfd00:1122:3344:101::5 fd00:1122:3344:101::7 oxControlService22 9000 3 UC 52 0
17232023-05-24T20:23:56.329Zfd00:1122:3344:101::7 -- oxControlService22 9000 2 UHL 0 36
17242023-05-24T20:23:56.356Zfe80::8:20ff:fe00:a0ad -- oxControlService22 9000 2 UHL 0 0
17252023-05-24T20:23:56.383Zfd00:1122:3344:101::/64 fd00:1122:3344:101::7 oxControlService22 9000 3 U 52 0
17262023-05-24T20:23:56.409Zfe80::/10 fe80::8:20ff:fe00:a0ad oxControlService22 9000 2 U 0 0
17272023-05-24T20:23:56.435Zdefault fd00:1122:3344:101::1 0 1 UG 0 0
17282023-05-24T20:23:56.461Z+ pfexec zlogin oxz_clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 netstat -anu
17292023-05-24T20:23:56.487Z
17302023-05-24T20:23:56.513ZUDP: IPv4
17312023-05-24T20:23:56.540Z Local Address Remote Address User Pid Command State
17322023-05-24T20:23:56.566Z-------------------- -------------------- -------- ------ -------------- ----------
17332023-05-24T20:23:56.592Z *.* root 11119 in.ndpd Unbound
17342023-05-24T20:23:56.618Z *.68 root 11085 dhcpagent Idle
17352023-05-24T20:23:56.645Z *.546 root 11085 dhcpagent Idle
17362023-05-24T20:23:56.671Z
17372023-05-24T20:23:57.793ZUDP: IPv6
17382023-05-24T20:23:57.824Z Local Address Remote Address User Pid Command State If
17392023-05-24T20:23:57.850Z--------------------------------- --------------------------------- -------- ------ -------------- ---------- -----
17402023-05-24T20:23:57.876Z *.* root 11119 in.ndpd Unbound
17412023-05-24T20:23:57.901Z *.546 root 11085 dhcpagent Idle
17422023-05-24T20:23:57.927Z
17432023-05-24T20:23:57.954ZTCP: IPv4
17442023-05-24T20:23:57.980Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State
17452023-05-24T20:23:58.016Z-------------------- -------------------- -------- ------ -------------- ------ ------ ------ ------ -----------
17462023-05-24T20:23:58.041Z127.0.0.1.4999 *.* root 11085 dhcpagent 0 0 128000 0 LISTEN
17472023-05-24T20:23:58.072Z
17482023-05-24T20:23:58.107ZTCP: IPv6
17492023-05-24T20:23:58.139Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State If
17502023-05-24T20:23:58.173Z--------------------------------- --------------------------------- -------- ------ -------------- ------ ------ ------ ------ ----------- -----
17512023-05-24T20:23:58.206Zfd00:1122:3344:101::7.8123 *.* root 10370 clickhouse 0 0 128000 0 LISTEN
17522023-05-24T20:23:58.243Zfd00:1122:3344:101::7.9000 *.* root 10370 clickhouse 0 0 128000 0 LISTEN
17532023-05-24T20:23:58.275Zfd00:1122:3344:101::7.9004 *.* root 10370 clickhouse 0 0 128000 0 LISTEN
17542023-05-24T20:23:58.307Zfd00:1122:3344:101::7.8123 fd00:1122:3344:101::5.61010 root 10370 clickhouse 142848 0 133920 0 ESTABLISHED
17552023-05-24T20:23:58.333Z
17562023-05-24T20:23:58.359ZActive UNIX domain sockets
17572023-05-24T20:23:58.387ZAddress Type User Pid Command Local Address Remote Address
17582023-05-24T20:23:58.413Z---------------- ---------- -------- ------ -------------- --------------------------------------- ---------------------------------------
17592023-05-24T20:23:58.440Zfffffe85f0d35490 dgram root 11119 in.ndpd /var/run/in.ndpd_mib
17602023-05-24T20:23:58.465Zfffffe85f0d35830 stream-ord root 11119 in.ndpd /var/run/in.ndpd_ipadm
17612023-05-24T20:23:58.491Z+ pfexec zlogin oxz_clickhouse_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 arp -an
17622023-05-24T20:23:58.518Z+ for z in $(zoneadm list -n)
17632023-05-24T20:23:58.544Z+ banner crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
17642023-05-24T20:23:58.662Z
17652023-05-24T20:23:58.687Z #### ##### # # #### # ##### # ###### ####
17662023-05-24T20:23:58.713Z # # # # # # # # # # # # # # #
17672023-05-24T20:23:58.739Z # # # # # # # ##### # ##### # #
17682023-05-24T20:23:58.765Z # ##### # # # # # # # # # #
17692023-05-24T20:23:59.410Z # # # # # # # # # # # # # # #
17702023-05-24T20:23:59.464Z #### # # #### #### # ##### ###### ###### ####### ####
17712023-05-24T20:23:59.496Z
17722023-05-24T20:23:59.521Z+ pfexec svcs -xv -z oxz_crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
17732023-05-24T20:23:59.547Zsvc:/network/dns/client:default (DNS resolver)
17742023-05-24T20:23:59.573Z Zone: oxz_crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
17752023-05-24T20:23:59.598Z State: offline since May 24, 2023 at 08:22:46 PM UTC
17762023-05-24T20:23:59.624ZReason: Unknown.
17772023-05-24T20:23:59.650Z See: http://illumos.org/msg/SMF-8000-AR
17782023-05-24T20:23:59.736Z See: man -M /usr/share/man -s 3RESOLV resolver
17792023-05-24T20:23:59.770ZImpact: This service is not running.
17802023-05-24T20:23:59.804Z+ pfexec ptree -z oxz_crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03
17812023-05-24T20:23:59.829Z20474 zsched
17822023-05-24T20:23:59.855Z 20490 /sbin/init
17832023-05-24T20:23:59.880Z20497 /lib/svc/bin/svc.startd
17842023-05-24T20:23:59.906Z 21003 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -
17852023-05-24T20:23:59.943Z 21063 /opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/
17862023-05-24T20:23:59.969Z20499 /lib/svc/bin/svc.configd
17872023-05-24T20:23:59.994Z20545 /lib/inet/ipmgmtd
17882023-05-24T20:24:00.021Z20546 /lib/inet/netcfgd
17892023-05-24T20:24:00.047Z20806 /usr/lib/pfexecd
17902023-05-24T20:24:00.073Z20835 /usr/lib/fm/fmd/fmd
17912023-05-24T20:24:00.098Z20941 /usr/sbin/nscd
17922023-05-24T20:24:00.124Z21000 /usr/lib/utmpd
17932023-05-24T20:24:00.150Z21031 /usr/sbin/syslogd
17942023-05-24T20:24:00.176Z21041 /usr/sbin/cron
17952023-05-24T20:24:00.202Z21859 /sbin/dhcpagent
17962023-05-24T20:24:00.228Z21889 /usr/lib/inet/in.ndpd
17972023-05-24T20:24:00.254Z+ pfexec zlogin oxz_crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 ipadm
17982023-05-24T20:24:00.280ZADDROBJ TYPE STATE ADDR
17992023-05-24T20:24:00.305Zlo0/v4 static ok 127.0.0.1/8
18002023-05-24T20:24:00.331Zlo0/v6 static ok ::1/128
18012023-05-24T20:24:00.357ZoxControlService48/ll addrconf ok fe80::8:20ff:fe5b:d61b%oxControlService48/10
18022023-05-24T20:24:00.383ZoxControlService48/omicron6 static ok fd00:1122:3344:101::8/64
18032023-05-24T20:24:00.409Z+ pfexec zlogin oxz_crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 netstat -rncva
18042023-05-24T20:24:01.069Z
18052023-05-24T20:24:01.106ZIRE Table: IPv4
18062023-05-24T20:24:01.131Z Destination Mask Gateway Device MTU Ref Flg Out In/Fwd
18072023-05-24T20:24:01.157Z-------------------- --------------- -------------------- ------ ----- --- --- ------ ------
18082023-05-24T20:24:01.183Z127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 18 18
18092023-05-24T20:24:01.209Z
18102023-05-24T20:24:01.239ZIRE Table: IPv6
18112023-05-24T20:24:01.273Z Destination/Mask Gateway If MTU Ref Flags Out In/Fwd
18122023-05-24T20:24:01.298Z--------------------------- --------------------------- ----- ----- --- ----- ------ ------
18132023-05-24T20:24:01.324Zfe80::8:20ff:fe5b:d61b -- oxControlService48 9000 2 UHL 0 0
18142023-05-24T20:24:01.350Z::1 ::1 lo0 8252 2 UH 0 0
18152023-05-24T20:24:01.376Zfd00:1122:3344:101::8 -- oxControlService48 9000 2 UHL 0 0
18162023-05-24T20:24:01.402Zfd00:1122:3344:101::/64 fd00:1122:3344:101::8 oxControlService48 9000 2 U 0 0
18172023-05-24T20:24:01.428Zfe80::/10 fe80::8:20ff:fe5b:d61b oxControlService48 9000 2 U 0 0
18182023-05-24T20:24:01.455Zdefault fd00:1122:3344:101::1 0 1 UG 0 0
18192023-05-24T20:24:01.481Z+ pfexec zlogin oxz_crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 netstat -anu
18202023-05-24T20:24:01.507Z
18212023-05-24T20:24:01.533ZUDP: IPv4
18222023-05-24T20:24:01.561Z Local Address Remote Address User Pid Command State
18232023-05-24T20:24:01.587Z-------------------- -------------------- -------- ------ -------------- ----------
18242023-05-24T20:24:01.612Z *.* root 21889 in.ndpd Unbound
18252023-05-24T20:24:01.638Z *.68 root 21859 dhcpagent Idle
18262023-05-24T20:24:01.664Z *.546 root 21859 dhcpagent Idle
18272023-05-24T20:24:01.690Z
18282023-05-24T20:24:01.716ZUDP: IPv6
18292023-05-24T20:24:01.742Z Local Address Remote Address User Pid Command State If
18302023-05-24T20:24:01.769Z--------------------------------- --------------------------------- -------- ------ -------------- ---------- -----
18312023-05-24T20:24:01.795Z *.* root 21889 in.ndpd Unbound
18322023-05-24T20:24:01.821Z *.546 root 21859 dhcpagent Idle
18332023-05-24T20:24:01.848Z
18342023-05-24T20:24:01.874ZTCP: IPv4
18352023-05-24T20:24:01.900Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State
18362023-05-24T20:24:01.926Z-------------------- -------------------- -------- ------ -------------- ------ ------ ------ ------ -----------
18372023-05-24T20:24:01.952Z127.0.0.1.4999 *.* root 21859 dhcpagent 0 0 128000 0 LISTEN
18382023-05-24T20:24:01.978Z
18392023-05-24T20:24:02.004ZTCP: IPv6
18402023-05-24T20:24:02.030Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State If
18412023-05-24T20:24:02.056Z--------------------------------- --------------------------------- -------- ------ -------------- ------ ------ ------ ------ ----------- -----
18422023-05-24T20:24:02.709Zfd00:1122:3344:101::8.32345 *.* root 21063 crucible-agent 0 0 128000 0 LISTEN
18432023-05-24T20:24:02.737Z
18442023-05-24T20:24:02.763ZActive UNIX domain sockets
18452023-05-24T20:24:02.789ZAddress Type User Pid Command Local Address Remote Address
18462023-05-24T20:24:02.814Z---------------- ---------- -------- ------ -------------- --------------------------------------- ---------------------------------------
18472023-05-24T20:24:02.841Zfffffe86022d4138 stream-ord root 21063 crucible-agent
18482023-05-24T20:24:02.866Zfffffe86022d4138 stream-ord root 21063 crucible-agent
18492023-05-24T20:24:02.892Zfffffe86022d44d8 stream-ord root 21063 crucible-agent
18502023-05-24T20:24:02.918Zfffffe85ca978bb0 stream-ord root 21889 in.ndpd /var/run/in.ndpd_ipadm
18512023-05-24T20:24:02.944Zfffffe85cccde808 dgram root 21889 in.ndpd /var/run/in.ndpd_mib
18522023-05-24T20:24:02.969Z+ pfexec zlogin oxz_crucible_oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 arp -an
18532023-05-24T20:24:02.995Z+ for z in $(zoneadm list -n)
18542023-05-24T20:24:03.022Z+ banner crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
18552023-05-24T20:24:03.048Z
18562023-05-24T20:24:03.075Z #### ##### # # #### # ##### # ###### ####
18572023-05-24T20:24:03.101Z # # # # # # # # # # # # # # #
18582023-05-24T20:24:03.127Z # # # # # # # ##### # ##### # #
18592023-05-24T20:24:03.153Z # ##### # # # # # # # # # #
18602023-05-24T20:24:03.179Z # # # # # # # # # # # # # # #
18612023-05-24T20:24:03.205Z #### # # #### #### # ##### ###### ###### ####### ####
18622023-05-24T20:24:03.231Z
18632023-05-24T20:24:03.347Z+ pfexec svcs -xv -z oxz_crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
18642023-05-24T20:24:03.373Zsvc:/network/dns/client:default (DNS resolver)
18652023-05-24T20:24:03.399Z Zone: oxz_crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
18662023-05-24T20:24:03.426Z State: offline since May 24, 2023 at 08:23:24 PM UTC
18672023-05-24T20:24:03.452ZReason: Unknown.
18682023-05-24T20:24:03.478Z See: http://illumos.org/msg/SMF-8000-AR
18692023-05-24T20:24:03.504Z See: man -M /usr/share/man -s 3RESOLV resolver
18702023-05-24T20:24:03.530ZImpact: This service is not running.
18712023-05-24T20:24:03.556Z+ pfexec ptree -z oxz_crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b
18722023-05-24T20:24:03.582Z23890 zsched
18732023-05-24T20:24:03.609Z 23906 /sbin/init
18742023-05-24T20:24:03.636Z23912 /lib/svc/bin/svc.startd
18752023-05-24T20:24:03.661Z 24409 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -
18762023-05-24T20:24:03.687Z 24455 /opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/
18772023-05-24T20:24:04.383Z23914 /lib/svc/bin/svc.configd
18782023-05-24T20:24:04.409Z23961 /lib/inet/ipmgmtd
18792023-05-24T20:24:04.442Z23965 /lib/inet/netcfgd
18802023-05-24T20:24:04.470Z24222 /usr/lib/pfexecd
18812023-05-24T20:24:04.514Z24253 /usr/lib/fm/fmd/fmd
18822023-05-24T20:24:04.540Z24358 /usr/sbin/nscd
18832023-05-24T20:24:04.566Z24408 /usr/lib/utmpd
18842023-05-24T20:24:04.591Z24431 /usr/sbin/cron
18852023-05-24T20:24:04.617Z24438 /usr/sbin/syslogd
18862023-05-24T20:24:04.643Z25268 /sbin/dhcpagent
18872023-05-24T20:24:04.669Z25308 /usr/lib/inet/in.ndpd
18882023-05-24T20:24:04.695Z+ pfexec zlogin oxz_crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b ipadm
18892023-05-24T20:24:04.721ZADDROBJ TYPE STATE ADDR
18902023-05-24T20:24:04.807Zlo0/v4 static ok 127.0.0.1/8
18912023-05-24T20:24:04.833Zlo0/v6 static ok ::1/128
18922023-05-24T20:24:04.859ZoxControlService64/ll addrconf ok fe80::8:20ff:feb2:cbd5%oxControlService64/10
18932023-05-24T20:24:04.885ZoxControlService64/omicron6 static ok fd00:1122:3344:101::9/64
18942023-05-24T20:24:04.919Z+ pfexec zlogin oxz_crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b netstat -rncva
18952023-05-24T20:24:04.945Z
18962023-05-24T20:24:04.972ZIRE Table: IPv4
18972023-05-24T20:24:04.998Z Destination Mask Gateway Device MTU Ref Flg Out In/Fwd
18982023-05-24T20:24:05.024Z-------------------- --------------- -------------------- ------ ----- --- --- ------ ------
18992023-05-24T20:24:05.050Z127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 18 18
19002023-05-24T20:24:05.076Z
19012023-05-24T20:24:05.101ZIRE Table: IPv6
19022023-05-24T20:24:05.128Z Destination/Mask Gateway If MTU Ref Flags Out In/Fwd
19032023-05-24T20:24:05.154Z--------------------------- --------------------------- ----- ----- --- ----- ------ ------
19042023-05-24T20:24:05.180Z::1 ::1 lo0 8252 2 UH 0 0
19052023-05-24T20:24:05.206Zfd00:1122:3344:101::9 -- oxControlService64 9000 2 UHL 0 0
19062023-05-24T20:24:05.232Zfe80::8:20ff:feb2:cbd5 -- oxControlService64 9000 2 UHL 0 0
19072023-05-24T20:24:05.259Zfd00:1122:3344:101::/64 fd00:1122:3344:101::9 oxControlService64 9000 2 U 0 0
19082023-05-24T20:24:05.285Zfe80::/10 fe80::8:20ff:feb2:cbd5 oxControlService64 9000 2 U 0 0
19092023-05-24T20:24:05.310Zdefault fd00:1122:3344:101::1 0 1 UG 0 0
19102023-05-24T20:24:05.336Z+ pfexec zlogin oxz_crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b netstat -anu
19112023-05-24T20:24:05.362Z
19122023-05-24T20:24:06.039ZUDP: IPv4
19132023-05-24T20:24:06.065Z Local Address Remote Address User Pid Command State
19142023-05-24T20:24:06.102Z-------------------- -------------------- -------- ------ -------------- ----------
19152023-05-24T20:24:06.144Z *.* root 25308 in.ndpd Unbound
19162023-05-24T20:24:06.170Z *.68 root 25268 dhcpagent Idle
19172023-05-24T20:24:06.196Z *.546 root 25268 dhcpagent Idle
19182023-05-24T20:24:06.222Z
19192023-05-24T20:24:06.256ZUDP: IPv6
19202023-05-24T20:24:06.283Z Local Address Remote Address User Pid Command State If
19212023-05-24T20:24:06.308Z--------------------------------- --------------------------------- -------- ------ -------------- ---------- -----
19222023-05-24T20:24:06.334Z *.* root 25308 in.ndpd Unbound
19232023-05-24T20:24:06.361Z *.546 root 25268 dhcpagent Idle
19242023-05-24T20:24:06.386Z
19252023-05-24T20:24:06.412ZTCP: IPv4
19262023-05-24T20:24:06.438Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State
19272023-05-24T20:24:06.465Z-------------------- -------------------- -------- ------ -------------- ------ ------ ------ ------ -----------
19282023-05-24T20:24:06.497Z127.0.0.1.4999 *.* root 25268 dhcpagent 0 0 128000 0 LISTEN
19292023-05-24T20:24:06.523Z
19302023-05-24T20:24:06.549ZTCP: IPv6
19312023-05-24T20:24:06.576Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State If
19322023-05-24T20:24:06.602Z--------------------------------- --------------------------------- -------- ------ -------------- ------ ------ ------ ------ ----------- -----
19332023-05-24T20:24:06.629Zfd00:1122:3344:101::9.32345 *.* root 24455 crucible-agent 0 0 128000 0 LISTEN
19342023-05-24T20:24:06.655Z
19352023-05-24T20:24:06.686ZActive UNIX domain sockets
19362023-05-24T20:24:06.720ZAddress Type User Pid Command Local Address Remote Address
19372023-05-24T20:24:06.746Z---------------- ---------- -------- ------ -------------- --------------------------------------- ---------------------------------------
19382023-05-24T20:24:06.778Zfffffe8609597c30 stream-ord root 24455 crucible-agent
19392023-05-24T20:24:06.807Zfffffe8609597c30 stream-ord root 24455 crucible-agent
19402023-05-24T20:24:06.832Zfffffe8608d8d148 stream-ord root 24455 crucible-agent
19412023-05-24T20:24:06.860Zfffffe8608d8d4e8 dgram root 25308 in.ndpd /var/run/in.ndpd_mib
19422023-05-24T20:24:06.886Zfffffe8608d8d888 stream-ord root 25308 in.ndpd /var/run/in.ndpd_ipadm
19432023-05-24T20:24:06.912Z+ pfexec zlogin oxz_crucible_oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b arp -an
19442023-05-24T20:24:06.939Z+ for z in $(zoneadm list -n)
19452023-05-24T20:24:06.964Z+ banner crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
19462023-05-24T20:24:06.990Z
19472023-05-24T20:24:07.017Z #### ##### # # #### # ##### # ###### ####
19482023-05-24T20:24:07.402Z # # # # # # # # # # # # # # #
19492023-05-24T20:24:07.497Z # # # # # # # ##### # ##### # #
19502023-05-24T20:24:07.524Z # ##### # # # # # # # # # #
19512023-05-24T20:24:07.709Z # # # # # # # # # # # # # # #
19522023-05-24T20:24:07.735Z #### # # #### #### # ##### ###### ###### ####### ####
19532023-05-24T20:24:07.768Z
19542023-05-24T20:24:07.802Z+ pfexec svcs -xv -z oxz_crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
19552023-05-24T20:24:07.833Zsvc:/network/dns/client:default (DNS resolver)
19562023-05-24T20:24:07.862Z Zone: oxz_crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
19572023-05-24T20:24:07.889Z State: offline since May 24, 2023 at 08:23:45 PM UTC
19582023-05-24T20:24:07.915ZReason: Unknown.
19592023-05-24T20:24:07.941Z See: http://illumos.org/msg/SMF-8000-AR
19602023-05-24T20:24:07.967Z See: man -M /usr/share/man -s 3RESOLV resolver
19612023-05-24T20:24:07.992ZImpact: This service is not running.
19622023-05-24T20:24:08.105Z+ pfexec ptree -z oxz_crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03
19632023-05-24T20:24:08.131Z25936 zsched
19642023-05-24T20:24:08.156Z 25952 /sbin/init
19652023-05-24T20:24:08.181Z25959 /lib/svc/bin/svc.startd
19662023-05-24T20:24:08.206Z 26460 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -
19672023-05-24T20:24:08.232Z 26521 /opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/
19682023-05-24T20:24:08.257Z25961 /lib/svc/bin/svc.configd
19692023-05-24T20:24:08.282Z26008 /lib/inet/netcfgd
19702023-05-24T20:24:08.308Z26012 /lib/inet/ipmgmtd
19712023-05-24T20:24:08.333Z26263 /usr/lib/pfexecd
19722023-05-24T20:24:08.362Z26291 /usr/lib/fm/fmd/fmd
19732023-05-24T20:24:08.388Z26403 /usr/sbin/nscd
19742023-05-24T20:24:08.413Z26457 /usr/lib/utmpd
19752023-05-24T20:24:08.440Z26486 /usr/sbin/cron
19762023-05-24T20:24:08.466Z26500 /usr/sbin/syslogd
19772023-05-24T20:24:08.498Z27544 /sbin/dhcpagent
19782023-05-24T20:24:08.523Z27660 /usr/lib/inet/in.ndpd
19792023-05-24T20:24:08.548Z+ pfexec zlogin oxz_crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 ipadm
19802023-05-24T20:24:08.574ZADDROBJ TYPE STATE ADDR
19812023-05-24T20:24:08.599Zlo0/v4 static ok 127.0.0.1/8
19822023-05-24T20:24:08.624Zlo0/v6 static ok ::1/128
19832023-05-24T20:24:08.649ZoxControlService75/ll addrconf ok fe80::8:20ff:fee7:c08a%oxControlService75/10
19842023-05-24T20:24:08.674ZoxControlService75/omicron6 static ok fd00:1122:3344:101::a/64
19852023-05-24T20:24:08.699Z+ pfexec zlogin oxz_crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 netstat -rncva
19862023-05-24T20:24:09.354Z
19872023-05-24T20:24:09.380ZIRE Table: IPv4
19882023-05-24T20:24:09.413Z Destination Mask Gateway Device MTU Ref Flg Out In/Fwd
19892023-05-24T20:24:09.438Z-------------------- --------------- -------------------- ------ ----- --- --- ------ ------
19902023-05-24T20:24:09.465Z127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 18 18
19912023-05-24T20:24:09.495Z
19922023-05-24T20:24:09.531ZIRE Table: IPv6
19932023-05-24T20:24:09.563Z Destination/Mask Gateway If MTU Ref Flags Out In/Fwd
19942023-05-24T20:24:09.596Z--------------------------- --------------------------- ----- ----- --- ----- ------ ------
19952023-05-24T20:24:09.621Z::1 ::1 lo0 8252 2 UH 0 0
19962023-05-24T20:24:09.652Zfd00:1122:3344:101::a -- oxControlService75 9000 2 UHL 0 0
19972023-05-24T20:24:09.679Zfe80::8:20ff:fee7:c08a -- oxControlService75 9000 2 UHL 0 0
19982023-05-24T20:24:09.705Zfd00:1122:3344:101::/64 fd00:1122:3344:101::a oxControlService75 9000 2 U 0 0
19992023-05-24T20:24:09.731Zfe80::/10 fe80::8:20ff:fee7:c08a oxControlService75 9000 2 U 0 0
20002023-05-24T20:24:09.756Zdefault fd00:1122:3344:101::1 0 1 UG 0 0
20012023-05-24T20:24:09.781Z+ pfexec zlogin oxz_crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 netstat -anu
20022023-05-24T20:24:09.875Z
20032023-05-24T20:24:09.900ZUDP: IPv4
20042023-05-24T20:24:09.925Z Local Address Remote Address User Pid Command State
20052023-05-24T20:24:09.960Z-------------------- -------------------- -------- ------ -------------- ----------
20062023-05-24T20:24:09.985Z *.* root 27660 in.ndpd Unbound
20072023-05-24T20:24:10.011Z *.68 root 27544 dhcpagent Idle
20082023-05-24T20:24:10.037Z *.546 root 27544 dhcpagent Idle
20092023-05-24T20:24:10.062Z
20102023-05-24T20:24:10.087ZUDP: IPv6
20112023-05-24T20:24:10.113Z Local Address Remote Address User Pid Command State If
20122023-05-24T20:24:10.139Z--------------------------------- --------------------------------- -------- ------ -------------- ---------- -----
20132023-05-24T20:24:10.164Z *.* root 27660 in.ndpd Unbound
20142023-05-24T20:24:10.189Z *.546 root 27544 dhcpagent Idle
20152023-05-24T20:24:10.215Z
20162023-05-24T20:24:10.240ZTCP: IPv4
20172023-05-24T20:24:10.266Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State
20182023-05-24T20:24:10.291Z-------------------- -------------------- -------- ------ -------------- ------ ------ ------ ------ -----------
20192023-05-24T20:24:10.316Z127.0.0.1.4999 *.* root 27544 dhcpagent 0 0 128000 0 LISTEN
20202023-05-24T20:24:10.341Z
20212023-05-24T20:24:11.002ZTCP: IPv6
20222023-05-24T20:24:11.027Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State If
20232023-05-24T20:24:11.052Z--------------------------------- --------------------------------- -------- ------ -------------- ------ ------ ------ ------ ----------- -----
20242023-05-24T20:24:11.077Zfd00:1122:3344:101::a.32345 *.* root 26521 crucible-agent 0 0 128000 0 LISTEN
20252023-05-24T20:24:11.106Z
20262023-05-24T20:24:11.137ZActive UNIX domain sockets
20272023-05-24T20:24:11.168ZAddress Type User Pid Command Local Address Remote Address
20282023-05-24T20:24:11.194Z---------------- ---------- -------- ------ -------------- --------------------------------------- ---------------------------------------
20292023-05-24T20:24:11.219Zfffffe86019f74c8 stream-ord root 27660 in.ndpd /var/run/in.ndpd_ipadm
20302023-05-24T20:24:11.244Zfffffe86019f7868 dgram root 27660 in.ndpd /var/run/in.ndpd_mib
20312023-05-24T20:24:11.272Zfffffe85c9fd40d8 stream-ord root 26521 crucible-agent
20322023-05-24T20:24:11.297Zfffffe85c9fd4478 stream-ord root 26521 crucible-agent
20332023-05-24T20:24:11.323Zfffffe85c9fd4478 stream-ord root 26521 crucible-agent
20342023-05-24T20:24:11.348Z+ pfexec zlogin oxz_crucible_oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 arp -an
20352023-05-24T20:24:11.373Z+ for z in $(zoneadm list -n)
20362023-05-24T20:24:11.398Z+ banner oximeter
20372023-05-24T20:24:11.423Z
20382023-05-24T20:24:11.448Z #### # # # # # ###### ##### ###### #####
20392023-05-24T20:24:11.473Z # # # # # ## ## # # # # #
20402023-05-24T20:24:11.498Z # # ## # # ## # ##### # ##### # #
20412023-05-24T20:24:11.523Z # # ## # # # # # # #####
20422023-05-24T20:24:11.548Z # # # # # # # # # # # #
20432023-05-24T20:24:11.573Z #### # # # # # ###### # ###### # #
20442023-05-24T20:24:11.599Z
20452023-05-24T20:24:11.624Z+ pfexec svcs -xv -z oxz_oximeter
20462023-05-24T20:24:11.649Zsvc:/network/dns/client:default (DNS resolver)
20472023-05-24T20:24:11.674Z Zone: oxz_oximeter
20482023-05-24T20:24:11.699Z State: offline since May 24, 2023 at 08:23:57 PM UTC
20492023-05-24T20:24:11.724ZReason: Unknown.
20502023-05-24T20:24:11.749Z See: http://illumos.org/msg/SMF-8000-AR
20512023-05-24T20:24:11.774Z See: man -M /usr/share/man -s 3RESOLV resolver
20522023-05-24T20:24:11.800ZImpact: This service is not running.
20532023-05-24T20:24:11.825Z+ pfexec ptree -z oxz_oximeter
20542023-05-24T20:24:11.850Z27263 zsched
20552023-05-24T20:24:11.875Z 27279 /sbin/init
20562023-05-24T20:24:11.900Z27286 /lib/svc/bin/svc.startd
20572023-05-24T20:24:11.926Z 27789 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -
20582023-05-24T20:24:11.950Z27290 /lib/svc/bin/svc.configd
20592023-05-24T20:24:11.975Z27336 /lib/inet/ipmgmtd
20602023-05-24T20:24:12.614Z27339 /lib/inet/netcfgd
20612023-05-24T20:24:12.732Z27599 /usr/lib/pfexecd
20622023-05-24T20:24:12.759Z27625 /usr/lib/fm/fmd/fmd
20632023-05-24T20:24:12.785Z27737 /usr/sbin/nscd
20642023-05-24T20:24:12.818Z27787 /usr/lib/utmpd
20652023-05-24T20:24:12.850Z27803 /usr/sbin/cron
20662023-05-24T20:24:12.879Z27816 /usr/sbin/syslogd
20672023-05-24T20:24:12.904Z27869 /usr/lib/inet/in.ndpd
20682023-05-24T20:24:12.932Z27951 ctrun -l child -o noorphan,regent /opt/oxide/oximeter/bin/oximeter run /
20692023-05-24T20:24:12.967Z 27952 /opt/oxide/oximeter/bin/oximeter run /var/svc/manifest/site/oximeter/c
20702023-05-24T20:24:12.992Z+ pfexec zlogin oxz_oximeter ipadm
20712023-05-24T20:24:13.018ZADDROBJ TYPE STATE ADDR
20722023-05-24T20:24:13.044Zlo0/v4 static ok 127.0.0.1/8
20732023-05-24T20:24:13.069Zlo0/v6 static ok ::1/128
20742023-05-24T20:24:13.095ZoxControlService77/ll addrconf ok fe80::8:20ff:fe5e:cb76%oxControlService77/10
20752023-05-24T20:24:13.120ZoxControlService77/omicron6 static ok fd00:1122:3344:101::5/64
20762023-05-24T20:24:13.145Z+ pfexec zlogin oxz_oximeter netstat -rncva
20772023-05-24T20:24:13.170Z
20782023-05-24T20:24:13.196ZIRE Table: IPv4
20792023-05-24T20:24:13.221Z Destination Mask Gateway Device MTU Ref Flg Out In/Fwd
20802023-05-24T20:24:13.247Z-------------------- --------------- -------------------- ------ ----- --- --- ------ ------
20812023-05-24T20:24:13.272Z127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 0 0
20822023-05-24T20:24:13.297Z
20832023-05-24T20:24:13.322ZIRE Table: IPv6
20842023-05-24T20:24:13.348Z Destination/Mask Gateway If MTU Ref Flags Out In/Fwd
20852023-05-24T20:24:13.374Z--------------------------- --------------------------- ----- ----- --- ----- ------ ------
20862023-05-24T20:24:13.399Z::1 ::1 lo0 8252 2 UH 0 0
20872023-05-24T20:24:13.424Zfe80::8:20ff:fe5e:cb76 -- oxControlService77 9000 2 UHL 0 0
20882023-05-24T20:24:13.450Zfd00:1122:3344:101::5 -- oxControlService77 9000 2 UHL 2 66
20892023-05-24T20:24:13.475Zfd00:1122:3344:101::4 fd00:1122:3344:101::5 oxControlService77 9000 3 UC 3 0
20902023-05-24T20:24:13.501Zfd00:1122:3344:101::7 fd00:1122:3344:101::5 oxControlService77 9000 3 UC 36 0
20912023-05-24T20:24:13.527Zfd00:1122:3344:101::1 fd00:1122:3344:101::5 oxControlService77 9000 3 UC 0 0
20922023-05-24T20:24:13.552Zfd00:1122:3344:101::/64 fd00:1122:3344:101::5 oxControlService77 9000 5 U 39 0
20932023-05-24T20:24:13.578Zfe80::/10 fe80::8:20ff:fe5e:cb76 oxControlService77 9000 2 U 0 0
20942023-05-24T20:24:13.603Zdefault fd00:1122:3344:101::1 0 2 UG 6 0
20952023-05-24T20:24:14.229Z+ pfexec zlogin oxz_oximeter netstat -anu
20962023-05-24T20:24:14.255Z
20972023-05-24T20:24:14.293ZUDP: IPv4
20982023-05-24T20:24:14.318Z Local Address Remote Address User Pid Command State
20992023-05-24T20:24:14.345Z-------------------- -------------------- -------- ------ -------------- ----------
21002023-05-24T20:24:14.370Z *.* root 27869 in.ndpd Unbound
21012023-05-24T20:24:14.396Z
21022023-05-24T20:24:14.422ZUDP: IPv6
21032023-05-24T20:24:14.447Z Local Address Remote Address User Pid Command State If
21042023-05-24T20:24:14.480Z--------------------------------- --------------------------------- -------- ------ -------------- ---------- -----
21052023-05-24T20:24:14.505Z *.* root 27869 in.ndpd Unbound
21062023-05-24T20:24:14.531Z
21072023-05-24T20:24:14.575ZTCP: IPv6
21082023-05-24T20:24:14.601Z Local Address Remote Address User Pid Command Swind Send-Q Rwind Recv-Q State If
21092023-05-24T20:24:14.627Z--------------------------------- --------------------------------- -------- ------ -------------- ------ ------ ------ ------ ----------- -----
21102023-05-24T20:24:14.652Zfd00:1122:3344:101::5.61010 fd00:1122:3344:101::7.8123 root 27952 oximeter 133920 0 142848 0 ESTABLISHED
21112023-05-24T20:24:14.677Zfd00:1122:3344:101::5.12223 *.* root 27952 oximeter 0 0 128000 0 LISTEN
21122023-05-24T20:24:14.711Zfd00:1122:3344:101::5.32829 fd00:1122:3344:101::4.12221 root 27952 oximeter 0 0 134100 0 SYN_SENT
21132023-05-24T20:24:14.736Z
21142023-05-24T20:24:14.762ZActive UNIX domain sockets
21152023-05-24T20:24:14.788ZAddress Type User Pid Command Local Address Remote Address
21162023-05-24T20:24:14.813Z---------------- ---------- -------- ------ -------------- --------------------------------------- ---------------------------------------
21172023-05-24T20:24:14.855Zfffffe8613c9e8a0 dgram root 27869 in.ndpd /var/run/in.ndpd_mib
21182023-05-24T20:24:14.940Zfffffe860e143c38 stream-ord root 27869 in.ndpd /var/run/in.ndpd_ipadm
21192023-05-24T20:24:14.965Zfffffe85c98e37e8 stream-ord root 27952 oximeter
21202023-05-24T20:24:14.991Zfffffe85c98e37e8 stream-ord root 27952 oximeter
21212023-05-24T20:24:15.017Zfffffe8586e673b8 stream-ord root 27952 oximeter
21222023-05-24T20:24:15.043Z+ pfexec zlogin oxz_oximeter arp -an
21232023-05-24T20:24:15.068Z+ for z in $(zoneadm list -n)
21242023-05-24T20:24:15.094Z+ banner crucible_pantry
21252023-05-24T20:24:15.120Z
21262023-05-24T20:24:15.146Z #### ##### # # #### # ##### # ###### #####
21272023-05-24T20:24:15.172Z # # # # # # # # # # # # # # #
21282023-05-24T20:24:15.198Z # # # # # # # ##### # ##### # #
21292023-05-24T20:24:15.223Z # ##### # # # # # # # # #####
21302023-05-24T20:24:15.880Z # # # # # # # # # # # # # #
21312023-05-24T20:24:15.905Z #### # # #### #### # ##### ###### ###### ####### #
21322023-05-24T20:24:15.931Z
21332023-05-24T20:24:15.957Z+ pfexec svcs -xv -z oxz_crucible_pantry
21342023-05-24T20:24:15.983Zsvc:/system/rbac:default (Assemble the RBAC *attr files.)
21352023-05-24T20:24:16.009Z Zone: oxz_crucible_pantry
21362023-05-24T20:24:16.034Z State: offline since May 24, 2023 at 08:24:03 PM UTC
21372023-05-24T20:24:16.060ZReason: Start method is running.
21382023-05-24T20:24:16.087Z See: http://illumos.org/msg/SMF-8000-C4
21392023-05-24T20:24:16.133Z See: /zone/oxz_crucible_pantry/root/var/svc/log/system-rbac:default.log
21402023-05-24T20:24:16.160ZImpact: 17 dependent services are not running:
21412023-05-24T20:24:16.185Z svc:/system/manifest-import:default
21422023-05-24T20:24:16.211Z svc:/system/boot-config:default
21432023-05-24T20:24:16.236Z svc:/milestone/single-user:default
21442023-05-24T20:24:16.262Z svc:/milestone/multi-user:default
21452023-05-24T20:24:16.288Z svc:/milestone/multi-user-server:default
21462023-05-24T20:24:16.314Z svc:/milestone/sysconfig:default
21472023-05-24T20:24:16.339Z svc:/system/system-log:default
21482023-05-24T20:24:16.365Z svc:/system/utmp:default
21492023-05-24T20:24:16.391Z svc:/system/console-login:default
21502023-05-24T20:24:16.416Z svc:/system/filesystem/local:default
21512023-05-24T20:24:16.442Z svc:/system/cron:default
21522023-05-24T20:24:16.468Z svc:/network/inetd-upgrade:default
21532023-05-24T20:24:16.493Z svc:/system/boot-archive-update:default
21542023-05-24T20:24:16.519Z svc:/network/shares/group:default
21552023-05-24T20:24:16.545Z svc:/network/routing-setup:default
21562023-05-24T20:24:16.570Z svc:/system/coreadm:default
21572023-05-24T20:24:16.596Z svc:/system/name-service-cache:default
21582023-05-24T20:24:16.622Z
21592023-05-24T20:24:16.649Zsvc:/network/dns/client:default (DNS resolver)
21602023-05-24T20:24:16.769Z Zone: oxz_crucible_pantry
21612023-05-24T20:24:16.796Z State: offline since May 24, 2023 at 08:24:03 PM UTC
21622023-05-24T20:24:16.823ZReason: Unknown.
21632023-05-24T20:24:16.850Z See: http://illumos.org/msg/SMF-8000-AR
21642023-05-24T20:24:17.507Z See: man -M /usr/share/man -s 3RESOLV resolver
21652023-05-24T20:24:17.533ZImpact: This service is not running.
21662023-05-24T20:24:17.560Z+ pfexec ptree -z oxz_crucible_pantry
21672023-05-24T20:24:17.587Z27964 zsched
21682023-05-24T20:24:17.614Z 27980 /sbin/init
21692023-05-24T20:24:17.640Z27986 /lib/svc/bin/svc.startd
21702023-05-24T20:24:17.667Z 28141 /usr/bin/sh /lib/svc/method/svc-rbac start
21712023-05-24T20:24:17.694Z 28424 /bin/sh /usr/sadm/install/scripts/i.rbac
21722023-05-24T20:24:17.720Z 28446 nawk # This script may be invoked with up to three file names.
21732023-05-24T20:24:17.747Z27989 /lib/svc/bin/svc.configd
21742023-05-24T20:24:17.774Z28065 /lib/inet/ipmgmtd
21752023-05-24T20:24:17.800Z28066 /lib/inet/netcfgd
21762023-05-24T20:24:17.827Z28373 /usr/lib/pfexecd
21772023-05-24T20:24:17.854Z28397 /usr/lib/fm/fmd/fmd
21782023-05-24T20:24:17.893Z+ pfexec zlogin oxz_crucible_pantry ipadm
21792023-05-24T20:24:17.920ZADDROBJ TYPE STATE ADDR
21802023-05-24T20:24:17.946Zlo0/v4 static ok 127.0.0.1/8
21812023-05-24T20:24:17.973Zlo0/v6 static ok ::1/128
21822023-05-24T20:24:18.000Z+ pfexec zlogin oxz_crucible_pantry netstat -rncva
21832023-05-24T20:24:18.027Z
21842023-05-24T20:24:18.057ZIRE Table: IPv4
21852023-05-24T20:24:18.084Z Destination Mask Gateway Device MTU Ref Flg Out In/Fwd
21862023-05-24T20:24:18.112Z-------------------- --------------- -------------------- ------ ----- --- --- ------ ------
21872023-05-24T20:24:18.139Z127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 0 0
21882023-05-24T20:24:18.165Z
21892023-05-24T20:24:18.192ZIRE Table: IPv6
21902023-05-24T20:24:18.218Z Destination/Mask Gateway If MTU Ref Flags Out In/Fwd
21912023-05-24T20:24:18.245Z--------------------------- --------------------------- ----- ----- --- ----- ------ ------
21922023-05-24T20:24:18.271Z::1 ::1 lo0 8252 2 UH 0 0
21932023-05-24T20:24:18.298Z+ pfexec zlogin oxz_crucible_pantry netstat -anu
21942023-05-24T20:24:18.324Z+ pfexec zlogin oxz_crucible_pantry arp -an
21952023-05-24T20:24:18.351Z+ exit 1
21962023-05-24T20:24:18.378Zprocess exited: duration 361441 ms, exit code 1
21972023-05-24T20:24:18.466Zexec warning: stdio descriptors remain open after task exit; waiting 60 seconds for them to close
21982023-05-24T20:24:18.493Zexec warning: stdout descriptor may be held open by a background process; giving up!
21992023-05-24T20:24:19.415Zexec warning: stderr descriptor may be held open by a background process; giving up!
 
22002023-05-24T20:24:19.903Zfound 13 output files
22012023-05-24T20:24:19.940Zuploading: /var/svc/log/oxide-sled-agent:default.log (712822 bytes)
22022023-05-24T20:24:19.998Zupload warning: file "/var/svc/log/oxide-sled-agent:default.log" changed size mid upload: 712822 -> 713976
22032023-05-24T20:24:20.029Zuploaded: /var/svc/log/oxide-sled-agent:default.log
22042023-05-24T20:24:20.056Zuploading: /zone/oxz_crucible_pantry/root/var/svc/log/oxide-crucible-pantry:default.log (1715 bytes)
22052023-05-24T20:24:20.086Zuploaded: /zone/oxz_crucible_pantry/root/var/svc/log/oxide-crucible-pantry:default.log
22062023-05-24T20:24:20.114Zuploading: /zone/oxz_external_dns/root/var/svc/log/oxide-external_dns:default.log (2324 bytes)
22072023-05-24T20:24:20.140Zuploaded: /zone/oxz_external_dns/root/var/svc/log/oxide-external_dns:default.log
22082023-05-24T20:24:20.798Zuploading: /zone/oxz_internal_dns/root/var/svc/log/oxide-internal_dns:default.log (3847 bytes)
22092023-05-24T20:24:20.827Zuploaded: /zone/oxz_internal_dns/root/var/svc/log/oxide-internal_dns:default.log
22102023-05-24T20:24:20.854Zuploading: /zone/oxz_nexus/root/var/svc/log/oxide-nexus:default.log (492240 bytes)
22112023-05-24T20:24:20.914Zupload warning: file "/zone/oxz_nexus/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 492240 -> 555723
22122023-05-24T20:24:20.950Zuploaded: /zone/oxz_nexus/root/var/svc/log/oxide-nexus:default.log
22132023-05-24T20:24:20.978Zuploading: /zone/oxz_ntp/root/var/svc/log/oxide-ntp:default.log (559 bytes)
22142023-05-24T20:24:21.004Zuploaded: /zone/oxz_ntp/root/var/svc/log/oxide-ntp:default.log
22152023-05-24T20:24:21.030Zuploading: /zone/oxz_oximeter/root/var/svc/log/oxide-oximeter:default.log (22093 bytes)
22162023-05-24T20:24:21.061Zuploaded: /zone/oxz_oximeter/root/var/svc/log/oxide-oximeter:default.log
22172023-05-24T20:24:21.087Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log (5056141 bytes)
22182023-05-24T20:24:21.784Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log" changed size mid upload: 5056141 -> 5082001
22192023-05-24T20:24:22.245Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log
22202023-05-24T20:24:22.282Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log (891 bytes)
22212023-05-24T20:24:22.470Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log
22222023-05-24T20:24:22.503Zuploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log (428372 bytes)
22232023-05-24T20:24:22.538Zupload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log" changed size mid upload: 428372 -> 431645
22242023-05-24T20:24:22.565Zuploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log
22252023-05-24T20:24:22.592Zuploading: /zone/oxz_switch/root/var/svc/log/system-illumos-dendrite:default.log (137138 bytes)
22262023-05-24T20:24:22.627Zuploaded: /zone/oxz_switch/root/var/svc/log/system-illumos-dendrite:default.log
22272023-05-24T20:24:22.653Zuploading: /zone/oxz_switch/root/var/svc/log/system-illumos-mg-ddm:default.log (433 bytes)
22282023-05-24T20:24:22.680Zuploaded: /zone/oxz_switch/root/var/svc/log/system-illumos-mg-ddm:default.log
22292023-05-24T20:24:22.706Zuploading: /zone/oxz_switch/root/var/svc/log/system-illumos-tfport:default.log (75 bytes)
22302023-05-24T20:24:22.733Zuploaded: /zone/oxz_switch/root/var/svc/log/system-illumos-tfport:default.log