|
|
|
1 | 2024-02-16T05:51:14.313Z | job dependencies complete; ready to run (waiting for 40 m 29 s) |
2 | 2024-02-16T06:46:13.760Z | job assigned to worker 01HPRABMPYBR5CGTCK4NA36CWH (queued for 54 m 59 s) |
3 | 2024-02-16T06:46:14.150Z | http://172.20.0.5:9969/ipxe/sock... ok |
4 | 2024-02-16T06:46:14.619Z | http://172.20.0.5:9969/os/sock/platform/i86pc/kernel/amd64/unix... ok |
5 | 2024-02-16T06:46:15.081Z | http://172.20.0.5:9969/os/sock/platform/i86pc/amd64/boot_archive... 21%Error: No response to keepalive - Terminating session |
6 | 2024-02-16T06:46:15.127Z | Error: No response de-activating SOL payload |
7 | 2024-02-16T06:46:15.162Z | [SOL Session operational. Use ~? for help] |
8 | 2024-02-16T06:46:15.204Z | ok |
9 | 2024-02-16T06:46:15.247Z | http://172.20.0.5:9969/os/sock/platform/i86pc/amd64/boot_archive.hash... ok |
10 | 2024-02-16T06:46:15.290Z | http://172.20.0.5:9969/postboot/sock... ok |
11 | 2024-02-16T06:46:15.339Z | wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U wcnCroia6U |
12 | 2024-02-16T06:46:15.895Z | cpu0: microcode has been updated from version 0xa001133 to 0xa0011d1 |
13 | 2024-02-16T06:46:15.943Z | Oxide Helios Version helios-2.0.22451 64-bit |
14 | 2024-02-16T06:46:16.953Z | downloading input: /input/ci-tools/work/end-to-end-tests/bootstrap-22e122cc756fa22c.gz |
15 | 2024-02-16T06:46:26.141Z | downloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap-22e122cc756fa22c.gz |
16 | 2024-02-16T06:46:26.141Z | downloading input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz |
17 | 2024-02-16T06:46:47.513Z | Error: No response to keepalive - Terminating session |
18 | 2024-02-16T06:47:03.352Z | Error: No response de-activating SOL payload |
19 | 2024-02-16T06:47:03.471Z | [SOL Session operational. Use ~? for help] |
20 | 2024-02-16T06:47:04.554Z | downloaded input: /input/ci-tools/work/end-to-end-tests/bootstrap.gz |
21 | 2024-02-16T06:47:04.604Z | downloading input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-4eb1211856b7c67f.gz |
22 | 2024-02-16T06:48:04.278Z | downloaded input: /input/ci-tools/work/end-to-end-tests/end_to_end_tests-4eb1211856b7c67f.gz |
23 | 2024-02-16T06:48:04.278Z | downloading input: /input/ci-tools/work/caboose-util.gz |
24 | 2024-02-16T06:48:07.321Z | downloaded input: /input/ci-tools/work/caboose-util.gz |
25 | 2024-02-16T06:48:07.805Z | downloading input: /input/ci-tools/work/tufaceous.gz |
26 | 2024-02-16T06:48:17.744Z | downloaded input: /input/ci-tools/work/tufaceous.gz |
27 | 2024-02-16T06:48:17.772Z | downloading input: /input/package/work/version.txt |
28 | 2024-02-16T06:48:17.869Z | downloaded input: /input/package/work/version.txt |
29 | 2024-02-16T06:48:17.898Z | downloading input: /input/package/work/package.tar.gz |
30 | 2024-02-16T06:49:09.858Z | downloaded input: /input/package/work/package.tar.gz |
31 | 2024-02-16T06:49:09.858Z | downloading input: /input/package/work/global-zone-packages.tar.gz |
32 | 2024-02-16T06:49:57.646Z | Error: No response to keepalive - Terminating session |
33 | 2024-02-16T06:50:07.821Z | Error: No response de-activating SOL payload |
34 | 2024-02-16T06:50:09.204Z | [SOL Session operational. Use ~? for help] |
35 | 2024-02-16T06:50:13.244Z | downloaded input: /input/package/work/global-zone-packages.tar.gz |
36 | 2024-02-16T06:50:13.244Z | downloading input: /input/package/work/trampoline-global-zone-packages.tar.gz |
37 | 2024-02-16T06:50:34.509Z | downloaded input: /input/package/work/trampoline-global-zone-packages.tar.gz |
38 | 2024-02-16T06:50:34.565Z | downloading input: /input/package/work/zones/clickhouse.tar.gz |
39 | 2024-02-16T06:52:42.844Z | downloaded input: /input/package/work/zones/clickhouse.tar.gz |
40 | 2024-02-16T06:52:42.870Z | downloading input: /input/package/work/zones/clickhouse_keeper.tar.gz |
41 | 2024-02-16T06:52:57.904Z | Error: No response to keepalive - Terminating session |
42 | 2024-02-16T06:53:07.834Z | Error: No response de-activating SOL payload |
43 | 2024-02-16T06:53:09.063Z | [SOL Session operational. Use ~? for help] |
44 | 2024-02-16T06:54:13.495Z | downloaded input: /input/package/work/zones/clickhouse_keeper.tar.gz |
45 | 2024-02-16T06:54:13.565Z | downloading input: /input/package/work/zones/cockroachdb.tar.gz |
46 | 2024-02-16T06:55:28.018Z | downloaded input: /input/package/work/zones/cockroachdb.tar.gz |
47 | 2024-02-16T06:55:28.019Z | downloading input: /input/package/work/zones/crucible-pantry.tar.gz |
48 | 2024-02-16T06:55:33.791Z | downloaded input: /input/package/work/zones/crucible-pantry.tar.gz |
49 | 2024-02-16T06:55:33.791Z | downloading input: /input/package/work/zones/crucible.tar.gz |
50 | 2024-02-16T06:55:43.303Z | downloaded input: /input/package/work/zones/crucible.tar.gz |
51 | 2024-02-16T06:55:43.304Z | downloading input: /input/package/work/zones/external-dns.tar.gz |
52 | 2024-02-16T06:55:57.729Z | Error: No response to keepalive - Terminating session |
53 | 2024-02-16T06:56:02.641Z | downloaded input: /input/package/work/zones/external-dns.tar.gz |
54 | 2024-02-16T06:56:02.682Z | downloading input: /input/package/work/zones/internal-dns.tar.gz |
55 | 2024-02-16T06:56:07.695Z | Error: No response de-activating SOL payload |
56 | 2024-02-16T06:56:09.493Z | [SOL Session operational. Use ~? for help] |
57 | 2024-02-16T06:56:11.532Z | downloaded input: /input/package/work/zones/internal-dns.tar.gz |
58 | 2024-02-16T06:56:11.532Z | downloading input: /input/package/work/zones/ntp.tar.gz |
59 | 2024-02-16T06:56:11.582Z | downloaded input: /input/package/work/zones/ntp.tar.gz |
60 | 2024-02-16T06:56:11.636Z | downloading input: /input/package/work/zones/omicron-gateway-asic.tar.gz |
61 | 2024-02-16T06:56:15.526Z | downloaded input: /input/package/work/zones/omicron-gateway-asic.tar.gz |
62 | 2024-02-16T06:56:15.526Z | downloading input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz |
63 | 2024-02-16T06:56:24.546Z | downloaded input: /input/package/work/zones/omicron-gateway-softnpu.tar.gz |
64 | 2024-02-16T06:56:24.546Z | downloading input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz |
65 | 2024-02-16T06:56:41.707Z | downloaded input: /input/package/work/zones/omicron-nexus-single-sled.tar.gz |
66 | 2024-02-16T06:56:41.707Z | downloading input: /input/package/work/zones/omicron-nexus.tar.gz |
67 | 2024-02-16T06:57:08.503Z | downloaded input: /input/package/work/zones/omicron-nexus.tar.gz |
68 | 2024-02-16T06:57:08.503Z | downloading input: /input/package/work/zones/overlay.tar.gz |
69 | 2024-02-16T06:57:08.592Z | downloaded input: /input/package/work/zones/overlay.tar.gz |
70 | 2024-02-16T06:57:08.592Z | downloading input: /input/package/work/zones/oximeter.tar.gz |
71 | 2024-02-16T06:57:25.818Z | downloaded input: /input/package/work/zones/oximeter.tar.gz |
72 | 2024-02-16T06:57:25.818Z | downloading input: /input/package/work/zones/propolis-server.tar.gz |
73 | 2024-02-16T06:57:44.593Z | downloaded input: /input/package/work/zones/propolis-server.tar.gz |
74 | 2024-02-16T06:57:44.593Z | downloading input: /input/package/work/zones/switch-asic.tar.gz |
75 | 2024-02-16T06:58:40.724Z | downloaded input: /input/package/work/zones/switch-asic.tar.gz |
76 | 2024-02-16T06:58:40.724Z | downloading input: /input/package/work/zones/switch-softnpu.tar.gz |
77 | 2024-02-16T06:59:13.783Z | Error: No response to keepalive - Terminating session |
78 | 2024-02-16T06:59:23.749Z | Error: No response de-activating SOL payload |
79 | 2024-02-16T06:59:25.145Z | [SOL Session operational. Use ~? for help] |
80 | 2024-02-16T06:59:30.278Z | downloaded input: /input/package/work/zones/switch-softnpu.tar.gz |
|
81 | 2024-02-16T06:59:30.304Z | starting task 0: "setup" |
82 | 2024-02-16T06:59:30.304Z | ++ uname -s |
83 | 2024-02-16T06:59:30.304Z | + kern=SunOS |
84 | 2024-02-16T06:59:30.305Z | + case "$kern" in |
85 | 2024-02-16T06:59:30.305Z | + groupadd -g 12345 build |
86 | 2024-02-16T06:59:30.305Z | + useradd -u 12345 -g build -d /home/build -s /bin/bash -c build -P 'Primary Administrator' build |
87 | 2024-02-16T06:59:32.297Z | + zfs create -o mountpoint=/work rpool/work |
88 | 2024-02-16T06:59:32.333Z | ++ awk '$2 == "/home" { print $3 }' /etc/mnttab |
89 | 2024-02-16T06:59:32.359Z | + home_fs=zfs |
90 | 2024-02-16T06:59:32.359Z | + [[ zfs == autofs ]] |
91 | 2024-02-16T06:59:32.359Z | + mkdir -p /home/build |
92 | 2024-02-16T06:59:32.359Z | + chown build:build /home/build /work |
93 | 2024-02-16T06:59:34.341Z | + chmod 0700 /home/build /work |
94 | 2024-02-16T06:59:34.367Z | process exited: duration 4065 ms, exit code 0 |
|
95 | 2024-02-16T06:59:34.420Z | starting task 1: "authentication" |
96 | 2024-02-16T06:59:34.465Z | process exited: duration 44 ms, exit code 0 |
|
97 | 2024-02-16T06:59:34.516Z | starting task 2: "build" |
98 | 2024-02-16T06:59:34.541Z | + trap _exit_trap EXIT |
99 | 2024-02-16T06:59:34.541Z | + set +x |
100 | 2024-02-16T06:59:59.685Z | + pfexec mdb -kwe 'secpolicy_fs_linkdir/v 55 48 89 e5 b8 01 00 00 00 5d c3' |
101 | 2024-02-16T06:59:59.937Z | secpolicy_fs_linkdir: 0x55 = 0x55 |
102 | 2024-02-16T06:59:59.938Z | secpolicy_fs_linkdir+1: 0xb8 = 0x48 |
103 | 2024-02-16T06:59:59.938Z | secpolicy_fs_linkdir+2: 0x1 = 0x89 |
104 | 2024-02-16T06:59:59.938Z | secpolicy_fs_linkdir+3: 0 = 0xe5 |
105 | 2024-02-16T06:59:59.938Z | secpolicy_fs_linkdir+4: 0 = 0xb8 |
106 | 2024-02-16T06:59:59.938Z | secpolicy_fs_linkdir+5: 0 = 0x1 |
107 | 2024-02-16T06:59:59.938Z | secpolicy_fs_linkdir+6: 0x48 = 0x0 |
108 | 2024-02-16T06:59:59.938Z | secpolicy_fs_linkdir+7: 0x89 = 0x0 |
109 | 2024-02-16T06:59:59.964Z | secpolicy_fs_linkdir+8: 0xe5 = 0x0 |
110 | 2024-02-16T06:59:59.964Z | secpolicy_fs_linkdir+9: 0x48 = 0x5d |
111 | 2024-02-16T06:59:59.965Z | secpolicy_fs_linkdir+0xa: 0x83 = 0xc3 |
112 | 2024-02-16T06:59:59.965Z | + [[ -d /opt/oxide ]] |
113 | 2024-02-16T06:59:59.965Z | + pfexec mv /opt/oxide /opt/oxide-underneath |
114 | 2024-02-16T06:59:59.965Z | + pfexec mkdir /opt/oxide |
115 | 2024-02-16T06:59:59.965Z | + pfexec mount -F tmpfs -O swap /opt/oxide |
116 | 2024-02-16T06:59:59.965Z | + [[ -d /opt/oxide-underneath ]] |
117 | 2024-02-16T06:59:59.965Z | + cd /opt/oxide-underneath |
118 | 2024-02-16T06:59:59.965Z | + pfexec tar ceEp@/f - . |
119 | 2024-02-16T06:59:59.965Z | + cd /opt/oxide |
120 | 2024-02-16T06:59:59.966Z | + pfexec tar xveEp@/f - |
121 | 2024-02-16T06:59:59.996Z | x ., 0 bytes, 0 tape blocks |
122 | 2024-02-16T06:59:59.996Z | x ./opte, 0 bytes, 0 tape blocks |
123 | 2024-02-16T06:59:59.997Z | x ./opte/bin, 0 bytes, 0 tape blocks |
124 | 2024-02-16T06:59:59.997Z | x ./opte/bin/opteadm, 51799224 bytes, 101171 tape blocks |
125 | 2024-02-16T07:00:00.023Z | + pfexec rm -rf /opt/oxide-underneath |
126 | 2024-02-16T07:00:00.023Z | + pfexec /sbin/zfs create -o mountpoint=/zone rpool/zone |
127 | 2024-02-16T07:00:00.049Z | + pfexec mkdir /opt/oxide/work |
128 | 2024-02-16T07:00:00.049Z | + pfexec chown build:build /opt/oxide/work |
129 | 2024-02-16T07:00:00.049Z | + cd /opt/oxide/work |
130 | 2024-02-16T07:00:00.049Z | + ptime -m tar xvzf /input/package/work/package.tar.gz |
131 | 2024-02-16T07:00:01.752Z | Decompressing '/input/package/work/package.tar.gz' with '/usr/bin/gzcat'... |
132 | 2024-02-16T07:00:01.752Z | x out/mg-ddm-gz.tar, 69308416 bytes, 135368 tape blocks |
133 | 2024-02-16T07:00:01.752Z | x out/omicron-sled-agent.tar, 122944512 bytes, 240126 tape blocks |
134 | 2024-02-16T07:00:01.752Z | x out/oxlog.tar, 8382464 bytes, 16372 tape blocks |
135 | 2024-02-16T07:00:01.753Z | x out/target/test, 75 bytes, 1 tape blocks |
136 | 2024-02-16T07:00:01.753Z | x out/npuzone/npuzone, 37184104 bytes, 72626 tape blocks |
137 | 2024-02-16T07:00:01.753Z | x package-manifest.toml, 23732 bytes, 47 tape blocks |
138 | 2024-02-16T07:00:01.753Z | x smf/sled-agent/non-gimlet/config.toml, 3448 bytes, 7 tape blocks |
139 | 2024-02-16T07:00:01.753Z | x target/release/omicron-package, 45826752 bytes, 89506 tape blocks |
140 | 2024-02-16T07:00:01.753Z | x tools/create_virtual_hardware.sh, 3184 bytes, 7 tape blocks |
141 | 2024-02-16T07:00:01.753Z | x tools/virtual_hardware.sh, 3416 bytes, 7 tape blocks |
142 | 2024-02-16T07:00:01.754Z | x tools/scrimlet/create-softnpu-zone.sh, 627 bytes, 2 tape blocks |
143 | 2024-02-16T07:00:01.754Z | x tools/scrimlet/destroy-softnpu-zone.sh, 203 bytes, 1 tape blocks |
144 | 2024-02-16T07:00:01.754Z | x tools/scrimlet/softnpu-init.sh, 2157 bytes, 5 tape blocks |
145 | 2024-02-16T07:00:01.754Z | x tools/scrimlet/softnpu-zone.txt, 330 bytes, 1 tape blocks |
146 | 2024-02-16T07:00:01.754Z | x tools/scrimlet/softnpu.toml, 171 bytes, 1 tape blocks |
147 | 2024-02-16T07:00:01.781Z | |
148 | 2024-02-16T07:00:01.781Z | real 1.710531143 |
149 | 2024-02-16T07:00:01.782Z | user 1.540470645 |
150 | 2024-02-16T07:00:01.782Z | sys 0.301438704 |
151 | 2024-02-16T07:00:01.782Z | trap 0.000066162 |
152 | 2024-02-16T07:00:01.782Z | tflt 0.000000000 |
153 | 2024-02-16T07:00:01.782Z | dflt 0.000000000 |
154 | 2024-02-16T07:00:01.782Z | kflt 0.000000000 |
155 | 2024-02-16T07:00:01.782Z | lock 0.000000000 |
156 | 2024-02-16T07:00:01.783Z | slp 1.540717664 |
157 | 2024-02-16T07:00:01.783Z | lat 0.036463608 |
158 | 2024-02-16T07:00:01.783Z | stop 0.000120983 |
159 | 2024-02-16T07:00:01.783Z | + cp /input/package/work/zones/clickhouse_keeper.tar.gz /input/package/work/zones/clickhouse.tar.gz /input/package/work/zones/cockroachdb.tar.gz /input/package/work/zones/crucible-pantry.tar.gz /input/package/work/zones/crucible.tar.gz /input/package/work/zones/external-dns.tar.gz /input/package/work/zones/internal-dns.tar.gz /input/package/work/zones/ntp.tar.gz /input/package/work/zones/omicron-gateway-asic.tar.gz /input/package/work/zones/omicron-gateway-softnpu.tar.gz /input/package/work/zones/omicron-nexus-single-sled.tar.gz /input/package/work/zones/omicron-nexus.tar.gz /input/package/work/zones/overlay.tar.gz /input/package/work/zones/oximeter.tar.gz /input/package/work/zones/propolis-server.tar.gz /input/package/work/zones/switch-asic.tar.gz /input/package/work/zones/switch-softnpu.tar.gz out/ |
160 | 2024-02-16T07:00:02.974Z | + mv out/omicron-nexus-single-sled.tar.gz out/omicron-nexus.tar.gz |
161 | 2024-02-16T07:00:03.013Z | + mkdir tests |
162 | 2024-02-16T07:00:03.013Z | + for p in /input/ci-tools/work/end-to-end-tests/*.gz |
163 | 2024-02-16T07:00:03.013Z | + ptime -m gunzip |
164 | 2024-02-16T07:00:03.014Z | ++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-22e122cc756fa22c |
165 | 2024-02-16T07:00:03.233Z | |
166 | 2024-02-16T07:00:03.259Z | real 0.231944448 |
167 | 2024-02-16T07:00:03.260Z | user 0.213737862 |
168 | 2024-02-16T07:00:03.260Z | sys 0.017942699 |
169 | 2024-02-16T07:00:03.260Z | trap 0.000019369 |
170 | 2024-02-16T07:00:03.260Z | tflt 0.000000000 |
171 | 2024-02-16T07:00:03.261Z | dflt 0.000000000 |
172 | 2024-02-16T07:00:03.261Z | kflt 0.000000000 |
173 | 2024-02-16T07:00:03.261Z | lock 0.000000000 |
174 | 2024-02-16T07:00:03.261Z | slp 0.000020020 |
175 | 2024-02-16T07:00:03.261Z | lat 0.000021270 |
176 | 2024-02-16T07:00:03.261Z | stop 0.000129113 |
177 | 2024-02-16T07:00:03.261Z | ++ basename /input/ci-tools/work/end-to-end-tests/bootstrap-22e122cc756fa22c |
178 | 2024-02-16T07:00:03.262Z | + chmod a+x tests/bootstrap-22e122cc756fa22c |
179 | 2024-02-16T07:00:03.262Z | + for p in /input/ci-tools/work/end-to-end-tests/*.gz |
180 | 2024-02-16T07:00:03.262Z | + ptime -m gunzip |
181 | 2024-02-16T07:00:03.262Z | ++ basename /input/ci-tools/work/end-to-end-tests/bootstrap |
182 | 2024-02-16T07:00:05.737Z | |
183 | 2024-02-16T07:00:05.764Z | real 2.492645817 |
184 | 2024-02-16T07:00:05.764Z | user 2.347784649 |
185 | 2024-02-16T07:00:05.764Z | sys 0.144467027 |
186 | 2024-02-16T07:00:05.765Z | trap 0.000171625 |
187 | 2024-02-16T07:00:05.765Z | tflt 0.000000000 |
188 | 2024-02-16T07:00:05.765Z | dflt 0.000000000 |
189 | 2024-02-16T07:00:05.765Z | kflt 0.000000000 |
190 | 2024-02-16T07:00:05.765Z | lock 0.000000000 |
191 | 2024-02-16T07:00:05.765Z | slp 0.000020450 |
192 | 2024-02-16T07:00:05.765Z | lat 0.000015950 |
193 | 2024-02-16T07:00:05.766Z | stop 0.000120992 |
194 | 2024-02-16T07:00:05.766Z | ++ basename /input/ci-tools/work/end-to-end-tests/bootstrap |
195 | 2024-02-16T07:00:05.766Z | + chmod a+x tests/bootstrap |
196 | 2024-02-16T07:00:05.766Z | + for p in /input/ci-tools/work/end-to-end-tests/*.gz |
197 | 2024-02-16T07:00:05.766Z | + ptime -m gunzip |
198 | 2024-02-16T07:00:05.766Z | ++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-4eb1211856b7c67f |
199 | 2024-02-16T07:00:08.304Z | |
200 | 2024-02-16T07:00:08.331Z | real 2.554396008 |
201 | 2024-02-16T07:00:08.331Z | user 2.409642040 |
202 | 2024-02-16T07:00:08.331Z | sys 0.144350035 |
203 | 2024-02-16T07:00:08.331Z | trap 0.000170607 |
204 | 2024-02-16T07:00:08.332Z | tflt 0.000000000 |
205 | 2024-02-16T07:00:08.332Z | dflt 0.000000000 |
206 | 2024-02-16T07:00:08.332Z | kflt 0.000000000 |
207 | 2024-02-16T07:00:08.332Z | lock 0.000000000 |
208 | 2024-02-16T07:00:08.332Z | slp 0.000023860 |
209 | 2024-02-16T07:00:08.332Z | lat 0.000021500 |
210 | 2024-02-16T07:00:08.333Z | stop 0.000122472 |
211 | 2024-02-16T07:00:08.333Z | ++ basename /input/ci-tools/work/end-to-end-tests/end_to_end_tests-4eb1211856b7c67f |
212 | 2024-02-16T07:00:08.333Z | + chmod a+x tests/end_to_end_tests-4eb1211856b7c67f |
213 | 2024-02-16T07:00:08.333Z | + bmat address ls |
214 | 2024-02-16T07:00:08.333Z | NAME CIDR FIRST LAST COUNT |
215 | 2024-02-16T07:00:08.333Z | extra 10.151.2.0/24 10.151.2.164 10.151.2.227 64 |
216 | 2024-02-16T07:00:08.333Z | ++ bmat address ls -f extra -Ho first,last,count |
217 | 2024-02-16T07:00:08.333Z | + set -- 10.151.2.164 10.151.2.227 64 |
218 | 2024-02-16T07:00:08.333Z | + EXTRA_IP_START=10.151.2.164 |
219 | 2024-02-16T07:00:08.334Z | + EXTRA_IP_END=10.151.2.227 |
220 | 2024-02-16T07:00:08.334Z | + EXTRA_IP_COUNT=64 |
221 | 2024-02-16T07:00:08.334Z | + (( EXTRA_IP_COUNT >= 32 )) |
222 | 2024-02-16T07:00:08.334Z | + EXTRA_IP_BASE=10.151.2 |
223 | 2024-02-16T07:00:08.334Z | + EXTRA_IP_FOCTET=164 |
224 | 2024-02-16T07:00:08.334Z | + EXTRA_IP_LOCTET=227 |
225 | 2024-02-16T07:00:08.334Z | + SERVICE_IP_POOL_START=10.151.2.164 |
226 | 2024-02-16T07:00:08.335Z | + SERVICE_IP_POOL_END=10.151.2.173 |
227 | 2024-02-16T07:00:08.335Z | + DNS_IP1=10.151.2.164 |
228 | 2024-02-16T07:00:08.335Z | + DNS_IP2=10.151.2.165 |
229 | 2024-02-16T07:00:08.335Z | + UPLINK_IP=10.151.2.174 |
230 | 2024-02-16T07:00:08.335Z | + IPPOOL_START=10.151.2.175 |
231 | 2024-02-16T07:00:08.335Z | + IPPOOL_END=10.151.2.227 |
232 | 2024-02-16T07:00:08.362Z | ++ ipadm show-addr -po type,addr |
233 | 2024-02-16T07:00:08.362Z | ++ awk '-F[:/]' '$1 == "dhcp" {print $2}' |
234 | 2024-02-16T07:00:08.362Z | + GATEWAY_IP=10.151.2.100 |
235 | 2024-02-16T07:00:08.362Z | + [[ -n 10.151.2.100 ]] |
236 | 2024-02-16T07:00:08.362Z | + ping -s 10.151.2.100 56 1 |
237 | 2024-02-16T07:00:08.363Z | PING 10.151.2.100: 56 data bytes |
238 | 2024-02-16T07:00:08.363Z | 64 bytes from 10.151.2.100: icmp_seq=0. time=0.011 ms |
239 | 2024-02-16T07:00:08.363Z | |
240 | 2024-02-16T07:00:08.363Z | ----10.151.2.100 PING Statistics---- |
241 | 2024-02-16T07:00:08.363Z | 1 packets transmitted, 1 packets received, 0% packet loss |
242 | 2024-02-16T07:00:08.363Z | round-trip (ms) min/avg/max/stddev = 0.011/0.011/0.011/-nan |
243 | 2024-02-16T07:00:08.363Z | ++ arp -an |
244 | 2024-02-16T07:00:08.364Z | ++ awk -vgw=10.151.2.100 '$2 == gw {print $NF}' |
245 | 2024-02-16T07:00:08.364Z | + GATEWAY_MAC=18:c0:4d:0d:a0:2a |
246 | 2024-02-16T07:00:08.364Z | + routeadm -e ipv4-forwarding -u |
247 | 2024-02-16T07:00:08.466Z | + PXA_START=10.151.2.164 |
248 | 2024-02-16T07:00:08.466Z | + PXA_END=10.151.2.227 |
249 | 2024-02-16T07:00:08.493Z | + export GATEWAY_IP GATEWAY_MAC PXA_START PXA_END |
250 | 2024-02-16T07:00:08.493Z | + pfexec zpool create -f scratch c1t1d0 c2t1d0 |
251 | 2024-02-16T07:00:08.673Z | + ZPOOL_VDEV_DIR=/scratch |
252 | 2024-02-16T07:00:08.673Z | + ptime -m pfexec ./tools/create_virtual_hardware.sh |
253 | 2024-02-16T07:00:08.700Z | +++ dirname ./tools/create_virtual_hardware.sh |
254 | 2024-02-16T07:00:08.701Z | ++ cd ./tools |
255 | 2024-02-16T07:00:08.701Z | ++ pwd |
256 | 2024-02-16T07:00:08.701Z | + SOURCE_DIR=/opt/oxide/work/tools |
257 | 2024-02-16T07:00:08.701Z | + OMICRON_TOP=/opt/oxide/work/tools/.. |
258 | 2024-02-16T07:00:08.701Z | + SOFTNPU_MODE=zone |
259 | 2024-02-16T07:00:08.701Z | + . /opt/oxide/work/tools/virtual_hardware.sh |
260 | 2024-02-16T07:00:08.701Z | ++ MARKER=/etc/opt/oxide/NO_INSTALL |
261 | 2024-02-16T07:00:08.702Z | ++ [[ -f /etc/opt/oxide/NO_INSTALL ]] |
262 | 2024-02-16T07:00:08.702Z | ++ dladm show-phys -p -o LINK |
263 | 2024-02-16T07:00:08.702Z | ++ head -1 |
264 | 2024-02-16T07:00:08.702Z | + PHYSICAL_LINK=igb0 |
265 | 2024-02-16T07:00:08.702Z | + echo 'Using igb0 as physical link' |
266 | 2024-02-16T07:00:08.702Z | Using igb0 as physical link |
267 | 2024-02-16T07:00:08.702Z | + ensure_run_as_root |
268 | 2024-02-16T07:00:08.730Z | ++ id -u |
269 | 2024-02-16T07:00:08.730Z | + [[ 0 -ne 0 ]] |
270 | 2024-02-16T07:00:08.730Z | + ensure_zpools |
271 | 2024-02-16T07:00:08.730Z | + ZPOOL_TYPES=('oxp_' 'oxi_') |
272 | 2024-02-16T07:00:08.731Z | + for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}" |
273 | 2024-02-16T07:00:08.731Z | + readarray -t ZPOOLS |
274 | 2024-02-16T07:00:08.731Z | ++ grep '"oxp_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml |
275 | 2024-02-16T07:00:08.731Z | ++ sed 's/[ ",]//g' |
276 | 2024-02-16T07:00:08.731Z | + for ZPOOL in "${ZPOOLS[@]}" |
277 | 2024-02-16T07:00:08.731Z | + echo 'Zpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b]' |
278 | 2024-02-16T07:00:08.732Z | Zpool: [oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b] |
279 | 2024-02-16T07:00:08.732Z | + VDEV_PATH=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev |
280 | 2024-02-16T07:00:08.732Z | + [[ -f /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]] |
281 | 2024-02-16T07:00:08.732Z | + dd if=/dev/zero of=/scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G |
282 | 2024-02-16T07:00:08.732Z | 0+0 records in |
283 | 2024-02-16T07:00:08.732Z | 0+0 records out |
284 | 2024-02-16T07:00:08.732Z | 0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec) |
285 | 2024-02-16T07:00:08.732Z | + success 'ZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists' |
286 | 2024-02-16T07:00:08.733Z | + set +x |
287 | 2024-02-16T07:00:08.733Z | ZFS vdev /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists |
288 | 2024-02-16T07:00:08.733Z | ++ zpool list -o name |
289 | 2024-02-16T07:00:08.733Z | ++ grep oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b |
290 | 2024-02-16T07:00:08.733Z | + [[ -z '' ]] |
291 | 2024-02-16T07:00:08.733Z | + zpool create -f oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev |
292 | 2024-02-16T07:00:08.765Z | + success 'ZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists' |
293 | 2024-02-16T07:00:08.792Z | + set +x |
294 | 2024-02-16T07:00:08.792Z | ZFS zpool oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b exists |
295 | 2024-02-16T07:00:08.792Z | + for ZPOOL in "${ZPOOLS[@]}" |
296 | 2024-02-16T07:00:08.792Z | + echo 'Zpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03]' |
297 | 2024-02-16T07:00:08.792Z | + VDEV_PATH=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev |
298 | 2024-02-16T07:00:08.792Z | Zpool: [oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03] |
299 | 2024-02-16T07:00:08.792Z | + [[ -f /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]] |
300 | 2024-02-16T07:00:08.792Z | + dd if=/dev/zero of=/scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G |
301 | 2024-02-16T07:00:08.793Z | 0+0 records in |
302 | 2024-02-16T07:00:08.793Z | 0+0 records out |
303 | 2024-02-16T07:00:08.793Z | 0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec) |
304 | 2024-02-16T07:00:08.793Z | + success 'ZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists' |
305 | 2024-02-16T07:00:08.793Z | + set +x |
306 | 2024-02-16T07:00:08.793Z | ZFS vdev /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists |
307 | 2024-02-16T07:00:08.793Z | ++ zpool list -o name |
308 | 2024-02-16T07:00:08.793Z | ++ grep oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 |
309 | 2024-02-16T07:00:08.793Z | + [[ -z '' ]] |
310 | 2024-02-16T07:00:08.793Z | + zpool create -f oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev |
311 | 2024-02-16T07:00:08.818Z | + success 'ZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists' |
312 | 2024-02-16T07:00:08.819Z | + set +x |
313 | 2024-02-16T07:00:08.819Z | ZFS zpool oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists |
314 | 2024-02-16T07:00:08.819Z | + for ZPOOL in "${ZPOOLS[@]}" |
315 | 2024-02-16T07:00:08.819Z | + echo 'Zpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03]' |
316 | 2024-02-16T07:00:08.819Z | Zpool: [oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03] |
317 | 2024-02-16T07:00:08.819Z | + VDEV_PATH=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev |
318 | 2024-02-16T07:00:08.819Z | + [[ -f /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]] |
319 | 2024-02-16T07:00:08.819Z | + dd if=/dev/zero of=/scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G |
320 | 2024-02-16T07:00:08.819Z | 0+0 records in |
321 | 2024-02-16T07:00:08.819Z | 0+0 records out |
322 | 2024-02-16T07:00:08.819Z | 0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec) |
323 | 2024-02-16T07:00:08.819Z | + success 'ZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists' |
324 | 2024-02-16T07:00:08.819Z | + set +x |
325 | 2024-02-16T07:00:08.820Z | ZFS vdev /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists |
326 | 2024-02-16T07:00:08.820Z | ++ zpool list -o name |
327 | 2024-02-16T07:00:08.820Z | ++ grep oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 |
328 | 2024-02-16T07:00:08.845Z | + [[ -z '' ]] |
329 | 2024-02-16T07:00:08.845Z | + zpool create -f oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev |
330 | 2024-02-16T07:00:08.872Z | + success 'ZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists' |
331 | 2024-02-16T07:00:08.872Z | + set +x |
332 | 2024-02-16T07:00:08.872Z | ZFS zpool oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03 exists |
333 | 2024-02-16T07:00:08.872Z | + for ZPOOL in "${ZPOOLS[@]}" |
334 | 2024-02-16T07:00:08.872Z | + echo 'Zpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03]' |
335 | 2024-02-16T07:00:08.872Z | Zpool: [oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03] |
336 | 2024-02-16T07:00:08.873Z | + VDEV_PATH=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev |
337 | 2024-02-16T07:00:08.873Z | + [[ -f /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]] |
338 | 2024-02-16T07:00:08.873Z | + dd if=/dev/zero of=/scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G |
339 | 2024-02-16T07:00:08.873Z | 0+0 records in |
340 | 2024-02-16T07:00:08.873Z | 0+0 records out |
341 | 2024-02-16T07:00:08.873Z | 0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec) |
342 | 2024-02-16T07:00:08.873Z | + success 'ZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists' |
343 | 2024-02-16T07:00:08.873Z | + set +x |
344 | 2024-02-16T07:00:08.874Z | ZFS vdev /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists |
345 | 2024-02-16T07:00:08.874Z | ++ zpool list -o name |
346 | 2024-02-16T07:00:08.874Z | ++ grep oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 |
347 | 2024-02-16T07:00:08.874Z | + [[ -z '' ]] |
348 | 2024-02-16T07:00:08.874Z | + zpool create -f oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev |
349 | 2024-02-16T07:00:08.911Z | + success 'ZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists' |
350 | 2024-02-16T07:00:08.911Z | + set +x |
351 | 2024-02-16T07:00:08.937Z | ZFS zpool oxp_14b4dc87-ab46-49fb-a4b4-d361ae214c03 exists |
352 | 2024-02-16T07:00:08.938Z | + for ZPOOL in "${ZPOOLS[@]}" |
353 | 2024-02-16T07:00:08.938Z | + echo 'Zpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03]' |
354 | 2024-02-16T07:00:08.938Z | Zpool: [oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03] |
355 | 2024-02-16T07:00:08.938Z | + VDEV_PATH=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev |
356 | 2024-02-16T07:00:08.938Z | + [[ -f /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev ]] |
357 | 2024-02-16T07:00:08.938Z | + dd if=/dev/zero of=/scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev bs=1 count=0 seek=20G |
358 | 2024-02-16T07:00:08.938Z | 0+0 records in |
359 | 2024-02-16T07:00:08.939Z | 0+0 records out |
360 | 2024-02-16T07:00:08.939Z | 0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec) |
361 | 2024-02-16T07:00:08.939Z | + success 'ZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists' |
362 | 2024-02-16T07:00:08.939Z | + set +x |
363 | 2024-02-16T07:00:08.939Z | ZFS vdev /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev exists |
364 | 2024-02-16T07:00:08.939Z | ++ zpool list -o name |
365 | 2024-02-16T07:00:08.940Z | ++ grep oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 |
366 | 2024-02-16T07:00:08.940Z | + [[ -z '' ]] |
367 | 2024-02-16T07:00:08.940Z | + zpool create -f oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 /scratch/oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03.vdev |
368 | 2024-02-16T07:00:08.965Z | + success 'ZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists' |
369 | 2024-02-16T07:00:08.966Z | + set +x |
370 | 2024-02-16T07:00:08.966Z | ZFS zpool oxp_24b4dc87-ab46-49fb-a4b4-d361ae214c03 exists |
371 | 2024-02-16T07:00:08.966Z | + for ZPOOL in "${ZPOOLS[@]}" |
372 | 2024-02-16T07:00:08.966Z | + echo 'Zpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930]' |
373 | 2024-02-16T07:00:08.966Z | Zpool: [oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930] |
374 | 2024-02-16T07:00:08.966Z | + VDEV_PATH=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev |
375 | 2024-02-16T07:00:08.966Z | + [[ -f /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev ]] |
376 | 2024-02-16T07:00:08.966Z | + dd if=/dev/zero of=/scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev bs=1 count=0 seek=20G |
377 | 2024-02-16T07:00:08.966Z | 0+0 records in |
378 | 2024-02-16T07:00:08.966Z | 0+0 records out |
379 | 2024-02-16T07:00:08.967Z | 0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec) |
380 | 2024-02-16T07:00:08.967Z | + success 'ZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists' |
381 | 2024-02-16T07:00:08.967Z | + set +x |
382 | 2024-02-16T07:00:08.967Z | ZFS vdev /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev exists |
383 | 2024-02-16T07:00:08.967Z | ++ zpool list -o name |
384 | 2024-02-16T07:00:08.967Z | ++ grep oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 |
385 | 2024-02-16T07:00:08.993Z | + [[ -z '' ]] |
386 | 2024-02-16T07:00:08.993Z | + zpool create -f oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 /scratch/oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930.vdev |
387 | 2024-02-16T07:00:09.019Z | + success 'ZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists' |
388 | 2024-02-16T07:00:09.019Z | + set +x |
389 | 2024-02-16T07:00:09.019Z | ZFS zpool oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930 exists |
390 | 2024-02-16T07:00:09.020Z | + for ZPOOL in "${ZPOOLS[@]}" |
391 | 2024-02-16T07:00:09.020Z | + echo 'Zpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60]' |
392 | 2024-02-16T07:00:09.020Z | Zpool: [oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60] |
393 | 2024-02-16T07:00:09.020Z | + VDEV_PATH=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev |
394 | 2024-02-16T07:00:09.020Z | + [[ -f /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev ]] |
395 | 2024-02-16T07:00:09.020Z | + dd if=/dev/zero of=/scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev bs=1 count=0 seek=20G |
396 | 2024-02-16T07:00:09.020Z | 0+0 records in |
397 | 2024-02-16T07:00:09.020Z | 0+0 records out |
398 | 2024-02-16T07:00:09.021Z | 0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec) |
399 | 2024-02-16T07:00:09.021Z | + success 'ZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists' |
400 | 2024-02-16T07:00:09.021Z | + set +x |
401 | 2024-02-16T07:00:09.021Z | ZFS vdev /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev exists |
402 | 2024-02-16T07:00:09.021Z | ++ zpool list -o name |
403 | 2024-02-16T07:00:09.021Z | ++ grep oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 |
404 | 2024-02-16T07:00:09.047Z | + [[ -z '' ]] |
405 | 2024-02-16T07:00:09.047Z | + zpool create -f oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 /scratch/oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60.vdev |
406 | 2024-02-16T07:00:09.072Z | + success 'ZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists' |
407 | 2024-02-16T07:00:09.073Z | + set +x |
408 | 2024-02-16T07:00:09.073Z | ZFS zpool oxp_ceb4461c-cf56-4719-ad3c-14430bfdfb60 exists |
409 | 2024-02-16T07:00:09.073Z | + for ZPOOL in "${ZPOOLS[@]}" |
410 | 2024-02-16T07:00:09.073Z | Zpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f] |
411 | 2024-02-16T07:00:09.073Z | + echo 'Zpool: [oxp_31bd71cd-4736-4a12-a387-9b74b050396f]' |
412 | 2024-02-16T07:00:09.074Z | + VDEV_PATH=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev |
413 | 2024-02-16T07:00:09.074Z | + [[ -f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev ]] |
414 | 2024-02-16T07:00:09.074Z | + dd if=/dev/zero of=/scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev bs=1 count=0 seek=20G |
415 | 2024-02-16T07:00:09.074Z | 0+0 records in |
416 | 2024-02-16T07:00:09.074Z | 0+0 records out |
417 | 2024-02-16T07:00:09.074Z | 0 bytes (0 iB) transferred in 0.000005 secs (0 iB/sec) |
418 | 2024-02-16T07:00:09.074Z | + success 'ZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists' |
419 | 2024-02-16T07:00:09.075Z | + set +x |
420 | 2024-02-16T07:00:09.075Z | ZFS vdev /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev exists |
421 | 2024-02-16T07:00:09.075Z | ++ zpool list -o name |
422 | 2024-02-16T07:00:09.075Z | ++ grep oxp_31bd71cd-4736-4a12-a387-9b74b050396f |
423 | 2024-02-16T07:00:09.075Z | + [[ -z '' ]] |
424 | 2024-02-16T07:00:09.075Z | + zpool create -f oxp_31bd71cd-4736-4a12-a387-9b74b050396f /scratch/oxp_31bd71cd-4736-4a12-a387-9b74b050396f.vdev |
425 | 2024-02-16T07:00:09.101Z | + success 'ZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists' |
426 | 2024-02-16T07:00:09.101Z | + set +x |
427 | 2024-02-16T07:00:09.127Z | ZFS zpool oxp_31bd71cd-4736-4a12-a387-9b74b050396f exists |
428 | 2024-02-16T07:00:09.127Z | + for ZPOOL in "${ZPOOLS[@]}" |
429 | 2024-02-16T07:00:09.127Z | + echo 'Zpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7]' |
430 | 2024-02-16T07:00:09.127Z | Zpool: [oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7] |
431 | 2024-02-16T07:00:09.127Z | + VDEV_PATH=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev |
432 | 2024-02-16T07:00:09.127Z | + [[ -f /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev ]] |
433 | 2024-02-16T07:00:09.128Z | + dd if=/dev/zero of=/scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev bs=1 count=0 seek=20G |
434 | 2024-02-16T07:00:09.128Z | 0+0 records in |
435 | 2024-02-16T07:00:09.128Z | 0+0 records out |
436 | 2024-02-16T07:00:09.128Z | 0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec) |
437 | 2024-02-16T07:00:09.128Z | + success 'ZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists' |
438 | 2024-02-16T07:00:09.128Z | + set +x |
439 | 2024-02-16T07:00:09.128Z | ZFS vdev /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev exists |
440 | 2024-02-16T07:00:09.128Z | ++ zpool list -o name |
441 | 2024-02-16T07:00:09.128Z | ++ grep oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 |
442 | 2024-02-16T07:00:09.129Z | + [[ -z '' ]] |
443 | 2024-02-16T07:00:09.129Z | + zpool create -f oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 /scratch/oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7.vdev |
444 | 2024-02-16T07:00:09.154Z | + success 'ZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists' |
445 | 2024-02-16T07:00:09.154Z | + set +x |
446 | 2024-02-16T07:00:09.155Z | ZFS zpool oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7 exists |
447 | 2024-02-16T07:00:09.155Z | + for ZPOOL_TYPE in "${ZPOOL_TYPES[@]}" |
448 | 2024-02-16T07:00:09.155Z | + readarray -t ZPOOLS |
449 | 2024-02-16T07:00:09.155Z | ++ grep '"oxi_' /opt/oxide/work/tools/../smf/sled-agent/non-gimlet/config.toml |
450 | 2024-02-16T07:00:09.180Z | ++ sed 's/[ ",]//g' |
451 | 2024-02-16T07:00:09.181Z | + for ZPOOL in "${ZPOOLS[@]}" |
452 | 2024-02-16T07:00:09.181Z | + echo 'Zpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b]' |
453 | 2024-02-16T07:00:09.181Z | Zpool: [oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b] |
454 | 2024-02-16T07:00:09.181Z | + VDEV_PATH=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev |
455 | 2024-02-16T07:00:09.181Z | + [[ -f /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]] |
456 | 2024-02-16T07:00:09.181Z | + dd if=/dev/zero of=/scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G |
457 | 2024-02-16T07:00:09.181Z | 0+0 records in |
458 | 2024-02-16T07:00:09.182Z | 0+0 records out |
459 | 2024-02-16T07:00:09.182Z | 0 bytes (0 iB) transferred in 0.000003 secs (0 iB/sec) |
460 | 2024-02-16T07:00:09.182Z | + success 'ZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists' |
461 | 2024-02-16T07:00:09.182Z | + set +x |
462 | 2024-02-16T07:00:09.182Z | ZFS vdev /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists |
463 | 2024-02-16T07:00:09.182Z | ++ zpool list -o name |
464 | 2024-02-16T07:00:09.183Z | ++ grep oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b |
465 | 2024-02-16T07:00:09.183Z | + [[ -z '' ]] |
466 | 2024-02-16T07:00:09.183Z | + zpool create -f oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev |
467 | 2024-02-16T07:00:09.208Z | + success 'ZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists' |
468 | 2024-02-16T07:00:09.209Z | + set +x |
469 | 2024-02-16T07:00:09.209Z | ZFS zpool oxi_a462a7f7-b628-40fe-80ff-4e4189e2d62b exists |
470 | 2024-02-16T07:00:09.209Z | + for ZPOOL in "${ZPOOLS[@]}" |
471 | 2024-02-16T07:00:09.209Z | + echo 'Zpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b]' |
472 | 2024-02-16T07:00:09.209Z | Zpool: [oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b] |
473 | 2024-02-16T07:00:09.209Z | + VDEV_PATH=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev |
474 | 2024-02-16T07:00:09.209Z | + [[ -f /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev ]] |
475 | 2024-02-16T07:00:09.210Z | + dd if=/dev/zero of=/scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev bs=1 count=0 seek=20G |
476 | 2024-02-16T07:00:09.210Z | 0+0 records in |
477 | 2024-02-16T07:00:09.210Z | 0+0 records out |
478 | 2024-02-16T07:00:09.210Z | 0 bytes (0 iB) transferred in 0.000004 secs (0 iB/sec) |
479 | 2024-02-16T07:00:09.210Z | + success 'ZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists' |
480 | 2024-02-16T07:00:09.210Z | + set +x |
481 | 2024-02-16T07:00:09.210Z | ZFS vdev /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev exists |
482 | 2024-02-16T07:00:09.265Z | ++ zpool list -o name |
483 | 2024-02-16T07:00:09.266Z | ++ grep oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b |
484 | 2024-02-16T07:00:09.266Z | + [[ -z '' ]] |
485 | 2024-02-16T07:00:09.266Z | + zpool create -f oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b /scratch/oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b.vdev |
486 | 2024-02-16T07:00:09.266Z | + success 'ZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists' |
487 | 2024-02-16T07:00:09.266Z | + set +x |
488 | 2024-02-16T07:00:09.266Z | ZFS zpool oxi_b462a7f7-b628-40fe-80ff-4e4189e2d62b exists |
489 | 2024-02-16T07:00:09.267Z | + [[ zone == \z\o\n\e ]] |
490 | 2024-02-16T07:00:09.267Z | + ensure_simulated_links igb0 |
491 | 2024-02-16T07:00:09.267Z | + local PHYSICAL_LINK=igb0 |
492 | 2024-02-16T07:00:09.267Z | + INDICES=("0" "1") |
493 | 2024-02-16T07:00:09.267Z | + for I in "${INDICES[@]}" |
494 | 2024-02-16T07:00:09.268Z | ++ get_simnet_name_if_exists net0 |
495 | 2024-02-16T07:00:09.268Z | ++ dladm show-simnet -p -o LINK net0 |
496 | 2024-02-16T07:00:09.268Z | ++ echo '' |
497 | 2024-02-16T07:00:09.268Z | + [[ -z '' ]] |
498 | 2024-02-16T07:00:09.268Z | + dladm create-simnet -t net0 |
499 | 2024-02-16T07:00:09.296Z | + dladm create-simnet -t sc0_0 |
500 | 2024-02-16T07:00:09.296Z | + dladm modify-simnet -t -p net0 sc0_0 |
501 | 2024-02-16T07:00:09.296Z | + dladm set-linkprop -p mtu=9000 sc0_0 |
502 | 2024-02-16T07:00:09.296Z | + success 'Simnet net0/sc0_0 exists' |
503 | 2024-02-16T07:00:09.296Z | + set +x |
504 | 2024-02-16T07:00:09.296Z | Simnet net0/sc0_0 exists |
505 | 2024-02-16T07:00:09.296Z | + for I in "${INDICES[@]}" |
506 | 2024-02-16T07:00:09.297Z | ++ get_simnet_name_if_exists net1 |
507 | 2024-02-16T07:00:09.297Z | ++ dladm show-simnet -p -o LINK net1 |
508 | 2024-02-16T07:00:09.297Z | ++ echo '' |
509 | 2024-02-16T07:00:09.297Z | + [[ -z '' ]] |
510 | 2024-02-16T07:00:09.297Z | + dladm create-simnet -t net1 |
511 | 2024-02-16T07:00:09.297Z | + dladm create-simnet -t sc1_0 |
512 | 2024-02-16T07:00:09.297Z | + dladm modify-simnet -t -p net1 sc1_0 |
513 | 2024-02-16T07:00:09.324Z | + dladm set-linkprop -p mtu=9000 sc1_0 |
514 | 2024-02-16T07:00:09.324Z | + success 'Simnet net1/sc1_0 exists' |
515 | 2024-02-16T07:00:09.324Z | + set +x |
516 | 2024-02-16T07:00:09.324Z | Simnet net1/sc1_0 exists |
517 | 2024-02-16T07:00:09.324Z | ++ get_vnic_name_if_exists sc0_1 |
518 | 2024-02-16T07:00:09.325Z | ++ dladm show-vnic -p -o LINK sc0_1 |
519 | 2024-02-16T07:00:09.325Z | ++ echo '' |
520 | 2024-02-16T07:00:09.325Z | + [[ -z '' ]] |
521 | 2024-02-16T07:00:09.325Z | + dladm create-vnic -t sc0_1 -l igb0 -m a8:e1:de:01:70:1d |
522 | 2024-02-16T07:00:09.325Z | + [[ -v PROMISC_FILT_OFF ]] |
523 | 2024-02-16T07:00:09.325Z | + success 'Vnic sc0_1 exists' |
524 | 2024-02-16T07:00:09.326Z | + set +x |
525 | 2024-02-16T07:00:09.326Z | Vnic sc0_1 exists |
526 | 2024-02-16T07:00:09.326Z | + warn_if_no_proxy_arp |
527 | 2024-02-16T07:00:09.326Z | + [[ -v PXA_START ]] |
528 | 2024-02-16T07:00:09.326Z | + [[ -v PXA_END ]] |
529 | 2024-02-16T07:00:09.326Z | + ensure_softnpu_zone |
530 | 2024-02-16T07:00:09.326Z | + zoneadm list |
531 | 2024-02-16T07:00:09.326Z | + grep -q sidecar_softnpu |
532 | 2024-02-16T07:00:09.326Z | + [[ -f out/npuzone/npuzone ]] |
533 | 2024-02-16T07:00:09.327Z | + out/npuzone/npuzone create sidecar --omicron-zone --ports sc0_0,tfportrear0_0 --ports sc0_1,tfportqsfp0_0 --sidecar-lite-commit e3ea4b495ba0a71801ded0776ae4bbd31df57e26 --softnpu-commit dbab082dfa89da5db5ca2325c257089d2f130092 |
534 | 2024-02-16T07:00:09.353Z | remote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/softnpu/image/dbab082dfa89da5db5ca2325c257089d2f130092/softnpu.sha256.txt |
535 | 2024-02-16T07:00:15.878Z | downloading softnpu ...done |
536 | 2024-02-16T07:00:15.904Z | remote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/libsidecar_lite.so.sha256.txt |
537 | 2024-02-16T07:00:20.850Z | downloading asic_program.so ...done |
538 | 2024-02-16T07:00:20.876Z | remote url: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/sidecar-lite/release/e3ea4b495ba0a71801ded0776ae4bbd31df57e26/scadm.sha256.txt |
539 | 2024-02-16T07:00:27.186Z | downloading scadm ...done |
540 | 2024-02-16T07:00:27.211Z | creating port tfportrear0_0 for sc0_0 |
541 | 2024-02-16T07:00:27.211Z | creating port tfportqsfp0_0 for sc0_1 |
542 | 2024-02-16T07:00:27.236Z | softnpu zone setup |
543 | 2024-02-16T07:00:27.236Z | configure zone |
544 | 2024-02-16T07:00:27.236Z | install zone |
545 | 2024-02-16T07:00:28.278Z | boot zone |
546 | 2024-02-16T07:00:31.398Z | [sidecar_softnpu] svcs milestone/network |
547 | 2024-02-16T07:00:31.424Z | STATE STIME FMRI |
548 | 2024-02-16T07:00:31.424Z | online 7:00:29 svc:/milestone/network:default |
549 | 2024-02-16T07:00:31.424Z | [sidecar_softnpu] /softnpu/npu start |
550 | 2024-02-16T07:00:31.449Z | |
551 | 2024-02-16T07:00:31.474Z | + /opt/oxide/work/tools/scrimlet/softnpu-init.sh |
552 | 2024-02-16T07:00:31.474Z | + GATEWAY_IP=10.151.2.100 |
553 | 2024-02-16T07:00:31.474Z | + echo 'Using 10.151.2.100 as gateway ip' |
554 | 2024-02-16T07:00:31.474Z | Using 10.151.2.100 as gateway ip |
555 | 2024-02-16T07:00:31.474Z | + [[ ! -v GATEWAY_MAC ]] |
556 | 2024-02-16T07:00:31.474Z | + GATEWAY_MAC=18:c0:4d:0d:a0:2a |
557 | 2024-02-16T07:00:31.474Z | ++ grep -c -E '^([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}$' /dev/fd/63 |
558 | 2024-02-16T07:00:31.474Z | +++ echo 18:c0:4d:0d:a0:2a |
559 | 2024-02-16T07:00:31.474Z | + COUNT=1 |
560 | 2024-02-16T07:00:31.475Z | + [[ 1 -ne 1 ]] |
561 | 2024-02-16T07:00:31.475Z | + echo 'Using 18:c0:4d:0d:a0:2a as gateway mac' |
562 | 2024-02-16T07:00:31.475Z | Using 18:c0:4d:0d:a0:2a as gateway mac |
563 | 2024-02-16T07:00:31.475Z | + z_scadm add-arp-entry 10.151.2.100 18:c0:4d:0d:a0:2a |
564 | 2024-02-16T07:00:31.475Z | + pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-arp-entry 10.151.2.100 18:c0:4d:0d:a0:2a |
565 | 2024-02-16T07:00:31.499Z | + PXA_MAC=a8:e1:de:01:70:1d |
566 | 2024-02-16T07:00:31.499Z | + [[ -v PXA_START ]] |
567 | 2024-02-16T07:00:31.500Z | + z_scadm add-proxy-arp 10.151.2.164 10.151.2.227 a8:e1:de:01:70:1d |
568 | 2024-02-16T07:00:31.500Z | + pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone add-proxy-arp 10.151.2.164 10.151.2.227 a8:e1:de:01:70:1d |
569 | 2024-02-16T07:00:31.524Z | + z_scadm dump-state |
570 | 2024-02-16T07:00:31.524Z | + pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state |
571 | 2024-02-16T07:00:31.549Z | local v6: |
572 | 2024-02-16T07:00:31.549Z | local v4: |
573 | 2024-02-16T07:00:31.549Z | router v6: |
574 | 2024-02-16T07:00:31.549Z | router v4_idx: |
575 | 2024-02-16T07:00:31.549Z | router v4_routes: |
576 | 2024-02-16T07:00:31.549Z | resolver v4: |
577 | 2024-02-16T07:00:31.549Z | 10.151.2.100 -> 18:c0:4d:0d:a0:2a |
578 | 2024-02-16T07:00:31.549Z | resolver v6: |
579 | 2024-02-16T07:00:31.549Z | nat v4: |
580 | 2024-02-16T07:00:31.549Z | nat_v6: |
581 | 2024-02-16T07:00:31.549Z | port_mac: |
582 | 2024-02-16T07:00:31.549Z | proxy arp: |
583 | 2024-02-16T07:00:31.549Z | 10.151.2.164/10.151.2.227: a8:e1:de:01:70:1d |
584 | 2024-02-16T07:00:31.549Z | + success 'softnpu zone exists' |
585 | 2024-02-16T07:00:31.549Z | + set +x |
586 | 2024-02-16T07:00:31.549Z | softnpu zone exists |
587 | 2024-02-16T07:00:31.550Z | |
588 | 2024-02-16T07:00:31.550Z | real 22.872712037 |
589 | 2024-02-16T07:00:31.550Z | user 0.710730136 |
590 | 2024-02-16T07:00:31.550Z | sys 1.602080393 |
591 | 2024-02-16T07:00:31.550Z | trap 0.000072129 |
592 | 2024-02-16T07:00:31.550Z | tflt 0.000214968 |
593 | 2024-02-16T07:00:31.550Z | dflt 0.000973054 |
594 | 2024-02-16T07:00:31.550Z | kflt 0.000000000 |
595 | 2024-02-16T07:00:31.550Z | lock 17:38.379648781 |
596 | 2024-02-16T07:00:31.550Z | slp 1:08.191340168 |
597 | 2024-02-16T07:00:31.550Z | lat 0.082326515 |
598 | 2024-02-16T07:00:31.550Z | stop 0.341505820 |
599 | 2024-02-16T07:00:31.575Z | + tar xf out/omicron-sled-agent.tar pkg/config-rss.toml |
600 | 2024-02-16T07:00:31.600Z | ++ sed -n 's/silo_name = "\(.*\)"/\1/p' pkg/config-rss.toml |
601 | 2024-02-16T07:00:31.600Z | + SILO_NAME=recovery |
602 | 2024-02-16T07:00:31.600Z | ++ sed -n 's/external_dns_zone_name = "\(.*\)"/\1/p' pkg/config-rss.toml |
603 | 2024-02-16T07:00:31.600Z | + EXTERNAL_DNS_DOMAIN=oxide.test |
604 | 2024-02-16T07:00:31.600Z | + sed -i~ ' |
605 | 2024-02-16T07:00:31.600Z | /^external_dns_ips/c\ |
606 | 2024-02-16T07:00:31.600Z | external_dns_ips = [ "10.151.2.164", "10.151.2.165" ] |
607 | 2024-02-16T07:00:31.600Z | /^\[\[internal_services_ip_pool_ranges/,/^$/ { |
608 | 2024-02-16T07:00:31.600Z | /^first/c\ |
609 | 2024-02-16T07:00:31.600Z | first = "10.151.2.164" |
610 | 2024-02-16T07:00:31.600Z | /^last/c\ |
611 | 2024-02-16T07:00:31.600Z | last = "10.151.2.173" |
612 | 2024-02-16T07:00:31.600Z | } |
613 | 2024-02-16T07:00:31.600Z | /^\[rack_network_config/,/^$/ { |
614 | 2024-02-16T07:00:31.600Z | /^infra_ip_first/c\ |
615 | 2024-02-16T07:00:31.601Z | infra_ip_first = "10.151.2.174" |
616 | 2024-02-16T07:00:31.601Z | /^infra_ip_last/c\ |
617 | 2024-02-16T07:00:31.601Z | infra_ip_last = "10.151.2.174" |
618 | 2024-02-16T07:00:31.601Z | } |
619 | 2024-02-16T07:00:31.601Z | /^\[\[rack_network_config.ports/,/^$/ { |
620 | 2024-02-16T07:00:31.601Z | /^routes/c\ |
621 | 2024-02-16T07:00:31.601Z | routes = \[{nexthop = "10.151.2.100", destination = "0.0.0.0/0"}\] |
622 | 2024-02-16T07:00:31.601Z | /^addresses/c\ |
623 | 2024-02-16T07:00:31.601Z | addresses = \["10.151.2.174/24"\] |
624 | 2024-02-16T07:00:31.601Z | } |
625 | 2024-02-16T07:00:31.601Z | ' pkg/config-rss.toml |
626 | 2024-02-16T07:00:31.601Z | + diff -u pkg/config-rss.toml~ pkg/config-rss.toml |
627 | 2024-02-16T07:00:31.601Z | --- pkg/config-rss.toml~ Mon Jul 24 01:21:28 2006 |
628 | 2024-02-16T07:00:31.601Z | +++ pkg/config-rss.toml Fri Feb 16 07:00:31 2024 |
629 | 2024-02-16T07:00:31.601Z | @@ -33,7 +33,7 @@ |
630 | 2024-02-16T07:00:31.601Z | # the DNS domain delegated to the rack by the customer. Each of these addresses |
631 | 2024-02-16T07:00:31.601Z | # must be contained in one of the "internal services" IP Pool ranges listed |
632 | 2024-02-16T07:00:31.601Z | # below. |
633 | 2024-02-16T07:00:31.601Z | -external_dns_ips = [ "192.168.1.20", "192.168.1.21" ] |
634 | 2024-02-16T07:00:31.601Z | +external_dns_ips = [ "10.151.2.164", "10.151.2.165" ] |
635 | 2024-02-16T07:00:31.601Z | |
636 | 2024-02-16T07:00:31.602Z | # Initial TLS certificates for the external API |
637 | 2024-02-16T07:00:31.602Z | # |
638 | 2024-02-16T07:00:31.602Z | @@ -69,8 +69,8 @@ |
639 | 2024-02-16T07:00:31.602Z | # |
640 | 2024-02-16T07:00:31.602Z | # For more on this and what to put here, see docs/how-to-run.adoc. |
641 | 2024-02-16T07:00:31.602Z | [[internal_services_ip_pool_ranges]] |
642 | 2024-02-16T07:00:31.602Z | -first = "192.168.1.20" |
643 | 2024-02-16T07:00:31.602Z | -last = "192.168.1.29" |
644 | 2024-02-16T07:00:31.602Z | +first = "10.151.2.164" |
645 | 2024-02-16T07:00:31.602Z | +last = "10.151.2.173" |
646 | 2024-02-16T07:00:31.602Z | |
647 | 2024-02-16T07:00:31.602Z | # TODO - this configuration is subject to change going forward. Ultimately these |
648 | 2024-02-16T07:00:31.602Z | # parameters should be provided to the control plane via wicket, but we need to |
649 | 2024-02-16T07:00:31.602Z | @@ -100,9 +100,9 @@ |
650 | 2024-02-16T07:00:31.602Z | # You can configure multiple uplinks by repeating the following stanza |
651 | 2024-02-16T07:00:31.602Z | [[rack_network_config.ports]] |
652 | 2024-02-16T07:00:31.602Z | # Routes associated with this port. |
653 | 2024-02-16T07:00:31.602Z | -routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}] |
654 | 2024-02-16T07:00:31.602Z | +routes = [{nexthop = "10.151.2.100", destination = "0.0.0.0/0"}] |
655 | 2024-02-16T07:00:31.602Z | # Addresses associated with this port. |
656 | 2024-02-16T07:00:31.602Z | -addresses = ["192.168.1.30/24"] |
657 | 2024-02-16T07:00:31.602Z | +addresses = ["10.151.2.174/24"] |
658 | 2024-02-16T07:00:31.602Z | # Name of the uplink port. This should always be "qsfp0" when using softnpu. |
659 | 2024-02-16T07:00:31.603Z | port = "qsfp0" |
660 | 2024-02-16T07:00:31.603Z | # The speed of this port. |
661 | 2024-02-16T07:00:31.603Z | + true |
662 | 2024-02-16T07:00:31.603Z | + tar rvf out/omicron-sled-agent.tar pkg/config-rss.toml |
663 | 2024-02-16T07:00:31.603Z | a pkg/config-rss.toml 7K |
664 | 2024-02-16T07:00:31.603Z | + rm -f pkg/config-rss.toml pkg/config-rss.toml~ |
665 | 2024-02-16T07:00:31.628Z | + TLS_NAME=recovery.sys.oxide.test |
666 | 2024-02-16T07:00:31.628Z | + openssl req -newkey rsa:4096 -x509 -sha256 -days 3 -nodes -out pkg/initial-tls-cert.pem -keyout pkg/initial-tls-key.pem -subj /CN=recovery.sys.oxide.test -addext subjectAltName=DNS:recovery.sys.oxide.test -addext basicConstraints=critical,CA:FALSE -config /dev/stdin |
667 | 2024-02-16T07:00:31.997Z | ...+.......+..+.+.........+...........+...+.+.........+.....+.+......+..+.+.....+...............+.......+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+...+...+......+.........+......+...+...+....+...+.....+.......+......+..+.+..+...+.........+.......+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.....+.+...+.....+....+..+...+.......+...........+....+...+...+..+...+...+.+...............+..............+.+...+..+........................+.......+...+...........+..........+...+........+...+.........+................+.....+..................................+............+...+.......................+.............+............+...+............+............+......+.....+....+..+.+..............+...+.+............+.....+.......+..+..................+.+..+...+.......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
668 | 2024-02-16T07:00:32.496Z | ...+.....+.......+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*......+...+...+....+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.......+............+......+.....................+.+........+.......+........+....+.....+...+..................+....+...+............+..+....+...........+................+..............+....+.....................+.....+.............+.....+.+.....+.+.....+................+.....+.......+...+........+....+...+.................+...............+.+......+...........+...+.......+..+............+....+..+.......+........+....+...+..+......+.........+...............................+..+......+.+.....+.+........+.+.....+.+..+.............+........+.......+.....+..........+..+.+..+...................+...+......+.........+...+........+...............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
669 | 2024-02-16T07:00:32.522Z | ----- |
670 | 2024-02-16T07:00:32.522Z | + tar rvf out/omicron-sled-agent.tar pkg/initial-tls-cert.pem pkg/initial-tls-key.pem |
671 | 2024-02-16T07:00:32.522Z | a pkg/initial-tls-cert.pem 2K |
672 | 2024-02-16T07:00:32.522Z | a pkg/initial-tls-key.pem 4K |
673 | 2024-02-16T07:00:32.522Z | + rm -f pkg/initial-tls-cert.pem pkg/initial-tls-key.pem |
674 | 2024-02-16T07:00:32.522Z | + rmdir pkg |
675 | 2024-02-16T07:00:32.523Z | + E2E_TLS_CERT=/opt/oxide/sled-agent/pkg/initial-tls-cert.pem |
676 | 2024-02-16T07:00:32.523Z | + pfexec mkdir -p /usr/oxide |
677 | 2024-02-16T07:00:32.523Z | + pfexec curl -sSfL -o /usr/oxide/oxide http://catacomb.eng.oxide.computer:12346/oxide-v0.1.0 |
678 | 2024-02-16T07:00:33.228Z | + pfexec chmod +x /usr/oxide/oxide |
679 | 2024-02-16T07:00:33.255Z | + curl -sSfL -o debian-11-genericcloud-amd64.raw http://catacomb.eng.oxide.computer:12346/debian-11-genericcloud-amd64.raw |
680 | 2024-02-16T07:00:51.485Z | + OMICRON_NO_UNINSTALL=1 |
681 | 2024-02-16T07:00:51.485Z | + ptime -m pfexec ./target/release/omicron-package -t test install |
682 | 2024-02-16T07:00:51.510Z | Logging to: /opt/oxide/work/out/LOG |
683 | 2024-02-16T07:00:51.989Z | |
684 | 2024-02-16T07:00:52.016Z | real 0.499541619 |
685 | 2024-02-16T07:00:52.016Z | user 0.109518711 |
686 | 2024-02-16T07:00:52.016Z | sys 1.991043609 |
687 | 2024-02-16T07:00:52.016Z | trap 0.000000320 |
688 | 2024-02-16T07:00:52.016Z | tflt 0.000095869 |
689 | 2024-02-16T07:00:52.016Z | dflt 0.000386070 |
690 | 2024-02-16T07:00:52.017Z | kflt 0.000000000 |
691 | 2024-02-16T07:00:52.017Z | lock 45.311181989 |
692 | 2024-02-16T07:00:52.017Z | slp 0.721710067 |
693 | 2024-02-16T07:00:52.017Z | lat 0.002950140 |
694 | 2024-02-16T07:00:52.017Z | stop 0.211367642 |
695 | 2024-02-16T07:00:52.017Z | + retry=0 |
696 | 2024-02-16T07:00:52.017Z | + curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/' |
697 | 2024-02-16T07:00:52.017Z | + [[ 0 -gt 30 ]] |
698 | 2024-02-16T07:00:52.017Z | + sleep 1 |
699 | 2024-02-16T07:00:53.004Z | + retry=1 |
700 | 2024-02-16T07:00:53.005Z | + curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/' |
701 | 2024-02-16T07:00:53.031Z | + [[ 1 -gt 30 ]] |
702 | 2024-02-16T07:00:53.031Z | + sleep 1 |
703 | 2024-02-16T07:00:54.021Z | + retry=2 |
704 | 2024-02-16T07:00:54.021Z | + curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/' |
705 | 2024-02-16T07:00:54.047Z | + [[ 2 -gt 30 ]] |
706 | 2024-02-16T07:00:54.047Z | + sleep 1 |
707 | 2024-02-16T07:00:55.033Z | + retry=3 |
708 | 2024-02-16T07:00:55.033Z | + curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/' |
709 | 2024-02-16T07:00:55.061Z | + [[ 3 -gt 30 ]] |
710 | 2024-02-16T07:00:55.061Z | + sleep 1 |
711 | 2024-02-16T07:00:56.049Z | + retry=4 |
712 | 2024-02-16T07:00:56.049Z | + curl --head --silent -o /dev/null 'http://[fd00:1122:3344:101::2]:12224/' |
713 | 2024-02-16T07:01:21.226Z | + pfexec zlogin sidecar_softnpu /softnpu/scadm --server /softnpu/server --client /softnpu/client standalone dump-state |
714 | 2024-02-16T07:01:21.258Z | local v6: |
715 | 2024-02-16T07:01:21.258Z | fd0e:61b4:e2c8:25fe::1 |
716 | 2024-02-16T07:01:21.258Z | fe80::ac41:9ff:fe14:d8cb |
717 | 2024-02-16T07:01:21.258Z | local v4: |
718 | 2024-02-16T07:01:21.258Z | 10.151.2.174 |
719 | 2024-02-16T07:01:21.258Z | router v6: |
720 | 2024-02-16T07:01:21.258Z | fd00:1122:3344:2::/64 -> fe80::8c38:94ff:fefa:6031 (1) |
721 | 2024-02-16T07:01:21.259Z | fd00:1122:3344:1::/64 -> fe80::8c38:94ff:fefa:6031 (1) |
722 | 2024-02-16T07:01:21.259Z | fd00:1122:3344:3::/64 -> fe80::8c38:94ff:fefa:6031 (1) |
723 | 2024-02-16T07:01:21.259Z | fd00:1122:3344:101::/64 -> fe80::8c38:94ff:fefa:6031 (1) |
724 | 2024-02-16T07:01:21.259Z | fdb0:18c0:4d0d:a02a::/64 -> fe80::8c38:94ff:fefa:6031 (1) |
725 | 2024-02-16T07:01:21.259Z | router v4_idx: |
726 | 2024-02-16T07:01:21.259Z | 0.0.0.0/0 -> 0 |
727 | 2024-02-16T07:01:21.259Z | router v4_routes: |
728 | 2024-02-16T07:01:21.284Z | 0 -> 10.151.2.100 (2) |
729 | 2024-02-16T07:01:21.284Z | resolver v4: |
730 | 2024-02-16T07:01:21.284Z | 10.151.2.100 -> 18:c0:4d:0d:a0:2a |
731 | 2024-02-16T07:01:21.284Z | 10.151.2.174 -> 46:91:56:8f:29:ac |
732 | 2024-02-16T07:01:21.284Z | resolver v6: |
733 | 2024-02-16T07:01:21.284Z | fe80::8c38:94ff:fefa:6031 -> 8e:38:94:fa:60:31 |
734 | 2024-02-16T07:01:21.284Z | fe80::ac41:9ff:fe14:d8cb -> ae:41:09:14:d8:cb |
735 | 2024-02-16T07:01:21.284Z | nat v4: |
736 | 2024-02-16T07:01:21.284Z | 10.151.2.169 0/16383 -> fd00:1122:3344:101::1 100/a8:40:25:ff:db:f5 |
737 | 2024-02-16T07:01:21.284Z | nat_v6: |
738 | 2024-02-16T07:01:21.284Z | port_mac: |
739 | 2024-02-16T07:01:21.284Z | 1: a8:40:25:53:ba:6f |
740 | 2024-02-16T07:01:21.284Z | 2: a8:40:25:53:ba:8f |
741 | 2024-02-16T07:01:21.284Z | 1000: a8:40:25:53:ba:6e |
742 | 2024-02-16T07:01:21.284Z | proxy arp: |
743 | 2024-02-16T07:01:21.284Z | 10.151.2.164/10.151.2.227: a8:e1:de:01:70:1d |
744 | 2024-02-16T07:01:21.284Z | + retry=0 |
745 | 2024-02-16T07:01:21.284Z | +++ zoneadm list -n |
746 | 2024-02-16T07:01:21.284Z | +++ grep oxz_ntp |
747 | 2024-02-16T07:01:21.284Z | ++ pfexec svcs -z oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092 -Hostate oxide/ntp |
748 | 2024-02-16T07:01:21.310Z | svcs: Pattern 'oxide/ntp' doesn't match any instances |
749 | 2024-02-16T07:01:21.310Z | ++ true |
750 | 2024-02-16T07:01:21.310Z | + [[ '' != online ]] |
751 | 2024-02-16T07:01:21.310Z | + [[ 0 -gt 60 ]] |
752 | 2024-02-16T07:01:21.310Z | + sleep 1 |
753 | 2024-02-16T07:01:22.291Z | + retry=1 |
754 | 2024-02-16T07:01:22.316Z | +++ zoneadm list -n |
755 | 2024-02-16T07:01:22.316Z | +++ grep oxz_ntp |
756 | 2024-02-16T07:01:22.316Z | ++ pfexec svcs -z oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092 -Hostate oxide/ntp |
757 | 2024-02-16T07:01:22.342Z | svcs: Pattern 'oxide/ntp' doesn't match any instances |
758 | 2024-02-16T07:01:22.342Z | ++ true |
759 | 2024-02-16T07:01:22.342Z | + [[ '' != online ]] |
760 | 2024-02-16T07:01:22.342Z | + [[ 1 -gt 60 ]] |
761 | 2024-02-16T07:01:22.343Z | + sleep 1 |
762 | 2024-02-16T07:01:23.320Z | + retry=2 |
763 | 2024-02-16T07:01:23.346Z | +++ zoneadm list -n |
764 | 2024-02-16T07:01:23.346Z | +++ grep oxz_ntp |
765 | 2024-02-16T07:01:23.346Z | ++ pfexec svcs -z oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092 -Hostate oxide/ntp |
766 | 2024-02-16T07:01:23.346Z | + [[ disabled != online ]] |
767 | 2024-02-16T07:01:23.346Z | + [[ 2 -gt 60 ]] |
768 | 2024-02-16T07:01:23.346Z | + sleep 1 |
769 | 2024-02-16T07:01:24.344Z | + retry=3 |
770 | 2024-02-16T07:01:24.370Z | +++ zoneadm list -n |
771 | 2024-02-16T07:01:24.370Z | +++ grep oxz_ntp |
772 | 2024-02-16T07:01:24.371Z | ++ pfexec svcs -z oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092 -Hostate oxide/ntp |
773 | 2024-02-16T07:01:24.371Z | + [[ disabled != online ]] |
774 | 2024-02-16T07:01:24.371Z | + [[ 3 -gt 60 ]] |
775 | 2024-02-16T07:01:24.371Z | + sleep 1 |
776 | 2024-02-16T07:01:25.367Z | + retry=4 |
777 | 2024-02-16T07:01:25.393Z | +++ zoneadm list -n |
778 | 2024-02-16T07:01:25.393Z | +++ grep oxz_ntp |
779 | 2024-02-16T07:01:25.393Z | ++ pfexec svcs -z oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092 -Hostate oxide/ntp |
780 | 2024-02-16T07:01:25.393Z | + [[ disabled != online ]] |
781 | 2024-02-16T07:01:25.393Z | + [[ 4 -gt 60 ]] |
782 | 2024-02-16T07:01:25.393Z | + sleep 1 |
783 | 2024-02-16T07:01:26.392Z | + retry=5 |
784 | 2024-02-16T07:01:26.422Z | +++ zoneadm list -n |
785 | 2024-02-16T07:01:26.422Z | +++ grep oxz_ntp |
786 | 2024-02-16T07:01:26.423Z | ++ pfexec svcs -z oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092 -Hostate oxide/ntp |
787 | 2024-02-16T07:01:26.423Z | + [[ disabled != online ]] |
788 | 2024-02-16T07:01:26.423Z | + [[ 5 -gt 60 ]] |
789 | 2024-02-16T07:01:26.423Z | + sleep 1 |
790 | 2024-02-16T07:01:27.413Z | + retry=6 |
791 | 2024-02-16T07:01:27.440Z | +++ zoneadm list -n |
792 | 2024-02-16T07:01:27.440Z | +++ grep oxz_ntp |
793 | 2024-02-16T07:01:27.440Z | ++ pfexec svcs -z oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092 -Hostate oxide/ntp |
794 | 2024-02-16T07:01:27.440Z | + [[ disabled != online ]] |
795 | 2024-02-16T07:01:27.440Z | + [[ 6 -gt 60 ]] |
796 | 2024-02-16T07:01:27.441Z | + sleep 1 |
797 | 2024-02-16T07:01:28.435Z | + retry=7 |
798 | 2024-02-16T07:01:28.461Z | +++ zoneadm list -n |
799 | 2024-02-16T07:01:28.461Z | +++ grep oxz_ntp |
800 | 2024-02-16T07:01:28.461Z | ++ pfexec svcs -z oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092 -Hostate oxide/ntp |
801 | 2024-02-16T07:01:28.461Z | + [[ disabled != online ]] |
802 | 2024-02-16T07:01:28.461Z | + [[ 7 -gt 60 ]] |
803 | 2024-02-16T07:01:28.461Z | + sleep 1 |
804 | 2024-02-16T07:01:29.457Z | + retry=8 |
805 | 2024-02-16T07:01:29.483Z | +++ zoneadm list -n |
806 | 2024-02-16T07:01:29.483Z | +++ grep oxz_ntp |
807 | 2024-02-16T07:01:29.483Z | ++ pfexec svcs -z oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092 -Hostate oxide/ntp |
808 | 2024-02-16T07:01:29.483Z | + [[ disabled != online ]] |
809 | 2024-02-16T07:01:29.483Z | + [[ 8 -gt 60 ]] |
810 | 2024-02-16T07:01:29.484Z | + sleep 1 |
811 | 2024-02-16T07:01:30.479Z | + retry=9 |
812 | 2024-02-16T07:01:30.505Z | +++ zoneadm list -n |
813 | 2024-02-16T07:01:30.505Z | +++ grep oxz_ntp |
814 | 2024-02-16T07:01:30.505Z | ++ pfexec svcs -z oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092 -Hostate oxide/ntp |
815 | 2024-02-16T07:01:30.505Z | + [[ disabled != online ]] |
816 | 2024-02-16T07:01:30.505Z | + [[ 9 -gt 60 ]] |
817 | 2024-02-16T07:01:30.505Z | + sleep 1 |
818 | 2024-02-16T07:01:31.501Z | + retry=10 |
819 | 2024-02-16T07:01:31.933Z | +++ zoneadm list -n |
820 | 2024-02-16T07:01:31.933Z | +++ grep oxz_ntp |
821 | 2024-02-16T07:01:31.933Z | ++ pfexec svcs -z oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092 -Hostate oxide/ntp |
822 | 2024-02-16T07:01:31.933Z | + [[ offline* != online ]] |
823 | 2024-02-16T07:01:31.933Z | + [[ 10 -gt 60 ]] |
824 | 2024-02-16T07:01:31.934Z | + sleep 1 |
825 | 2024-02-16T07:01:32.525Z | + retry=11 |
826 | 2024-02-16T07:01:32.551Z | +++ zoneadm list -n |
827 | 2024-02-16T07:01:32.552Z | +++ grep oxz_ntp |
828 | 2024-02-16T07:01:32.552Z | ++ pfexec svcs -z oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092 -Hostate oxide/ntp |
829 | 2024-02-16T07:01:32.552Z | + [[ online != online ]] |
830 | 2024-02-16T07:01:32.552Z | + echo 'Waited for chrony: 11s' |
831 | 2024-02-16T07:01:32.552Z | + export RUST_BACKTRACE=1 |
832 | 2024-02-16T07:01:32.552Z | Waited for chrony: 11s |
833 | 2024-02-16T07:01:32.552Z | + RUST_BACKTRACE=1 |
834 | 2024-02-16T07:01:32.552Z | + export E2E_TLS_CERT IPPOOL_START IPPOOL_END |
835 | 2024-02-16T07:01:32.552Z | ++ ./tests/bootstrap |
836 | 2024-02-16T07:01:32.580Z | Using external DNS server at 10.151.2.164:53 |
837 | 2024-02-16T07:01:32.580Z | 2024-02-16 07:01:32.604076830 UTC: attempting to log into API |
838 | 2024-02-16T07:01:47.611Z | 2024-02-16 07:01:47.610841064 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out |
839 | 2024-02-16T07:01:48.611Z | 2024-02-16 07:01:48.609208564 UTC: attempting to log into API |
840 | 2024-02-16T07:02:03.642Z | 2024-02-16 07:02:03.628262920 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out |
841 | 2024-02-16T07:02:04.646Z | 2024-02-16 07:02:04.629418162 UTC: attempting to log into API |
|
842 | 2024-02-16T07:02:13.848Z | Error: No response to keepalive - Terminating session |
|
843 | 2024-02-16T07:02:19.693Z | 2024-02-16 07:02:19.664339870 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out |
844 | 2024-02-16T07:02:20.681Z | 2024-02-16 07:02:20.666690892 UTC: attempting to log into API |
|
845 | 2024-02-16T07:02:23.717Z | Error: No response de-activating SOL payload |
846 | 2024-02-16T07:02:25.080Z | [SOL Session operational. Use ~? for help] |
|
847 | 2024-02-16T07:02:36.426Z | 2024-02-16 07:02:36.023810558 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out |
848 | 2024-02-16T07:02:37.284Z | 2024-02-16 07:02:37.270020794 UTC: attempting to log into API |
849 | 2024-02-16T07:02:52.575Z | 2024-02-16 07:02:52.535356544 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out |
850 | 2024-02-16T07:02:53.552Z | 2024-02-16 07:02:53.536664290 UTC: attempting to log into API |
|
851 | 2024-02-16T07:03:07.274Z | Feb 16 07:03:06 sock ip: ip: joining multicasts failed (4) on vopte2 - will use link layer broadcasts for multicast |
|
852 | 2024-02-16T07:03:08.583Z | 2024-02-16 07:03:08.568418511 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out |
853 | 2024-02-16T07:03:09.584Z | 2024-02-16 07:03:09.569134742 UTC: attempting to log into API |
|
854 | 2024-02-16T07:03:11.321Z | SUNW-MSG-ID: ZFS-8000-FD, TYPE: Fault, VER: 1, SEVERITY: Major |
855 | 2024-02-16T07:03:11.374Z | EVENT-TIME: Fri Feb 16 07:03:10 UTC 2024 |
856 | 2024-02-16T07:03:11.415Z | PLATFORM: R152-Z32-00, CSN: GKG8N7212A0003, HOSTNAME: sock |
857 | 2024-02-16T07:03:11.452Z | SOURCE: zfs-diagnosis, REV: 1.0 |
858 | 2024-02-16T07:03:11.485Z | EVENT-ID: ba329890-533f-42ea-a445-52cb673b52c5 |
859 | 2024-02-16T07:03:11.518Z | DESC: The number of I/O errors associated with a ZFS device exceeded |
860 | 2024-02-16T07:03:11.552Z | acceptable levels. Refer to http://illumos.org/msg/ZFS-8000-FD for more information. |
861 | 2024-02-16T07:03:11.585Z | AUTO-RESPONSE: The device has been offlined and marked as faulted. An attempt |
862 | 2024-02-16T07:03:11.619Z | will be made to activate a hot spare if available. |
863 | 2024-02-16T07:03:11.652Z | IMPACT: Fault tolerance of the pool may be compromised. |
864 | 2024-02-16T07:03:11.686Z | REC-ACTION: Run 'zpool status -x' and replace the bad device. |
865 | 2024-02-16T07:03:14.897Z | ^@Feb 16 07:03:14 sock ip: ip: joining multicasts failed (4) on vopte3 - will use link layer broadcasts for multicast |
866 | 2024-02-16T07:03:21.032Z | Feb 16 07:03:20 sock ip: ip: joining multicasts failed (4) on vopte4 - will use link layer broadcasts for multicast |
|
867 | 2024-02-16T07:03:24.615Z | 2024-02-16 07:03:24.600366630 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out |
868 | 2024-02-16T07:03:25.617Z | 2024-02-16 07:03:25.601459790 UTC: attempting to log into API |
|
869 | 2024-02-16T07:03:27.704Z | Feb 16 07:03:27 sock ip: ip: joining multicasts failed (4) on vopte5 - will use link layer broadcasts for multicast |
870 | 2024-02-16T07:03:34.380Z | Feb 16 07:03:33 sock ip: ip: joining multicasts failed (4) on vopte1 - will use link layer broadcasts for multicast |
|
871 | 2024-02-16T07:03:40.650Z | 2024-02-16 07:03:40.634835963 UTC: login failed: logging in: error sending request for url (https://recovery.sys.oxide.test/v1/login/recovery/local): error trying to connect: operation timed out |
872 | 2024-02-16T07:03:41.651Z | 2024-02-16 07:03:41.635941534 UTC: attempting to log into API |
873 | 2024-02-16T07:03:52.445Z | 2024-02-16 07:03:52.428863209 UTC: login succeeded |
874 | 2024-02-16T07:03:52.480Z | waiting for nexus to come up... |
875 | 2024-02-16T07:03:52.546Z | creating IP pool... 10.151.2.175 - 10.151.2.227 |
876 | 2024-02-16T07:03:52.710Z | setting up quotas... |
877 | 2024-02-16T07:03:52.753Z | ensuring datasets are ready... |
878 | 2024-02-16T07:03:54.645Z | let's roll. |
879 | 2024-02-16T07:03:54.670Z | + eval 'OXIDE_HOST=https://recovery.sys.oxide.test |
880 | 2024-02-16T07:03:54.671Z | OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.2.166 |
881 | 2024-02-16T07:03:54.671Z | OXIDE_TOKEN=oxide-token-70310831eecb0dad3a8b7be5c2af557584891292' |
882 | 2024-02-16T07:03:54.671Z | ++ OXIDE_HOST=https://recovery.sys.oxide.test |
883 | 2024-02-16T07:03:54.671Z | ++ OXIDE_RESOLVE=recovery.sys.oxide.test:443:10.151.2.166 |
884 | 2024-02-16T07:03:54.671Z | ++ OXIDE_TOKEN=oxide-token-70310831eecb0dad3a8b7be5c2af557584891292 |
885 | 2024-02-16T07:03:54.671Z | + export OXIDE_HOST OXIDE_TOKEN |
886 | 2024-02-16T07:03:54.671Z | + retry=0 |
887 | 2024-02-16T07:03:54.672Z | + curl -sSf https://recovery.sys.oxide.test/v1/ping --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem |
888 | 2024-02-16T07:03:54.755Z | + /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem project create --name images --description 'some images' |
889 | 2024-02-16T07:03:54.991Z | {"status":"ok"}success |
890 | 2024-02-16T07:03:54.992Z | Project { |
891 | 2024-02-16T07:03:54.992Z | description: "some images", |
892 | 2024-02-16T07:03:54.992Z | id: 19dbbbf0-f86a-4831-b865-dbe837563cc0, |
893 | 2024-02-16T07:03:54.992Z | name: Name( |
894 | 2024-02-16T07:03:54.992Z | "images", |
895 | 2024-02-16T07:03:54.993Z | ), |
896 | 2024-02-16T07:03:54.993Z | time_created: 2024-02-16T07:03:54.805314Z, |
897 | 2024-02-16T07:03:54.993Z | time_modified: 2024-02-16T07:03:54.805314Z, |
898 | 2024-02-16T07:03:54.993Z | } |
899 | 2024-02-16T07:03:55.018Z | + /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem disk import --path debian-11-genericcloud-amd64.raw --disk debian11-boot --project images --description 'debian 11 cloud image from distros' --snapshot debian11-snapshot --image debian11 --image-description 'debian 11 original base image' --image-os debian --image-version 11 |
|
900 | 2024-02-16T07:05:13.507Z | Error: No response to keepalive - Terminating session |
901 | 2024-02-16T07:05:23.665Z | Error: No response de-activating SOL payload |
902 | 2024-02-16T07:05:24.765Z | [SOL Session operational. Use ~? for help] |
903 | 2024-02-16T07:08:29.424Z | Error: No response to keepalive - Terminating session |
904 | 2024-02-16T07:08:39.619Z | Error: No response de-activating SOL payload |
905 | 2024-02-16T07:08:40.734Z | [SOL Session operational. Use ~? for help] |
|
906 | 2024-02-16T07:09:38.222Z | done! |
907 | 2024-02-16T07:09:38.334Z | + /usr/oxide/oxide --resolve recovery.sys.oxide.test:443:10.151.2.166 --cacert /opt/oxide/sled-agent/pkg/initial-tls-cert.pem image promote --project images --image debian11 |
908 | 2024-02-16T07:09:38.424Z | success |
909 | 2024-02-16T07:09:38.425Z | Image { |
910 | 2024-02-16T07:09:38.425Z | block_size: ByteCount( |
911 | 2024-02-16T07:09:38.425Z | 512, |
912 | 2024-02-16T07:09:38.425Z | ), |
913 | 2024-02-16T07:09:38.425Z | description: "debian 11 original base image", |
914 | 2024-02-16T07:09:38.426Z | digest: None, |
915 | 2024-02-16T07:09:38.426Z | id: 77c4f281-0452-44a8-95fc-989865a46dd9, |
916 | 2024-02-16T07:09:38.426Z | name: Name( |
917 | 2024-02-16T07:09:38.456Z | "debian11", |
918 | 2024-02-16T07:09:38.456Z | ), |
919 | 2024-02-16T07:09:38.456Z | os: "debian", |
920 | 2024-02-16T07:09:38.456Z | project_id: None, |
921 | 2024-02-16T07:09:38.457Z | size: ByteCount( |
922 | 2024-02-16T07:09:38.457Z | 2147483648, |
923 | 2024-02-16T07:09:38.457Z | ), |
924 | 2024-02-16T07:09:38.457Z | time_created: 2024-02-16T07:09:38.175333Z, |
925 | 2024-02-16T07:09:38.457Z | time_modified: 2024-02-16T07:09:38.396547Z, |
926 | 2024-02-16T07:09:38.457Z | url: None, |
927 | 2024-02-16T07:09:38.457Z | version: "11", |
928 | 2024-02-16T07:09:38.457Z | } |
929 | 2024-02-16T07:09:38.457Z | + rm ./tests/bootstrap |
930 | 2024-02-16T07:09:38.499Z | + for test_bin in tests/* |
931 | 2024-02-16T07:09:38.500Z | + ./tests/bootstrap-22e122cc756fa22c |
932 | 2024-02-16T07:09:38.549Z | |
933 | 2024-02-16T07:09:38.549Z | running 0 tests |
934 | 2024-02-16T07:09:38.549Z | |
935 | 2024-02-16T07:09:38.583Z | test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s |
936 | 2024-02-16T07:09:38.583Z | |
937 | 2024-02-16T07:09:38.584Z | + for test_bin in tests/* |
938 | 2024-02-16T07:09:38.584Z | + ./tests/end_to_end_tests-4eb1211856b7c67f |
939 | 2024-02-16T07:09:38.753Z | |
940 | 2024-02-16T07:09:38.754Z | running 2 tests |
941 | 2024-02-16T07:09:39.011Z | test no_spoof::no_spoof_here ... ok |
942 | 2024-02-16T07:10:22.998Z | test instance_launch::instance_launch ... ok |
943 | 2024-02-16T07:10:22.998Z | |
944 | 2024-02-16T07:10:22.998Z | test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 44.24s |
945 | 2024-02-16T07:10:22.998Z | |
946 | 2024-02-16T07:10:23.023Z | + _exit_trap |
947 | 2024-02-16T07:10:23.063Z | + local status=0 |
948 | 2024-02-16T07:10:23.098Z | + [[ 0 -eq 0 ]] |
949 | 2024-02-16T07:10:23.099Z | + exit 0 |
950 | 2024-02-16T07:10:23.099Z | process exited: duration 648553 ms, exit code 0 |
|
951 | 2024-02-16T07:10:23.473Z | found 72 output files |
952 | 2024-02-16T07:10:23.524Z | uploading: /var/svc/log/oxide-sled-agent:default.log (906327 bytes) |
953 | 2024-02-16T07:10:23.841Z | upload warning: file "/var/svc/log/oxide-sled-agent:default.log" changed size mid upload: 906327 -> 908550 |
954 | 2024-02-16T07:10:24.921Z | uploaded: /var/svc/log/oxide-sled-agent:default.log |
955 | 2024-02-16T07:10:24.945Z | uploading: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log (2360808 bytes) |
956 | 2024-02-16T07:10:25.846Z | upload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log" changed size mid upload: 2360808 -> 2371267 |
957 | 2024-02-16T07:10:26.947Z | uploaded: /zone/oxz_switch/root/var/svc/log/oxide-dendrite:default.log |
958 | 2024-02-16T07:10:26.947Z | uploading: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log (8459 bytes) |
959 | 2024-02-16T07:10:28.057Z | uploaded: /zone/oxz_switch/root/var/svc/log/oxide-mg-ddm:default.log |
960 | 2024-02-16T07:10:28.058Z | uploading: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log (24447 bytes) |
961 | 2024-02-16T07:10:29.304Z | uploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgd:default.log |
962 | 2024-02-16T07:10:29.337Z | uploading: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log (1520798 bytes) |
963 | 2024-02-16T07:10:30.047Z | upload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log" changed size mid upload: 1520798 -> 1548129 |
964 | 2024-02-16T07:10:31.130Z | uploaded: /zone/oxz_switch/root/var/svc/log/oxide-mgs:default.log |
965 | 2024-02-16T07:10:31.130Z | uploading: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log (876135 bytes) |
966 | 2024-02-16T07:10:31.905Z | upload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log" changed size mid upload: 876135 -> 889999 |
967 | 2024-02-16T07:10:33.365Z | uploaded: /zone/oxz_switch/root/var/svc/log/oxide-sp-sim:default.log |
968 | 2024-02-16T07:10:33.366Z | uploading: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log (1388 bytes) |
969 | 2024-02-16T07:10:34.460Z | uploaded: /zone/oxz_switch/root/var/svc/log/oxide-switch_zone_setup:default.log |
970 | 2024-02-16T07:10:34.460Z | uploading: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log (267856 bytes) |
971 | 2024-02-16T07:10:34.662Z | upload warning: file "/zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log" changed size mid upload: 267856 -> 273900 |
972 | 2024-02-16T07:10:35.745Z | uploaded: /zone/oxz_switch/root/var/svc/log/oxide-tfport:default.log |
973 | 2024-02-16T07:10:35.782Z | uploading: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log (617 bytes) |
974 | 2024-02-16T07:10:37.258Z | uploaded: /zone/oxz_switch/root/var/svc/log/oxide-uplink:default.log |
975 | 2024-02-16T07:10:37.259Z | uploading: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log (5392 bytes) |
976 | 2024-02-16T07:10:39.226Z | uploaded: /zone/oxz_switch/root/var/svc/log/oxide-wicketd:default.log |
977 | 2024-02-16T07:10:39.226Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_64ded72d-bc8a-4810-b289-b5210197921e/root/var/svc/log/oxide-clickhouse:default.log (1191279 bytes) |
978 | 2024-02-16T07:10:39.665Z | upload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_64ded72d-bc8a-4810-b289-b5210197921e/root/var/svc/log/oxide-clickhouse:default.log" changed size mid upload: 1191279 -> 1242649 |
979 | 2024-02-16T07:10:40.605Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_64ded72d-bc8a-4810-b289-b5210197921e/root/var/svc/log/oxide-clickhouse:default.log |
980 | 2024-02-16T07:10:40.606Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_64ded72d-bc8a-4810-b289-b5210197921e/root/var/svc/log/oxide-zone-network-setup:default.log (1708 bytes) |
981 | 2024-02-16T07:10:41.711Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_clickhouse_64ded72d-bc8a-4810-b289-b5210197921e/root/var/svc/log/oxide-zone-network-setup:default.log |
982 | 2024-02-16T07:10:41.740Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_10503827-57aa-4e38-8591-34b9522f64c1/root/var/svc/log/oxide-cockroachdb:default.log (4456 bytes) |
983 | 2024-02-16T07:10:42.818Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_10503827-57aa-4e38-8591-34b9522f64c1/root/var/svc/log/oxide-cockroachdb:default.log |
984 | 2024-02-16T07:10:42.818Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_10503827-57aa-4e38-8591-34b9522f64c1/root/var/svc/log/oxide-zone-network-setup:default.log (1705 bytes) |
985 | 2024-02-16T07:10:43.955Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_10503827-57aa-4e38-8591-34b9522f64c1/root/var/svc/log/oxide-zone-network-setup:default.log |
986 | 2024-02-16T07:10:43.955Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_08d5efd9-1a86-4b11-a0d0-0d93b60d80dc/root/var/svc/log/oxide-crucible-agent:default.log (27836 bytes) |
987 | 2024-02-16T07:10:45.116Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_08d5efd9-1a86-4b11-a0d0-0d93b60d80dc/root/var/svc/log/oxide-crucible-agent:default.log |
988 | 2024-02-16T07:10:45.117Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_08d5efd9-1a86-4b11-a0d0-0d93b60d80dc/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes) |
989 | 2024-02-16T07:10:46.241Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_08d5efd9-1a86-4b11-a0d0-0d93b60d80dc/root/var/svc/log/oxide-crucible-downstairs:default.log |
990 | 2024-02-16T07:10:46.242Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_08d5efd9-1a86-4b11-a0d0-0d93b60d80dc/root/var/svc/log/oxide-crucible-downstairs:downstairs-30ef8df5-4489-421e-9a5e-98893b355ad4.log (5611 bytes) |
991 | 2024-02-16T07:10:47.335Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_08d5efd9-1a86-4b11-a0d0-0d93b60d80dc/root/var/svc/log/oxide-crucible-downstairs:downstairs-30ef8df5-4489-421e-9a5e-98893b355ad4.log |
992 | 2024-02-16T07:10:47.335Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_08d5efd9-1a86-4b11-a0d0-0d93b60d80dc/root/var/svc/log/oxide-crucible-downstairs:downstairs-c709deda-1414-4f58-a3d6-38ba03679d53.log (5692 bytes) |
993 | 2024-02-16T07:10:48.444Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_08d5efd9-1a86-4b11-a0d0-0d93b60d80dc/root/var/svc/log/oxide-crucible-downstairs:downstairs-c709deda-1414-4f58-a3d6-38ba03679d53.log |
994 | 2024-02-16T07:10:48.476Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_08d5efd9-1a86-4b11-a0d0-0d93b60d80dc/root/var/svc/log/oxide-crucible-downstairs:snapshot-c709deda-1414-4f58-a3d6-38ba03679d53-c5ff36fd-fda4-4d5b-8854-5b065543fc40.log (5305 bytes) |
995 | 2024-02-16T07:10:49.540Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_08d5efd9-1a86-4b11-a0d0-0d93b60d80dc/root/var/svc/log/oxide-crucible-downstairs:snapshot-c709deda-1414-4f58-a3d6-38ba03679d53-c5ff36fd-fda4-4d5b-8854-5b065543fc40.log |
996 | 2024-02-16T07:10:49.540Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_2fe79fc8-476d-4676-9095-5a487c5270f1/root/var/svc/log/oxide-external_dns:default.log (26936 bytes) |
997 | 2024-02-16T07:10:49.656Z | upload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_2fe79fc8-476d-4676-9095-5a487c5270f1/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 26936 -> 30712 |
998 | 2024-02-16T07:10:50.730Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_2fe79fc8-476d-4676-9095-5a487c5270f1/root/var/svc/log/oxide-external_dns:default.log |
999 | 2024-02-16T07:10:50.730Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_b30c21bb-8bc3-4926-89ff-16fa9d865c11/root/var/svc/log/oxide-internal_dns:default.log (108253 bytes) |
1000 | 2024-02-16T07:10:50.892Z | upload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_b30c21bb-8bc3-4926-89ff-16fa9d865c11/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 108253 -> 112009 |
1001 | 2024-02-16T07:10:51.966Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_b30c21bb-8bc3-4926-89ff-16fa9d865c11/root/var/svc/log/oxide-internal_dns:default.log |
1002 | 2024-02-16T07:10:51.966Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_8bd8801a-18ce-48ba-9ef2-3cec813ea995/root/var/svc/log/oxide-nexus:default.log (1230103 bytes) |
1003 | 2024-02-16T07:10:52.788Z | upload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_8bd8801a-18ce-48ba-9ef2-3cec813ea995/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 1230103 -> 1315757 |
1004 | 2024-02-16T07:10:53.860Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_8bd8801a-18ce-48ba-9ef2-3cec813ea995/root/var/svc/log/oxide-nexus:default.log |
1005 | 2024-02-16T07:10:53.860Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_88637054-2fdd-4a3c-b3a5-058e6e0fc436/root/var/svc/log/oxide-oximeter:default.log (322988 bytes) |
1006 | 2024-02-16T07:10:54.100Z | upload warning: file "/pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_88637054-2fdd-4a3c-b3a5-058e6e0fc436/root/var/svc/log/oxide-oximeter:default.log" changed size mid upload: 322988 -> 347899 |
1007 | 2024-02-16T07:10:55.194Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_88637054-2fdd-4a3c-b3a5-058e6e0fc436/root/var/svc/log/oxide-oximeter:default.log |
1008 | 2024-02-16T07:10:55.194Z | uploading: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_88637054-2fdd-4a3c-b3a5-058e6e0fc436/root/var/svc/log/oxide-zone-network-setup:default.log (1696 bytes) |
1009 | 2024-02-16T07:10:56.291Z | uploaded: /pool/ext/14b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_oximeter_88637054-2fdd-4a3c-b3a5-058e6e0fc436/root/var/svc/log/oxide-zone-network-setup:default.log |
1010 | 2024-02-16T07:10:56.291Z | uploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_21d4fd23-096c-49f7-82bd-9d4d51c58401/root/var/svc/log/oxide-cockroachdb:default.log (4407 bytes) |
1011 | 2024-02-16T07:10:57.393Z | uploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_21d4fd23-096c-49f7-82bd-9d4d51c58401/root/var/svc/log/oxide-cockroachdb:default.log |
1012 | 2024-02-16T07:10:57.393Z | uploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_21d4fd23-096c-49f7-82bd-9d4d51c58401/root/var/svc/log/oxide-zone-network-setup:default.log (1704 bytes) |
1013 | 2024-02-16T07:10:58.492Z | uploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_cockroachdb_21d4fd23-096c-49f7-82bd-9d4d51c58401/root/var/svc/log/oxide-zone-network-setup:default.log |
1014 | 2024-02-16T07:10:58.493Z | uploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_ed4ad331-7705-4b9b-9dde-52e410173374/root/var/svc/log/oxide-crucible-agent:default.log (3436 bytes) |
1015 | 2024-02-16T07:10:59.591Z | uploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_ed4ad331-7705-4b9b-9dde-52e410173374/root/var/svc/log/oxide-crucible-agent:default.log |
1016 | 2024-02-16T07:10:59.591Z | uploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_ed4ad331-7705-4b9b-9dde-52e410173374/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes) |
1017 | 2024-02-16T07:11:00.710Z | uploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_ed4ad331-7705-4b9b-9dde-52e410173374/root/var/svc/log/oxide-crucible-downstairs:default.log |
1018 | 2024-02-16T07:11:00.710Z | uploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_b5c12663-b663-4eae-957e-f7d9a47ec7d4/root/var/svc/log/oxide-external_dns:default.log (26936 bytes) |
1019 | 2024-02-16T07:11:00.848Z | upload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_b5c12663-b663-4eae-957e-f7d9a47ec7d4/root/var/svc/log/oxide-external_dns:default.log" changed size mid upload: 26936 -> 30713 |
1020 | 2024-02-16T07:11:01.934Z | uploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_external_dns_b5c12663-b663-4eae-957e-f7d9a47ec7d4/root/var/svc/log/oxide-external_dns:default.log |
1021 | 2024-02-16T07:11:01.934Z | uploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_62b2d7eb-013c-4138-be10-2e4de2464146/root/var/svc/log/oxide-internal_dns:default.log (113530 bytes) |
1022 | 2024-02-16T07:11:02.127Z | upload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_62b2d7eb-013c-4138-be10-2e4de2464146/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 113530 -> 117284 |
1023 | 2024-02-16T07:11:03.201Z | uploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_internal_dns_62b2d7eb-013c-4138-be10-2e4de2464146/root/var/svc/log/oxide-internal_dns:default.log |
1024 | 2024-02-16T07:11:03.201Z | uploading: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_9391ae64-1acb-4daa-8a34-f254cb193cb6/root/var/svc/log/oxide-nexus:default.log (1228527 bytes) |
1025 | 2024-02-16T07:11:03.611Z | upload warning: file "/pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_9391ae64-1acb-4daa-8a34-f254cb193cb6/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 1228527 -> 1318090 |
1026 | 2024-02-16T07:11:04.681Z | uploaded: /pool/ext/24b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_nexus_9391ae64-1acb-4daa-8a34-f254cb193cb6/root/var/svc/log/oxide-nexus:default.log |
1027 | 2024-02-16T07:11:04.705Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_2cc36482-64d9-49f9-8342-269af02f542a/root/var/svc/log/oxide-cockroachdb:default.log (4482 bytes) |
1028 | 2024-02-16T07:11:05.799Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_2cc36482-64d9-49f9-8342-269af02f542a/root/var/svc/log/oxide-cockroachdb:default.log |
1029 | 2024-02-16T07:11:05.799Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_2cc36482-64d9-49f9-8342-269af02f542a/root/var/svc/log/oxide-zone-network-setup:default.log (1706 bytes) |
1030 | 2024-02-16T07:11:06.911Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_cockroachdb_2cc36482-64d9-49f9-8342-269af02f542a/root/var/svc/log/oxide-zone-network-setup:default.log |
1031 | 2024-02-16T07:11:06.912Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ab16e63a-5348-4997-80d6-1afa7667a236/root/var/svc/log/oxide-crucible-agent:default.log (23475 bytes) |
1032 | 2024-02-16T07:11:08.048Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ab16e63a-5348-4997-80d6-1afa7667a236/root/var/svc/log/oxide-crucible-agent:default.log |
1033 | 2024-02-16T07:11:08.048Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ab16e63a-5348-4997-80d6-1afa7667a236/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes) |
1034 | 2024-02-16T07:11:09.140Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ab16e63a-5348-4997-80d6-1afa7667a236/root/var/svc/log/oxide-crucible-downstairs:default.log |
1035 | 2024-02-16T07:11:09.193Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ab16e63a-5348-4997-80d6-1afa7667a236/root/var/svc/log/oxide-crucible-downstairs:downstairs-9ae5bc01-4b77-4ad7-8518-ee45f44bd4d7.log (5694 bytes) |
1036 | 2024-02-16T07:11:10.257Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ab16e63a-5348-4997-80d6-1afa7667a236/root/var/svc/log/oxide-crucible-downstairs:downstairs-9ae5bc01-4b77-4ad7-8518-ee45f44bd4d7.log |
1037 | 2024-02-16T07:11:10.282Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ab16e63a-5348-4997-80d6-1afa7667a236/root/var/svc/log/oxide-crucible-downstairs:downstairs-ac5fc8c3-a73e-479a-b44a-b3c0116b30f2.log (3019 bytes) |
1038 | 2024-02-16T07:11:11.351Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ab16e63a-5348-4997-80d6-1afa7667a236/root/var/svc/log/oxide-crucible-downstairs:downstairs-ac5fc8c3-a73e-479a-b44a-b3c0116b30f2.log |
1039 | 2024-02-16T07:11:11.351Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ab16e63a-5348-4997-80d6-1afa7667a236/root/var/svc/log/oxide-crucible-downstairs:snapshot-9ae5bc01-4b77-4ad7-8518-ee45f44bd4d7-c5ff36fd-fda4-4d5b-8854-5b065543fc40.log (5305 bytes) |
1040 | 2024-02-16T07:11:12.503Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_ab16e63a-5348-4997-80d6-1afa7667a236/root/var/svc/log/oxide-crucible-downstairs:snapshot-9ae5bc01-4b77-4ad7-8518-ee45f44bd4d7-c5ff36fd-fda4-4d5b-8854-5b065543fc40.log |
1041 | 2024-02-16T07:11:12.503Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_pantry_faec7c14-a96a-4e55-a346-ab5b27f1dd3c/root/var/svc/log/oxide-crucible-pantry:default.log (2333 bytes) |
1042 | 2024-02-16T07:11:13.596Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_crucible_pantry_faec7c14-a96a-4e55-a346-ab5b27f1dd3c/root/var/svc/log/oxide-crucible-pantry:default.log |
1043 | 2024-02-16T07:11:13.596Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_81afef0c-bfcd-4ddf-a253-fbf74130bdc6/root/var/svc/log/oxide-internal_dns:default.log (110280 bytes) |
1044 | 2024-02-16T07:11:13.760Z | upload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_81afef0c-bfcd-4ddf-a253-fbf74130bdc6/root/var/svc/log/oxide-internal_dns:default.log" changed size mid upload: 110280 -> 114036 |
1045 | 2024-02-16T07:11:14.831Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_internal_dns_81afef0c-bfcd-4ddf-a253-fbf74130bdc6/root/var/svc/log/oxide-internal_dns:default.log |
1046 | 2024-02-16T07:11:14.831Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_34f0afd4-b6d4-468b-bac9-5a05bf460779/root/var/svc/log/oxide-nexus:default.log (39391182 bytes) |
1047 | 2024-02-16T07:11:23.794Z | upload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_34f0afd4-b6d4-468b-bac9-5a05bf460779/root/var/svc/log/oxide-nexus:default.log" changed size mid upload: 39391182 -> 39692626 |
1048 | 2024-02-16T07:11:24.865Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_nexus_34f0afd4-b6d4-468b-bac9-5a05bf460779/root/var/svc/log/oxide-nexus:default.log |
1049 | 2024-02-16T07:11:24.890Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092/root/var/svc/log/oxide-ntp:default.log (847 bytes) |
1050 | 2024-02-16T07:11:25.961Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092/root/var/svc/log/oxide-ntp:default.log |
1051 | 2024-02-16T07:11:25.961Z | uploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_d2dd05a0-7998-4b4e-b483-b23c52dfdb2c/root/var/svc/log/oxide-cockroachdb:default.log (4406 bytes) |
1052 | 2024-02-16T07:11:27.055Z | uploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_d2dd05a0-7998-4b4e-b483-b23c52dfdb2c/root/var/svc/log/oxide-cockroachdb:default.log |
1053 | 2024-02-16T07:11:27.055Z | uploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_d2dd05a0-7998-4b4e-b483-b23c52dfdb2c/root/var/svc/log/oxide-zone-network-setup:default.log (1705 bytes) |
1054 | 2024-02-16T07:11:28.146Z | uploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_cockroachdb_d2dd05a0-7998-4b4e-b483-b23c52dfdb2c/root/var/svc/log/oxide-zone-network-setup:default.log |
1055 | 2024-02-16T07:11:28.171Z | uploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_17fb421d-08d1-4991-aedd-9d8dda406f71/root/var/svc/log/oxide-crucible-agent:default.log (14583 bytes) |
1056 | 2024-02-16T07:11:29.265Z | uploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_17fb421d-08d1-4991-aedd-9d8dda406f71/root/var/svc/log/oxide-crucible-agent:default.log |
1057 | 2024-02-16T07:11:29.289Z | uploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_17fb421d-08d1-4991-aedd-9d8dda406f71/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes) |
1058 | 2024-02-16T07:11:29.518Z | Error: No response to keepalive - Terminating session |
1059 | 2024-02-16T07:11:30.357Z | uploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_17fb421d-08d1-4991-aedd-9d8dda406f71/root/var/svc/log/oxide-crucible-downstairs:default.log |
1060 | 2024-02-16T07:11:30.358Z | uploading: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_17fb421d-08d1-4991-aedd-9d8dda406f71/root/var/svc/log/oxide-crucible-downstairs:downstairs-e5553cc4-5ab5-4967-8f91-fc964a8c5747.log (5612 bytes) |
1061 | 2024-02-16T07:11:31.451Z | uploaded: /pool/ext/616b26df-e62a-4c68-b506-f4a923d8aaf7/crypt/zone/oxz_crucible_17fb421d-08d1-4991-aedd-9d8dda406f71/root/var/svc/log/oxide-crucible-downstairs:downstairs-e5553cc4-5ab5-4967-8f91-fc964a8c5747.log |
1062 | 2024-02-16T07:11:31.475Z | uploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_5be34856-ef50-4e5c-ad6a-349bf498d312/root/var/svc/log/oxide-cockroachdb:default.log (4407 bytes) |
1063 | 2024-02-16T07:11:32.546Z | uploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_5be34856-ef50-4e5c-ad6a-349bf498d312/root/var/svc/log/oxide-cockroachdb:default.log |
1064 | 2024-02-16T07:11:32.546Z | uploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_5be34856-ef50-4e5c-ad6a-349bf498d312/root/var/svc/log/oxide-zone-network-setup:default.log (1705 bytes) |
1065 | 2024-02-16T07:11:33.640Z | uploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_cockroachdb_5be34856-ef50-4e5c-ad6a-349bf498d312/root/var/svc/log/oxide-zone-network-setup:default.log |
1066 | 2024-02-16T07:11:33.664Z | uploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_a244a68e-123d-4e69-a61e-467a08d75e02/root/var/svc/log/oxide-crucible-agent:default.log (19612 bytes) |
1067 | 2024-02-16T07:11:34.776Z | uploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_a244a68e-123d-4e69-a61e-467a08d75e02/root/var/svc/log/oxide-crucible-agent:default.log |
1068 | 2024-02-16T07:11:34.776Z | uploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_a244a68e-123d-4e69-a61e-467a08d75e02/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes) |
1069 | 2024-02-16T07:11:35.869Z | uploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_a244a68e-123d-4e69-a61e-467a08d75e02/root/var/svc/log/oxide-crucible-downstairs:default.log |
1070 | 2024-02-16T07:11:35.869Z | uploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_a244a68e-123d-4e69-a61e-467a08d75e02/root/var/svc/log/oxide-crucible-downstairs:downstairs-4846bc02-2e5b-42be-8178-48bbe87df59d.log (3125 bytes) |
1071 | 2024-02-16T07:11:36.978Z | uploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_a244a68e-123d-4e69-a61e-467a08d75e02/root/var/svc/log/oxide-crucible-downstairs:downstairs-4846bc02-2e5b-42be-8178-48bbe87df59d.log |
1072 | 2024-02-16T07:11:36.978Z | uploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_a244a68e-123d-4e69-a61e-467a08d75e02/root/var/svc/log/oxide-crucible-downstairs:downstairs-8b06923f-9a49-4986-9172-6964a218a12c.log (3018 bytes) |
1073 | 2024-02-16T07:11:38.108Z | uploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_a244a68e-123d-4e69-a61e-467a08d75e02/root/var/svc/log/oxide-crucible-downstairs:downstairs-8b06923f-9a49-4986-9172-6964a218a12c.log |
1074 | 2024-02-16T07:11:38.142Z | uploading: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_10968697-a7e8-4d79-9cf9-2ac2c5ee91b0/root/var/svc/log/oxide-crucible-pantry:default.log (1218788 bytes) |
1075 | 2024-02-16T07:11:39.541Z | uploaded: /pool/ext/cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crypt/zone/oxz_crucible_pantry_10968697-a7e8-4d79-9cf9-2ac2c5ee91b0/root/var/svc/log/oxide-crucible-pantry:default.log |
1076 | 2024-02-16T07:11:39.541Z | uploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_a98b71d2-3e2f-4fb6-90c5-2e50de4d705b/root/var/svc/log/oxide-crucible-agent:default.log (14579 bytes) |
1077 | 2024-02-16T07:11:39.611Z | Error: No response de-activating SOL payload |
1078 | 2024-02-16T07:11:40.668Z | uploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_a98b71d2-3e2f-4fb6-90c5-2e50de4d705b/root/var/svc/log/oxide-crucible-agent:default.log |
1079 | 2024-02-16T07:11:40.668Z | uploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_a98b71d2-3e2f-4fb6-90c5-2e50de4d705b/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes) |
1080 | 2024-02-16T07:11:40.737Z | [SOL Session operational. Use ~? for help] |
1081 | 2024-02-16T07:11:41.775Z | uploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_a98b71d2-3e2f-4fb6-90c5-2e50de4d705b/root/var/svc/log/oxide-crucible-downstairs:default.log |
1082 | 2024-02-16T07:11:41.775Z | uploading: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_a98b71d2-3e2f-4fb6-90c5-2e50de4d705b/root/var/svc/log/oxide-crucible-downstairs:downstairs-14898c1d-6568-4472-80a0-bb67e5306dca.log (5611 bytes) |
1083 | 2024-02-16T07:11:42.867Z | uploaded: /pool/ext/ceb4461c-cf56-4719-ad3c-14430bfdfb60/crypt/zone/oxz_crucible_a98b71d2-3e2f-4fb6-90c5-2e50de4d705b/root/var/svc/log/oxide-crucible-downstairs:downstairs-14898c1d-6568-4472-80a0-bb67e5306dca.log |
1084 | 2024-02-16T07:11:42.867Z | uploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2c8f8f6b-7426-4579-b888-98c5d4f92e15/root/var/svc/log/oxide-crucible-agent:default.log (13686 bytes) |
1085 | 2024-02-16T07:11:43.984Z | uploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2c8f8f6b-7426-4579-b888-98c5d4f92e15/root/var/svc/log/oxide-crucible-agent:default.log |
1086 | 2024-02-16T07:11:43.984Z | uploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2c8f8f6b-7426-4579-b888-98c5d4f92e15/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes) |
1087 | 2024-02-16T07:11:45.079Z | uploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2c8f8f6b-7426-4579-b888-98c5d4f92e15/root/var/svc/log/oxide-crucible-downstairs:default.log |
1088 | 2024-02-16T07:11:45.079Z | uploading: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2c8f8f6b-7426-4579-b888-98c5d4f92e15/root/var/svc/log/oxide-crucible-downstairs:downstairs-93e97db7-8bfd-4738-a28c-a0c558469bea.log (3126 bytes) |
1089 | 2024-02-16T07:11:46.172Z | uploaded: /pool/ext/d462a7f7-b628-40fe-80ff-4e4189e2d62b/crypt/zone/oxz_crucible_2c8f8f6b-7426-4579-b888-98c5d4f92e15/root/var/svc/log/oxide-crucible-downstairs:downstairs-93e97db7-8bfd-4738-a28c-a0c558469bea.log |
1090 | 2024-02-16T07:11:46.172Z | uploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_643e9df1-7d24-4551-813d-7c58f246ccbc/root/var/svc/log/oxide-crucible-agent:default.log (20514 bytes) |
1091 | 2024-02-16T07:11:47.313Z | uploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_643e9df1-7d24-4551-813d-7c58f246ccbc/root/var/svc/log/oxide-crucible-agent:default.log |
1092 | 2024-02-16T07:11:47.314Z | uploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_643e9df1-7d24-4551-813d-7c58f246ccbc/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes) |
1093 | 2024-02-16T07:11:48.409Z | uploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_643e9df1-7d24-4551-813d-7c58f246ccbc/root/var/svc/log/oxide-crucible-downstairs:default.log |
1094 | 2024-02-16T07:11:48.409Z | uploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_643e9df1-7d24-4551-813d-7c58f246ccbc/root/var/svc/log/oxide-crucible-downstairs:downstairs-123be6e7-fe64-4260-87a8-72eae8e9f906.log (3018 bytes) |
1095 | 2024-02-16T07:11:49.501Z | uploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_643e9df1-7d24-4551-813d-7c58f246ccbc/root/var/svc/log/oxide-crucible-downstairs:downstairs-123be6e7-fe64-4260-87a8-72eae8e9f906.log |
1096 | 2024-02-16T07:11:49.525Z | uploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_643e9df1-7d24-4551-813d-7c58f246ccbc/root/var/svc/log/oxide-crucible-downstairs:downstairs-cb2654c1-0214-46e1-aaf0-733fbb9be756.log (3126 bytes) |
1097 | 2024-02-16T07:11:50.610Z | uploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_643e9df1-7d24-4551-813d-7c58f246ccbc/root/var/svc/log/oxide-crucible-downstairs:downstairs-cb2654c1-0214-46e1-aaf0-733fbb9be756.log |
1098 | 2024-02-16T07:11:50.648Z | uploading: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_d3f2ecd3-c994-4a91-b7a8-ca8b1c02475c/root/var/svc/log/oxide-crucible-pantry:default.log (2338 bytes) |
1099 | 2024-02-16T07:11:51.752Z | uploaded: /pool/ext/e4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_pantry_d3f2ecd3-c994-4a91-b7a8-ca8b1c02475c/root/var/svc/log/oxide-crucible-pantry:default.log |
1100 | 2024-02-16T07:11:51.787Z | uploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c093a4e8-46ea-4e77-a429-3cf96c6a0a25/root/var/svc/log/oxide-crucible-agent:default.log (17118 bytes) |
1101 | 2024-02-16T07:11:52.922Z | uploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c093a4e8-46ea-4e77-a429-3cf96c6a0a25/root/var/svc/log/oxide-crucible-agent:default.log |
1102 | 2024-02-16T07:11:52.922Z | uploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c093a4e8-46ea-4e77-a429-3cf96c6a0a25/root/var/svc/log/oxide-crucible-downstairs:default.log (75 bytes) |
1103 | 2024-02-16T07:11:54.070Z | uploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c093a4e8-46ea-4e77-a429-3cf96c6a0a25/root/var/svc/log/oxide-crucible-downstairs:default.log |
1104 | 2024-02-16T07:11:54.070Z | uploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c093a4e8-46ea-4e77-a429-3cf96c6a0a25/root/var/svc/log/oxide-crucible-downstairs:downstairs-dc4c94a1-8fb5-4950-b6de-e83fac607f24.log (5690 bytes) |
1105 | 2024-02-16T07:11:55.244Z | uploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c093a4e8-46ea-4e77-a429-3cf96c6a0a25/root/var/svc/log/oxide-crucible-downstairs:downstairs-dc4c94a1-8fb5-4950-b6de-e83fac607f24.log |
1106 | 2024-02-16T07:11:55.244Z | uploading: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c093a4e8-46ea-4e77-a429-3cf96c6a0a25/root/var/svc/log/oxide-crucible-downstairs:snapshot-dc4c94a1-8fb5-4950-b6de-e83fac607f24-c5ff36fd-fda4-4d5b-8854-5b065543fc40.log (5302 bytes) |
1107 | 2024-02-16T07:11:56.366Z | uploaded: /pool/ext/f4b4dc87-ab46-49fb-a4b4-d361ae214c03/crypt/zone/oxz_crucible_c093a4e8-46ea-4e77-a429-3cf96c6a0a25/root/var/svc/log/oxide-crucible-downstairs:snapshot-dc4c94a1-8fb5-4950-b6de-e83fac607f24-c5ff36fd-fda4-4d5b-8854-5b065543fc40.log |
1108 | 2024-02-16T07:11:56.366Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092/root/var/log/chrony/measurements.log (20413 bytes) |
1109 | 2024-02-16T07:11:56.488Z | upload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092/root/var/log/chrony/measurements.log" changed size mid upload: 20413 -> 23564 |
1110 | 2024-02-16T07:11:57.561Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092/root/var/log/chrony/measurements.log |
1111 | 2024-02-16T07:11:57.561Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092/root/var/log/chrony/statistics.log (13338 bytes) |
1112 | 2024-02-16T07:11:57.676Z | upload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092/root/var/log/chrony/statistics.log" changed size mid upload: 13338 -> 15912 |
1113 | 2024-02-16T07:11:58.773Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092/root/var/log/chrony/statistics.log |
1114 | 2024-02-16T07:11:58.774Z | uploading: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092/root/var/log/chrony/tracking.log (1849 bytes) |
1115 | 2024-02-16T07:11:58.823Z | upload warning: file "/pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092/root/var/log/chrony/tracking.log" changed size mid upload: 1849 -> 1981 |
1116 | 2024-02-16T07:11:59.902Z | uploaded: /pool/ext/31bd71cd-4736-4a12-a387-9b74b050396f/crypt/zone/oxz_ntp_c327c44a-8044-4bad-bedf-47ac60177092/root/var/log/chrony/tracking.log |