Max level shown:
1 [ Dec 21 01:16:02 Enabled. ]
2 [ Dec 21 01:16:02 Rereading configuration. ]
3 [ Dec 21 01:16:04 Rereading configuration. ]
4 [ Dec 21 01:16:11 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible -l [fd00:1122:3344:101::15]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-12-21T01:16:11.637ZINFOcrucible-agent: dataset: "oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible"
72024-12-21T01:16:11.641ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::15]:32345
82024-12-21T01:16:11.641ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-12-21T01:16:11.656ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-12-21T01:16:11.759ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::15]:32345
112024-12-21T01:17:23.556ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:63834
122024-12-21T01:17:23.596ZINFOcrucible-agent (datafile): region b2700033-df0d-4689-9eac-0104ae39e1e2 state: Requested
132024-12-21T01:17:23.603ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
142024-12-21T01:17:23.608ZINFOcrucible-agent (dropshot): request completed latency_us = 17770 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63834 req_id = 50399521-8c42-4893-8b19-d9d7a1c86582 response_code = 200 uri = /crucible/0/regions
152024-12-21T01:17:23.613ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible/regions/b2700033-df0d-4689-9eac-0104ae39e1e2
162024-12-21T01:17:23.613ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible/regions/b2700033-df0d-4689-9eac-0104ae39e1e2
172024-12-21T01:17:23.668ZINFOcrucible-agent (worker): creating region Region { id: RegionId("b2700033-df0d-4689-9eac-0104ae39e1e2"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/b2700033-df0d-4689-9eac-0104ae39e1e2" region = b2700033-df0d-4689-9eac-0104ae39e1e2
182024-12-21T01:17:23.670ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "b2700033-df0d-4689-9eac-0104ae39e1e2" "--data" "/data/regions/b2700033-df0d-4689-9eac-0104ae39e1e2" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = b2700033-df0d-4689-9eac-0104ae39e1e2
192024-12-21T01:17:23.817ZINFOcrucible-agent (dropshot): request completed latency_us = 78 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63834 req_id = 7ce40cc7-1868-44b9-b2ae-2803984b990a response_code = 200 uri = /crucible/0/regions
202024-12-21T01:17:23.825ZINFOcrucible-agent (worker): region files created ok region = b2700033-df0d-4689-9eac-0104ae39e1e2
212024-12-21T01:17:23.825ZINFOcrucible-agent (datafile): region b2700033-df0d-4689-9eac-0104ae39e1e2 state: Requested -> Created
222024-12-21T01:17:23.825ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-12-21T01:17:23.825ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-b2700033-df0d-4689-9eac-0104ae39e1e2
242024-12-21T01:17:23.830ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-b2700033-df0d-4689-9eac-0104ae39e1e2
252024-12-21T01:17:23.842ZINFOcrucible-agent (worker): creating config property group
262024-12-21T01:17:23.845ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-b2700033-df0d-4689-9eac-0104ae39e1e2
272024-12-21T01:17:23.846ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b2700033-df0d-4689-9eac-0104ae39e1e2
282024-12-21T01:17:23.846ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-12-21T01:17:23.846ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
302024-12-21T01:17:23.846ZINFOcrucible-agent (worker): commit
312024-12-21T01:17:23.849ZINFOcrucible-agent (worker): ok!
322024-12-21T01:17:23.853ZINFOcrucible-agent (worker): SMF ok!
332024-12-21T01:17:24.331ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63834 req_id = 731d9140-fd28-47d2-911e-1daa3b381600 response_code = 200 uri = /crucible/0/regions
342024-12-21T01:27:37.638ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:55730
352024-12-21T01:27:37.642ZINFOcrucible-agent (dropshot): request completed latency_us = 4291 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:55730 req_id = a4d8e19a-5f49-4327-8fe0-95ce63e79031 response_code = 200 uri = /crucible/0/regions/b2700033-df0d-4689-9eac-0104ae39e1e2
362024-12-21T01:27:37.650ZINFOcrucible-agent (datafile): path is "/data/regions/b2700033-df0d-4689-9eac-0104ae39e1e2"
372024-12-21T01:27:37.665ZINFOcrucible-agent (datafile): dataset is oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible/regions/b2700033-df0d-4689-9eac-0104ae39e1e2
382024-12-21T01:27:37.679ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible/regions/b2700033-df0d-4689-9eac-0104ae39e1e2@f46300ad-91ae-40ca-bb40-7dba2e257bcc
392024-12-21T01:27:37.679ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible/regions/b2700033-df0d-4689-9eac-0104ae39e1e2", "f46300ad-91ae-40ca-bb40-7dba2e257bcc"]
402024-12-21T01:27:37.697ZINFOcrucible-agent (dropshot): request completed latency_us = 47307 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:55730 req_id = 83806f93-1591-46e8-adbf-9c95236d3377 response_code = 200 uri = /crucible/0/regions/b2700033-df0d-4689-9eac-0104ae39e1e2/snapshots/f46300ad-91ae-40ca-bb40-7dba2e257bcc
412024-12-21T01:27:37.706ZINFOcrucible-agent (datafile): path is "/data/regions/b2700033-df0d-4689-9eac-0104ae39e1e2"
422024-12-21T01:27:37.719ZINFOcrucible-agent (datafile): dataset is oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible/regions/b2700033-df0d-4689-9eac-0104ae39e1e2
432024-12-21T01:27:37.730ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible/regions/b2700033-df0d-4689-9eac-0104ae39e1e2@f46300ad-91ae-40ca-bb40-7dba2e257bcc
442024-12-21T01:27:37.730ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible/regions/b2700033-df0d-4689-9eac-0104ae39e1e2", "f46300ad-91ae-40ca-bb40-7dba2e257bcc"]
452024-12-21T01:27:37.739ZINFOcrucible-agent (datafile): requesting running snapshot b2700033-df0d-4689-9eac-0104ae39e1e2-f46300ad-91ae-40ca-bb40-7dba2e257bcc state: Requested
462024-12-21T01:27:37.740ZINFOcrucible-agent (worker): applying SMF actions for region b2700033-df0d-4689-9eac-0104ae39e1e2 running snapshot f46300ad-91ae-40ca-bb40-7dba2e257bcc (state Requested)...
472024-12-21T01:27:37.740ZINFOcrucible-agent (dropshot): request completed latency_us = 34667 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55730 req_id = 73ac571b-ae0d-46b8-8c40-1ca561269d44 response_code = 200 uri = /crucible/0/regions/b2700033-df0d-4689-9eac-0104ae39e1e2/snapshots/f46300ad-91ae-40ca-bb40-7dba2e257bcc/run
482024-12-21T01:27:37.749ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-b2700033-df0d-4689-9eac-0104ae39e1e2-f46300ad-91ae-40ca-bb40-7dba2e257bcc
492024-12-21T01:27:37.753ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-b2700033-df0d-4689-9eac-0104ae39e1e2-f46300ad-91ae-40ca-bb40-7dba2e257bcc
502024-12-21T01:27:37.766ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-b2700033-df0d-4689-9eac-0104ae39e1e2-f46300ad-91ae-40ca-bb40-7dba2e257bcc
512024-12-21T01:27:37.767ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b2700033-df0d-4689-9eac-0104ae39e1e2/.zfs/snapshot/f46300ad-91ae-40ca-bb40-7dba2e257bcc
522024-12-21T01:27:37.767ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
532024-12-21T01:27:37.767ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
542024-12-21T01:27:37.767ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
552024-12-21T01:27:37.767ZINFOcrucible-agent (worker): commit
562024-12-21T01:27:37.771ZINFOcrucible-agent (worker): ok!
572024-12-21T01:27:37.775ZINFOcrucible-agent (worker): SMF ok!
582024-12-21T01:27:37.775ZINFOcrucible-agent (datafile): region b2700033-df0d-4689-9eac-0104ae39e1e2 running snapshot f46300ad-91ae-40ca-bb40-7dba2e257bcc state: Requested -> Created
592024-12-21T01:27:37.980ZINFOcrucible-agent (datafile): path is "/data/regions/b2700033-df0d-4689-9eac-0104ae39e1e2"
602024-12-21T01:27:37.989ZINFOcrucible-agent (datafile): dataset is oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible/regions/b2700033-df0d-4689-9eac-0104ae39e1e2
612024-12-21T01:27:37.997ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible/regions/b2700033-df0d-4689-9eac-0104ae39e1e2@f46300ad-91ae-40ca-bb40-7dba2e257bcc
622024-12-21T01:27:37.997ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible/regions/b2700033-df0d-4689-9eac-0104ae39e1e2", "f46300ad-91ae-40ca-bb40-7dba2e257bcc"]
632024-12-21T01:27:38.005ZINFOcrucible-agent (dropshot): request completed latency_us = 25433 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55730 req_id = 87bf8b3f-a353-48d0-818e-5cb6b72fa4c3 response_code = 200 uri = /crucible/0/regions/b2700033-df0d-4689-9eac-0104ae39e1e2/snapshots/f46300ad-91ae-40ca-bb40-7dba2e257bcc/run