Max level shown:
12025-05-08T22:10:13.064ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:52485
22025-05-08T22:10:13.064ZINFOcrucible-agent (datafile): region 7b7199ff-ff33-47b5-818f-1ac4262702ba state: Requested
32025-05-08T22:10:13.065ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-05-08T22:10:13.065ZINFOcrucible-agent (dropshot): request completed latency_us = 608 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52485 req_id = a136a099-9468-4356-a385-8f70ed29be44 response_code = 200 uri = /crucible/0/regions
52025-05-08T22:10:13.076ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_b15b0dc6-d15d-4130-bb63-393ab527855b/crucible/regions/7b7199ff-ff33-47b5-818f-1ac4262702ba
62025-05-08T22:10:13.076ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_b15b0dc6-d15d-4130-bb63-393ab527855b/crucible/regions/7b7199ff-ff33-47b5-818f-1ac4262702ba
72025-05-08T22:10:13.135ZINFOcrucible-agent (worker): creating region Region { id: RegionId("7b7199ff-ff33-47b5-818f-1ac4262702ba"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/7b7199ff-ff33-47b5-818f-1ac4262702ba" region = 7b7199ff-ff33-47b5-818f-1ac4262702ba
82025-05-08T22:10:13.135ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "7b7199ff-ff33-47b5-818f-1ac4262702ba" "--data" "/data/regions/7b7199ff-ff33-47b5-818f-1ac4262702ba" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 7b7199ff-ff33-47b5-818f-1ac4262702ba
92025-05-08T22:10:13.218ZINFOcrucible-agent (worker): region files created ok region = 7b7199ff-ff33-47b5-818f-1ac4262702ba
102025-05-08T22:10:13.218ZINFOcrucible-agent (datafile): region 7b7199ff-ff33-47b5-818f-1ac4262702ba state: Requested -> Created
112025-05-08T22:10:13.218ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-05-08T22:10:13.218ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-7b7199ff-ff33-47b5-818f-1ac4262702ba
132025-05-08T22:10:13.229ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-7b7199ff-ff33-47b5-818f-1ac4262702ba
142025-05-08T22:10:13.248ZINFOcrucible-agent (worker): creating config property group
152025-05-08T22:10:13.251ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-7b7199ff-ff33-47b5-818f-1ac4262702ba
162025-05-08T22:10:13.251ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/7b7199ff-ff33-47b5-818f-1ac4262702ba
172025-05-08T22:10:13.251ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-05-08T22:10:13.251ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
192025-05-08T22:10:13.251ZINFOcrucible-agent (worker): commit
202025-05-08T22:10:13.256ZINFOcrucible-agent (worker): ok!
212025-05-08T22:10:13.261ZINFOcrucible-agent (worker): SMF ok!
222025-05-08T22:10:13.404ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52485 req_id = 7ef65b59-434d-4829-ba61-e94a795c5636 response_code = 200 uri = /crucible/0/regions
232025-05-08T22:10:14.257ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:52485 req_id = bd1d2549-9d7a-4040-860e-a096ee10d525 response_code = 200 uri = /crucible/0/regions/8580f062-50ca-4b0b-9125-158c35adaf2c
242025-05-08T22:10:14.263ZINFOcrucible-agent (datafile): path is "/data/regions/8580f062-50ca-4b0b-9125-158c35adaf2c"
252025-05-08T22:10:14.273ZINFOcrucible-agent (datafile): dataset is oxp_b15b0dc6-d15d-4130-bb63-393ab527855b/crucible/regions/8580f062-50ca-4b0b-9125-158c35adaf2c
262025-05-08T22:10:14.282ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b15b0dc6-d15d-4130-bb63-393ab527855b/crucible/regions/8580f062-50ca-4b0b-9125-158c35adaf2c@224bfdfa-283e-4809-b8e8-e1c76df49e6c
272025-05-08T22:10:14.282ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b15b0dc6-d15d-4130-bb63-393ab527855b/crucible/regions/8580f062-50ca-4b0b-9125-158c35adaf2c", "224bfdfa-283e-4809-b8e8-e1c76df49e6c"]
282025-05-08T22:10:14.294ZINFOcrucible-agent (dropshot): request completed latency_us = 31354 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:52485 req_id = e9ec31ba-1999-4f03-9ef6-56edf850bd17 response_code = 200 uri = /crucible/0/regions/8580f062-50ca-4b0b-9125-158c35adaf2c/snapshots/224bfdfa-283e-4809-b8e8-e1c76df49e6c
292025-05-08T22:10:14.297ZINFOcrucible-agent (datafile): path is "/data/regions/8580f062-50ca-4b0b-9125-158c35adaf2c"
302025-05-08T22:10:14.305ZINFOcrucible-agent (datafile): dataset is oxp_b15b0dc6-d15d-4130-bb63-393ab527855b/crucible/regions/8580f062-50ca-4b0b-9125-158c35adaf2c
312025-05-08T22:10:14.314ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b15b0dc6-d15d-4130-bb63-393ab527855b/crucible/regions/8580f062-50ca-4b0b-9125-158c35adaf2c@224bfdfa-283e-4809-b8e8-e1c76df49e6c
322025-05-08T22:10:14.314ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b15b0dc6-d15d-4130-bb63-393ab527855b/crucible/regions/8580f062-50ca-4b0b-9125-158c35adaf2c", "224bfdfa-283e-4809-b8e8-e1c76df49e6c"]
332025-05-08T22:10:14.325ZINFOcrucible-agent (datafile): requesting running snapshot 8580f062-50ca-4b0b-9125-158c35adaf2c-224bfdfa-283e-4809-b8e8-e1c76df49e6c state: Requested
342025-05-08T22:10:14.325ZINFOcrucible-agent (worker): applying SMF actions for region 8580f062-50ca-4b0b-9125-158c35adaf2c running snapshot 224bfdfa-283e-4809-b8e8-e1c76df49e6c (state Requested)...
352025-05-08T22:10:14.325ZINFOcrucible-agent (dropshot): request completed latency_us = 27963 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52485 req_id = ce960484-4b7d-4451-9fcd-b9f34c362c51 response_code = 200 uri = /crucible/0/regions/8580f062-50ca-4b0b-9125-158c35adaf2c/snapshots/224bfdfa-283e-4809-b8e8-e1c76df49e6c/run
362025-05-08T22:10:14.328ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-8580f062-50ca-4b0b-9125-158c35adaf2c-224bfdfa-283e-4809-b8e8-e1c76df49e6c
372025-05-08T22:10:14.338ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-8580f062-50ca-4b0b-9125-158c35adaf2c-224bfdfa-283e-4809-b8e8-e1c76df49e6c
382025-05-08T22:10:14.349ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-8580f062-50ca-4b0b-9125-158c35adaf2c-224bfdfa-283e-4809-b8e8-e1c76df49e6c
392025-05-08T22:10:14.349ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8580f062-50ca-4b0b-9125-158c35adaf2c/.zfs/snapshot/224bfdfa-283e-4809-b8e8-e1c76df49e6c
402025-05-08T22:10:14.349ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-05-08T22:10:14.349ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-05-08T22:10:14.349ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
432025-05-08T22:10:14.349ZINFOcrucible-agent (worker): commit
442025-05-08T22:10:14.353ZINFOcrucible-agent (worker): ok!
452025-05-08T22:10:14.358ZINFOcrucible-agent (worker): SMF ok!
462025-05-08T22:10:14.358ZINFOcrucible-agent (datafile): region 8580f062-50ca-4b0b-9125-158c35adaf2c running snapshot 224bfdfa-283e-4809-b8e8-e1c76df49e6c state: Requested -> Created
472025-05-08T22:10:14.482ZINFOcrucible-agent (datafile): path is "/data/regions/8580f062-50ca-4b0b-9125-158c35adaf2c"
482025-05-08T22:10:14.490ZINFOcrucible-agent (datafile): dataset is oxp_b15b0dc6-d15d-4130-bb63-393ab527855b/crucible/regions/8580f062-50ca-4b0b-9125-158c35adaf2c
492025-05-08T22:10:14.499ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b15b0dc6-d15d-4130-bb63-393ab527855b/crucible/regions/8580f062-50ca-4b0b-9125-158c35adaf2c@224bfdfa-283e-4809-b8e8-e1c76df49e6c
502025-05-08T22:10:14.499ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b15b0dc6-d15d-4130-bb63-393ab527855b/crucible/regions/8580f062-50ca-4b0b-9125-158c35adaf2c", "224bfdfa-283e-4809-b8e8-e1c76df49e6c"]
512025-05-08T22:10:14.507ZINFOcrucible-agent (dropshot): request completed latency_us = 25666 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52485 req_id = acf8d89b-4e44-4644-9c6e-a62625f1f62f response_code = 200 uri = /crucible/0/regions/8580f062-50ca-4b0b-9125-158c35adaf2c/snapshots/224bfdfa-283e-4809-b8e8-e1c76df49e6c/run