Max level shown:
12024-05-20T20:49:38.147ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:48784
22024-05-20T20:49:38.205ZINFOcrucible-agent (datafile): region 06957e4f-165e-4db6-b395-14af712e465f state: Requested
32024-05-20T20:49:38.206ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42024-05-20T20:49:38.206ZINFOcrucible-agent (dropshot): request completed latency_us = 742 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48784 req_id = 611e75d6-37c6-4cff-91f9-922ec0472b38 response_code = 200 uri = /crucible/0/regions
52024-05-20T20:49:38.216ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_b1ad2e3e-e2af-4c2a-94f1-4865bf8462a0/crucible/regions/06957e4f-165e-4db6-b395-14af712e465f
62024-05-20T20:49:38.216ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_b1ad2e3e-e2af-4c2a-94f1-4865bf8462a0/crucible/regions/06957e4f-165e-4db6-b395-14af712e465f
72024-05-20T20:49:38.275ZINFOcrucible-agent (worker): creating region Region { id: RegionId("06957e4f-165e-4db6-b395-14af712e465f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/06957e4f-165e-4db6-b395-14af712e465f" region = 06957e4f-165e-4db6-b395-14af712e465f
82024-05-20T20:49:38.277ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "06957e4f-165e-4db6-b395-14af712e465f" "--data" "/data/regions/06957e4f-165e-4db6-b395-14af712e465f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 06957e4f-165e-4db6-b395-14af712e465f
92024-05-20T20:49:38.381ZINFOcrucible-agent (worker): region files created ok region = 06957e4f-165e-4db6-b395-14af712e465f
102024-05-20T20:49:38.381ZINFOcrucible-agent (datafile): region 06957e4f-165e-4db6-b395-14af712e465f state: Requested -> Created
112024-05-20T20:49:38.381ZINFOcrucible-agent (worker): applying SMF actions post create...
122024-05-20T20:49:38.381ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-06957e4f-165e-4db6-b395-14af712e465f
132024-05-20T20:49:38.390ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-06957e4f-165e-4db6-b395-14af712e465f
142024-05-20T20:49:38.411ZINFOcrucible-agent (worker): creating config property group
152024-05-20T20:49:38.417ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-06957e4f-165e-4db6-b395-14af712e465f
162024-05-20T20:49:38.417ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/06957e4f-165e-4db6-b395-14af712e465f
172024-05-20T20:49:38.417ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182024-05-20T20:49:38.417ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
192024-05-20T20:49:38.417ZINFOcrucible-agent (worker): commit
202024-05-20T20:49:38.426ZINFOcrucible-agent (worker): ok!
212024-05-20T20:49:38.432ZINFOcrucible-agent (dropshot): request completed latency_us = 69 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48784 req_id = 5a2bcc85-98dd-49b6-a7d8-e2eae95a8f3a response_code = 200 uri = /crucible/0/regions
222024-05-20T20:49:38.435ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6
232024-05-20T20:49:38.435ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
242024-05-20T20:49:38.435ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::18
252024-05-20T20:49:38.435ZINFOcrucible-agent (worker): reconfig required
262024-05-20T20:49:38.435ZINFOcrucible-agent (worker): using existing config property group
272024-05-20T20:49:38.435ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-b89aed98-e92d-4d99-a5f2-8479c9aa45c6
282024-05-20T20:49:38.435ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6
292024-05-20T20:49:38.435ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
302024-05-20T20:49:38.435ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
312024-05-20T20:49:38.435ZINFOcrucible-agent (worker): commit
322024-05-20T20:49:38.455ZINFOcrucible-agent (worker): ok!
332024-05-20T20:49:38.455ZINFOcrucible-agent (worker): SMF ok!
342024-05-20T20:49:39.484ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:49564
352024-05-20T20:49:39.490ZINFOcrucible-agent (dropshot): request completed latency_us = 5566 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:49564 req_id = a7a08058-bead-4c43-9e36-c9147a5ed5a0 response_code = 200 uri = /crucible/0/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6
362024-05-20T20:49:39.490ZINFOcrucible-agent (datafile): path is "/data/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6"
372024-05-20T20:49:39.499ZINFOcrucible-agent (datafile): dataset is oxp_b1ad2e3e-e2af-4c2a-94f1-4865bf8462a0/crucible/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6
382024-05-20T20:49:39.507ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b1ad2e3e-e2af-4c2a-94f1-4865bf8462a0/crucible/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6@d410121f-bcfb-49e7-8009-f5046fce91de
392024-05-20T20:49:39.507ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b1ad2e3e-e2af-4c2a-94f1-4865bf8462a0/crucible/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6", "d410121f-bcfb-49e7-8009-f5046fce91de"]
402024-05-20T20:49:39.522ZINFOcrucible-agent (dropshot): request completed latency_us = 31603 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:49564 req_id = 190adc62-d0f1-4043-a3ac-46a5802d37ac response_code = 200 uri = /crucible/0/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6/snapshots/d410121f-bcfb-49e7-8009-f5046fce91de
412024-05-20T20:49:39.522ZINFOcrucible-agent (datafile): path is "/data/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6"
422024-05-20T20:49:39.529ZINFOcrucible-agent (datafile): dataset is oxp_b1ad2e3e-e2af-4c2a-94f1-4865bf8462a0/crucible/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6
432024-05-20T20:49:39.537ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b1ad2e3e-e2af-4c2a-94f1-4865bf8462a0/crucible/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6@d410121f-bcfb-49e7-8009-f5046fce91de
442024-05-20T20:49:39.537ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b1ad2e3e-e2af-4c2a-94f1-4865bf8462a0/crucible/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6", "d410121f-bcfb-49e7-8009-f5046fce91de"]
452024-05-20T20:49:39.546ZINFOcrucible-agent (datafile): requesting running snapshot b89aed98-e92d-4d99-a5f2-8479c9aa45c6-d410121f-bcfb-49e7-8009-f5046fce91de state: Requested
462024-05-20T20:49:39.547ZINFOcrucible-agent (worker): applying SMF actions for region b89aed98-e92d-4d99-a5f2-8479c9aa45c6 running snapshot d410121f-bcfb-49e7-8009-f5046fce91de (state Requested)...
472024-05-20T20:49:39.547ZINFOcrucible-agent (dropshot): request completed latency_us = 24887 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49564 req_id = b809530c-b12f-4085-bbb4-ccfc995bd6b9 response_code = 200 uri = /crucible/0/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6/snapshots/d410121f-bcfb-49e7-8009-f5046fce91de/run
482024-05-20T20:49:39.548ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6
492024-05-20T20:49:39.548ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
502024-05-20T20:49:39.548ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::18
512024-05-20T20:49:39.548ZINFOcrucible-agent (worker): reconfig required
522024-05-20T20:49:39.548ZINFOcrucible-agent (worker): using existing config property group
532024-05-20T20:49:39.548ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-b89aed98-e92d-4d99-a5f2-8479c9aa45c6
542024-05-20T20:49:39.549ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6
552024-05-20T20:49:39.549ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
562024-05-20T20:49:39.549ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
572024-05-20T20:49:39.549ZINFOcrucible-agent (worker): commit
582024-05-20T20:49:39.558ZINFOcrucible-agent (worker): ok!
592024-05-20T20:49:39.559ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-b89aed98-e92d-4d99-a5f2-8479c9aa45c6-d410121f-bcfb-49e7-8009-f5046fce91de
602024-05-20T20:49:39.569ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-b89aed98-e92d-4d99-a5f2-8479c9aa45c6-d410121f-bcfb-49e7-8009-f5046fce91de
612024-05-20T20:49:39.586ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-b89aed98-e92d-4d99-a5f2-8479c9aa45c6-d410121f-bcfb-49e7-8009-f5046fce91de
622024-05-20T20:49:39.586ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b89aed98-e92d-4d99-a5f2-8479c9aa45c6/.zfs/snapshot/d410121f-bcfb-49e7-8009-f5046fce91de
632024-05-20T20:49:39.586ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
642024-05-20T20:49:39.586ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
652024-05-20T20:49:39.586ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
662024-05-20T20:49:39.586ZINFOcrucible-agent (worker): commit
672024-05-20T20:49:39.591ZINFOcrucible-agent (worker): ok!
682024-05-20T20:49:39.599ZINFOcrucible-agent (worker): SMF ok!
692024-05-20T20:49:39.599ZINFOcrucible-agent (datafile): region b89aed98-e92d-4d99-a5f2-8479c9aa45c6 running snapshot d410121f-bcfb-49e7-8009-f5046fce91de state: Requested -> Created