Max level shown:
12024-08-26T20:34:25.055ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:50616
22024-08-26T20:34:25.055ZINFOcrucible-agent (datafile): region a542524c-9257-4898-aa60-97c6d13df819 state: Requested
32024-08-26T20:34:25.056ZINFOcrucible-agent (dropshot): request completed latency_us = 621 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50616 req_id = bbd56417-252f-434f-aea2-cbbdae7afc8a response_code = 200 uri = /crucible/0/regions
42024-08-26T20:34:25.056ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52024-08-26T20:34:25.067ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_ce2b758a-3ca2-45e5-9d43-b4915d227251/crucible/regions/a542524c-9257-4898-aa60-97c6d13df819
62024-08-26T20:34:25.067ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_ce2b758a-3ca2-45e5-9d43-b4915d227251/crucible/regions/a542524c-9257-4898-aa60-97c6d13df819
72024-08-26T20:34:25.194ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50616 req_id = 9f21068a-16f7-4f15-b0de-7dd65bfe12d4 response_code = 200 uri = /crucible/0/regions
82024-08-26T20:34:25.220ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a542524c-9257-4898-aa60-97c6d13df819"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/a542524c-9257-4898-aa60-97c6d13df819" region = a542524c-9257-4898-aa60-97c6d13df819
92024-08-26T20:34:25.220ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "a542524c-9257-4898-aa60-97c6d13df819" "--data" "/data/regions/a542524c-9257-4898-aa60-97c6d13df819" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = a542524c-9257-4898-aa60-97c6d13df819
102024-08-26T20:34:25.573ZINFOcrucible-agent (worker): region files created ok region = a542524c-9257-4898-aa60-97c6d13df819
112024-08-26T20:34:25.573ZINFOcrucible-agent (datafile): region a542524c-9257-4898-aa60-97c6d13df819 state: Requested -> Created
122024-08-26T20:34:25.573ZINFOcrucible-agent (worker): applying SMF actions post create...
132024-08-26T20:34:25.591ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/64e1983d-1c63-49de-9dea-6afe9026c728
142024-08-26T20:34:25.591ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
152024-08-26T20:34:25.591ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::18
162024-08-26T20:34:25.591ZINFOcrucible-agent (worker): reconfig required
172024-08-26T20:34:25.592ZINFOcrucible-agent (worker): using existing config property group
182024-08-26T20:34:25.592ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-64e1983d-1c63-49de-9dea-6afe9026c728
192024-08-26T20:34:25.592ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/64e1983d-1c63-49de-9dea-6afe9026c728
202024-08-26T20:34:25.592ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
212024-08-26T20:34:25.592ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
222024-08-26T20:34:25.592ZINFOcrucible-agent (worker): commit
232024-08-26T20:34:25.617ZINFOcrucible-agent (worker): ok!
242024-08-26T20:34:25.618ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a542524c-9257-4898-aa60-97c6d13df819
252024-08-26T20:34:25.642ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a542524c-9257-4898-aa60-97c6d13df819
262024-08-26T20:34:25.691ZINFOcrucible-agent (worker): creating config property group
272024-08-26T20:34:25.696ZINFOcrucible-agent (dropshot): request completed latency_us = 613 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50616 req_id = abfa306c-907e-4de5-ad77-a3694a960e68 response_code = 200 uri = /crucible/0/regions
282024-08-26T20:34:25.737ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a542524c-9257-4898-aa60-97c6d13df819
292024-08-26T20:34:25.738ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a542524c-9257-4898-aa60-97c6d13df819
302024-08-26T20:34:25.738ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
312024-08-26T20:34:25.738ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
322024-08-26T20:34:25.738ZINFOcrucible-agent (worker): commit
332024-08-26T20:34:25.762ZINFOcrucible-agent (worker): ok!
342024-08-26T20:34:25.804ZINFOcrucible-agent (worker): SMF ok!
352024-08-26T20:34:27.737ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:50616 req_id = 579b9e6e-3608-4ef7-ae59-7c38f81acaef response_code = 200 uri = /crucible/0/regions/64e1983d-1c63-49de-9dea-6afe9026c728
362024-08-26T20:34:27.750ZINFOcrucible-agent (datafile): path is "/data/regions/64e1983d-1c63-49de-9dea-6afe9026c728"
372024-08-26T20:34:27.759ZINFOcrucible-agent (datafile): dataset is oxp_ce2b758a-3ca2-45e5-9d43-b4915d227251/crucible/regions/64e1983d-1c63-49de-9dea-6afe9026c728
382024-08-26T20:34:27.769ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ce2b758a-3ca2-45e5-9d43-b4915d227251/crucible/regions/64e1983d-1c63-49de-9dea-6afe9026c728@1908eac5-177e-4cfe-9731-3b16b18b0242
392024-08-26T20:34:27.769ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ce2b758a-3ca2-45e5-9d43-b4915d227251/crucible/regions/64e1983d-1c63-49de-9dea-6afe9026c728", "1908eac5-177e-4cfe-9731-3b16b18b0242"]
402024-08-26T20:34:27.784ZINFOcrucible-agent (dropshot): request completed latency_us = 42812 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:50616 req_id = 49ded5d5-be9b-40f8-801d-74527bb68f21 response_code = 200 uri = /crucible/0/regions/64e1983d-1c63-49de-9dea-6afe9026c728/snapshots/1908eac5-177e-4cfe-9731-3b16b18b0242
412024-08-26T20:34:27.788ZINFOcrucible-agent (datafile): path is "/data/regions/64e1983d-1c63-49de-9dea-6afe9026c728"
422024-08-26T20:34:27.795ZINFOcrucible-agent (datafile): dataset is oxp_ce2b758a-3ca2-45e5-9d43-b4915d227251/crucible/regions/64e1983d-1c63-49de-9dea-6afe9026c728
432024-08-26T20:34:27.804ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ce2b758a-3ca2-45e5-9d43-b4915d227251/crucible/regions/64e1983d-1c63-49de-9dea-6afe9026c728@1908eac5-177e-4cfe-9731-3b16b18b0242
442024-08-26T20:34:27.804ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ce2b758a-3ca2-45e5-9d43-b4915d227251/crucible/regions/64e1983d-1c63-49de-9dea-6afe9026c728", "1908eac5-177e-4cfe-9731-3b16b18b0242"]
452024-08-26T20:34:27.813ZINFOcrucible-agent (datafile): requesting running snapshot 64e1983d-1c63-49de-9dea-6afe9026c728-1908eac5-177e-4cfe-9731-3b16b18b0242 state: Requested
462024-08-26T20:34:27.813ZINFOcrucible-agent (worker): applying SMF actions for region 64e1983d-1c63-49de-9dea-6afe9026c728 running snapshot 1908eac5-177e-4cfe-9731-3b16b18b0242 (state Requested)...
472024-08-26T20:34:27.813ZINFOcrucible-agent (dropshot): request completed latency_us = 25835 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50616 req_id = f1def9b8-50f6-45db-8cf8-f9ce2216338c response_code = 200 uri = /crucible/0/regions/64e1983d-1c63-49de-9dea-6afe9026c728/snapshots/1908eac5-177e-4cfe-9731-3b16b18b0242/run
482024-08-26T20:34:27.814ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/64e1983d-1c63-49de-9dea-6afe9026c728
492024-08-26T20:34:27.814ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
502024-08-26T20:34:27.814ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::18
512024-08-26T20:34:27.814ZINFOcrucible-agent (worker): reconfig required
522024-08-26T20:34:27.814ZINFOcrucible-agent (worker): using existing config property group
532024-08-26T20:34:27.814ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-64e1983d-1c63-49de-9dea-6afe9026c728
542024-08-26T20:34:27.814ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/64e1983d-1c63-49de-9dea-6afe9026c728
552024-08-26T20:34:27.814ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
562024-08-26T20:34:27.814ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
572024-08-26T20:34:27.815ZINFOcrucible-agent (worker): commit
582024-08-26T20:34:27.835ZINFOcrucible-agent (worker): ok!
592024-08-26T20:34:27.837ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-64e1983d-1c63-49de-9dea-6afe9026c728-1908eac5-177e-4cfe-9731-3b16b18b0242
602024-08-26T20:34:27.848ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-64e1983d-1c63-49de-9dea-6afe9026c728-1908eac5-177e-4cfe-9731-3b16b18b0242
612024-08-26T20:34:27.863ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-64e1983d-1c63-49de-9dea-6afe9026c728-1908eac5-177e-4cfe-9731-3b16b18b0242
622024-08-26T20:34:27.864ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/64e1983d-1c63-49de-9dea-6afe9026c728/.zfs/snapshot/1908eac5-177e-4cfe-9731-3b16b18b0242
632024-08-26T20:34:27.864ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
642024-08-26T20:34:27.864ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
652024-08-26T20:34:27.864ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
662024-08-26T20:34:27.864ZINFOcrucible-agent (worker): commit
672024-08-26T20:34:27.868ZINFOcrucible-agent (worker): ok!
682024-08-26T20:34:27.875ZINFOcrucible-agent (worker): SMF ok!
692024-08-26T20:34:27.875ZINFOcrucible-agent (datafile): region 64e1983d-1c63-49de-9dea-6afe9026c728 running snapshot 1908eac5-177e-4cfe-9731-3b16b18b0242 state: Requested -> Created
702024-08-26T20:34:28.146ZINFOcrucible-agent (datafile): path is "/data/regions/64e1983d-1c63-49de-9dea-6afe9026c728"
712024-08-26T20:34:28.154ZINFOcrucible-agent (datafile): dataset is oxp_ce2b758a-3ca2-45e5-9d43-b4915d227251/crucible/regions/64e1983d-1c63-49de-9dea-6afe9026c728
722024-08-26T20:34:28.163ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ce2b758a-3ca2-45e5-9d43-b4915d227251/crucible/regions/64e1983d-1c63-49de-9dea-6afe9026c728@1908eac5-177e-4cfe-9731-3b16b18b0242
732024-08-26T20:34:28.163ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ce2b758a-3ca2-45e5-9d43-b4915d227251/crucible/regions/64e1983d-1c63-49de-9dea-6afe9026c728", "1908eac5-177e-4cfe-9731-3b16b18b0242"]
742024-08-26T20:34:28.171ZINFOcrucible-agent (dropshot): request completed latency_us = 25742 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50616 req_id = 9640d5ff-b74c-4483-afcd-a40267d86d60 response_code = 200 uri = /crucible/0/regions/64e1983d-1c63-49de-9dea-6afe9026c728/snapshots/1908eac5-177e-4cfe-9731-3b16b18b0242/run