Max level shown:
12025-12-19T02:50:04.509ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::b]:41998
22025-12-19T02:50:04.509ZINFOcrucible-agent (datafile): region e677d7f0-b9e1-4e21-8e29-7914c81a75f3 state: Requested
32025-12-19T02:50:04.510ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-12-19T02:50:04.510ZINFOcrucible-agent (dropshot): request completed latency_us = 746 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:41998 req_id = 6e94ed45-866f-4fa4-81eb-a5b9e822db97 response_code = 200 uri = /crucible/0/regions
52025-12-19T02:50:04.522ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_500b0602-f58b-4938-a6a6-82653c23b377/crucible/regions/e677d7f0-b9e1-4e21-8e29-7914c81a75f3
62025-12-19T02:50:04.522ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_500b0602-f58b-4938-a6a6-82653c23b377/crucible/regions/e677d7f0-b9e1-4e21-8e29-7914c81a75f3
72025-12-19T02:50:04.583ZINFOcrucible-agent (worker): creating region Region { id: RegionId("e677d7f0-b9e1-4e21-8e29-7914c81a75f3"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/e677d7f0-b9e1-4e21-8e29-7914c81a75f3" region = e677d7f0-b9e1-4e21-8e29-7914c81a75f3
82025-12-19T02:50:04.584ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "e677d7f0-b9e1-4e21-8e29-7914c81a75f3" "--data" "/data/regions/e677d7f0-b9e1-4e21-8e29-7914c81a75f3" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = e677d7f0-b9e1-4e21-8e29-7914c81a75f3
92025-12-19T02:50:04.667ZINFOcrucible-agent (worker): region files created ok region = e677d7f0-b9e1-4e21-8e29-7914c81a75f3
102025-12-19T02:50:04.667ZINFOcrucible-agent (datafile): region e677d7f0-b9e1-4e21-8e29-7914c81a75f3 state: Requested -> Created
112025-12-19T02:50:04.667ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-12-19T02:50:04.681ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:41998 req_id = 3636c59d-1699-4b5e-bea8-aabe6179a9a1 response_code = 200 uri = /crucible/0/regions
132025-12-19T02:50:04.687ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-e677d7f0-b9e1-4e21-8e29-7914c81a75f3
142025-12-19T02:50:04.694ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-e677d7f0-b9e1-4e21-8e29-7914c81a75f3
152025-12-19T02:50:04.705ZINFOcrucible-agent (worker): creating config property group
162025-12-19T02:50:04.709ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-e677d7f0-b9e1-4e21-8e29-7914c81a75f3
172025-12-19T02:50:04.710ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/e677d7f0-b9e1-4e21-8e29-7914c81a75f3
182025-12-19T02:50:04.710ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-12-19T02:50:04.710ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
202025-12-19T02:50:04.710ZINFOcrucible-agent (worker): commit
212025-12-19T02:50:04.714ZINFOcrucible-agent (worker): ok!
222025-12-19T02:50:04.718ZINFOcrucible-agent (worker): SMF ok!
232025-12-19T02:50:06.156ZINFOcrucible-agent (dropshot): request completed latency_us = 71 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:41998 req_id = 1b0002be-bfbf-42fc-aa94-7c3d9c1d485d response_code = 200 uri = /crucible/0/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0
242025-12-19T02:50:06.159ZINFOcrucible-agent (datafile): path is "/data/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0"
252025-12-19T02:50:06.165ZINFOcrucible-agent (datafile): dataset is oxp_500b0602-f58b-4938-a6a6-82653c23b377/crucible/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0
262025-12-19T02:50:06.172ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_500b0602-f58b-4938-a6a6-82653c23b377/crucible/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0@7f5ccddd-a253-4cbd-997e-3e2353f6af65
272025-12-19T02:50:06.172ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_500b0602-f58b-4938-a6a6-82653c23b377/crucible/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0", "7f5ccddd-a253-4cbd-997e-3e2353f6af65"]
282025-12-19T02:50:06.181ZINFOcrucible-agent (dropshot): request completed latency_us = 21698 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:41998 req_id = 8978a14c-8bb7-4a1a-9c98-b3d3f8517af6 response_code = 200 uri = /crucible/0/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0/snapshots/7f5ccddd-a253-4cbd-997e-3e2353f6af65
292025-12-19T02:50:06.184ZINFOcrucible-agent (datafile): path is "/data/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0"
302025-12-19T02:50:06.190ZINFOcrucible-agent (datafile): dataset is oxp_500b0602-f58b-4938-a6a6-82653c23b377/crucible/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0
312025-12-19T02:50:06.196ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_500b0602-f58b-4938-a6a6-82653c23b377/crucible/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0@7f5ccddd-a253-4cbd-997e-3e2353f6af65
322025-12-19T02:50:06.196ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_500b0602-f58b-4938-a6a6-82653c23b377/crucible/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0", "7f5ccddd-a253-4cbd-997e-3e2353f6af65"]
332025-12-19T02:50:06.203ZINFOcrucible-agent (datafile): requesting running snapshot 2a67a846-11ff-4479-b5db-7d9e0ff60fd0-7f5ccddd-a253-4cbd-997e-3e2353f6af65 state: Requested
342025-12-19T02:50:06.203ZINFOcrucible-agent (worker): applying SMF actions for region 2a67a846-11ff-4479-b5db-7d9e0ff60fd0 running snapshot 7f5ccddd-a253-4cbd-997e-3e2353f6af65 (state Requested)...
352025-12-19T02:50:06.203ZINFOcrucible-agent (dropshot): request completed latency_us = 19151 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:41998 req_id = 8e35b623-eabc-4e71-afe5-3f434b50be79 response_code = 200 uri = /crucible/0/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0/snapshots/7f5ccddd-a253-4cbd-997e-3e2353f6af65/run
362025-12-19T02:50:06.208ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-2a67a846-11ff-4479-b5db-7d9e0ff60fd0-7f5ccddd-a253-4cbd-997e-3e2353f6af65
372025-12-19T02:50:06.215ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-2a67a846-11ff-4479-b5db-7d9e0ff60fd0-7f5ccddd-a253-4cbd-997e-3e2353f6af65
382025-12-19T02:50:06.226ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-2a67a846-11ff-4479-b5db-7d9e0ff60fd0-7f5ccddd-a253-4cbd-997e-3e2353f6af65
392025-12-19T02:50:06.226ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0/.zfs/snapshot/7f5ccddd-a253-4cbd-997e-3e2353f6af65
402025-12-19T02:50:06.226ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-12-19T02:50:06.226ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-12-19T02:50:06.226ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432025-12-19T02:50:06.226ZINFOcrucible-agent (worker): commit
442025-12-19T02:50:06.231ZINFOcrucible-agent (worker): ok!
452025-12-19T02:50:06.236ZINFOcrucible-agent (worker): SMF ok!
462025-12-19T02:50:06.236ZINFOcrucible-agent (datafile): region 2a67a846-11ff-4479-b5db-7d9e0ff60fd0 running snapshot 7f5ccddd-a253-4cbd-997e-3e2353f6af65 state: Requested -> Created
472025-12-19T02:50:06.366ZINFOcrucible-agent (datafile): path is "/data/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0"
482025-12-19T02:50:06.372ZINFOcrucible-agent (datafile): dataset is oxp_500b0602-f58b-4938-a6a6-82653c23b377/crucible/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0
492025-12-19T02:50:06.378ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_500b0602-f58b-4938-a6a6-82653c23b377/crucible/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0@7f5ccddd-a253-4cbd-997e-3e2353f6af65
502025-12-19T02:50:06.378ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_500b0602-f58b-4938-a6a6-82653c23b377/crucible/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0", "7f5ccddd-a253-4cbd-997e-3e2353f6af65"]
512025-12-19T02:50:06.384ZINFOcrucible-agent (dropshot): request completed latency_us = 17947 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:41998 req_id = 1e7b8f30-850c-4ced-9487-beff198829d0 response_code = 200 uri = /crucible/0/regions/2a67a846-11ff-4479-b5db-7d9e0ff60fd0/snapshots/7f5ccddd-a253-4cbd-997e-3e2353f6af65/run