Max level shown:
12025-12-23T18:24:55.877ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:34003
22025-12-23T18:24:55.877ZINFOcrucible-agent (datafile): region 39388aaf-6fdc-4c21-8150-fe09971eb2ad state: Requested
32025-12-23T18:24:55.878ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-12-23T18:24:55.878ZINFOcrucible-agent (dropshot): request completed latency_us = 1108 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34003 req_id = 7b8f525c-fbda-4ddb-a438-aac37e8ef22b response_code = 200 uri = /crucible/0/regions
52025-12-23T18:24:55.887ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_41ca4dbe-008f-40f6-b715-38966cfaa8ce/crucible/regions/39388aaf-6fdc-4c21-8150-fe09971eb2ad
62025-12-23T18:24:55.887ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_41ca4dbe-008f-40f6-b715-38966cfaa8ce/crucible/regions/39388aaf-6fdc-4c21-8150-fe09971eb2ad
72025-12-23T18:24:55.937ZINFOcrucible-agent (worker): creating region Region { id: RegionId("39388aaf-6fdc-4c21-8150-fe09971eb2ad"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/39388aaf-6fdc-4c21-8150-fe09971eb2ad" region = 39388aaf-6fdc-4c21-8150-fe09971eb2ad
82025-12-23T18:24:55.937ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "39388aaf-6fdc-4c21-8150-fe09971eb2ad" "--data" "/data/regions/39388aaf-6fdc-4c21-8150-fe09971eb2ad" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 39388aaf-6fdc-4c21-8150-fe09971eb2ad
92025-12-23T18:24:56.061ZINFOcrucible-agent (worker): region files created ok region = 39388aaf-6fdc-4c21-8150-fe09971eb2ad
102025-12-23T18:24:56.061ZINFOcrucible-agent (datafile): region 39388aaf-6fdc-4c21-8150-fe09971eb2ad state: Requested -> Created
112025-12-23T18:24:56.062ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-12-23T18:24:56.062ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-39388aaf-6fdc-4c21-8150-fe09971eb2ad
132025-12-23T18:24:56.072ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-39388aaf-6fdc-4c21-8150-fe09971eb2ad
142025-12-23T18:24:56.084ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34003 req_id = 8ca5cc7d-482a-4263-badd-a109086e13e9 response_code = 200 uri = /crucible/0/regions
152025-12-23T18:24:56.092ZINFOcrucible-agent (worker): creating config property group
162025-12-23T18:24:56.096ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-39388aaf-6fdc-4c21-8150-fe09971eb2ad
172025-12-23T18:24:56.096ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/39388aaf-6fdc-4c21-8150-fe09971eb2ad
182025-12-23T18:24:56.096ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-12-23T18:24:56.096ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
202025-12-23T18:24:56.096ZINFOcrucible-agent (worker): commit
212025-12-23T18:24:56.101ZINFOcrucible-agent (worker): ok!
222025-12-23T18:24:56.119ZINFOcrucible-agent (worker): SMF ok!
232025-12-23T18:24:57.029ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:34003 req_id = c2a03879-0d8e-428d-a64f-5a88d285b740 response_code = 200 uri = /crucible/0/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8
242025-12-23T18:24:57.035ZINFOcrucible-agent (datafile): path is "/data/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8"
252025-12-23T18:24:57.041ZINFOcrucible-agent (datafile): dataset is oxp_41ca4dbe-008f-40f6-b715-38966cfaa8ce/crucible/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8
262025-12-23T18:24:57.048ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_41ca4dbe-008f-40f6-b715-38966cfaa8ce/crucible/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8@179df76f-1165-411a-94d9-e0da4df54d1e
272025-12-23T18:24:57.048ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_41ca4dbe-008f-40f6-b715-38966cfaa8ce/crucible/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8", "179df76f-1165-411a-94d9-e0da4df54d1e"]
282025-12-23T18:24:57.056ZINFOcrucible-agent (dropshot): request completed latency_us = 20554 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:34003 req_id = a9cb04bc-edd0-41c4-b76b-7398d6174b80 response_code = 200 uri = /crucible/0/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8/snapshots/179df76f-1165-411a-94d9-e0da4df54d1e
292025-12-23T18:24:57.068ZINFOcrucible-agent (datafile): path is "/data/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8"
302025-12-23T18:24:57.075ZINFOcrucible-agent (datafile): dataset is oxp_41ca4dbe-008f-40f6-b715-38966cfaa8ce/crucible/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8
312025-12-23T18:24:57.081ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_41ca4dbe-008f-40f6-b715-38966cfaa8ce/crucible/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8@179df76f-1165-411a-94d9-e0da4df54d1e
322025-12-23T18:24:57.081ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_41ca4dbe-008f-40f6-b715-38966cfaa8ce/crucible/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8", "179df76f-1165-411a-94d9-e0da4df54d1e"]
332025-12-23T18:24:57.088ZINFOcrucible-agent (datafile): requesting running snapshot fc424d5b-62d8-4ab1-91c1-24be030983d8-179df76f-1165-411a-94d9-e0da4df54d1e state: Requested
342025-12-23T18:24:57.088ZINFOcrucible-agent (dropshot): request completed latency_us = 19895 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34003 req_id = 89ea3c89-2406-4f7a-8edc-34de9f416b85 response_code = 200 uri = /crucible/0/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8/snapshots/179df76f-1165-411a-94d9-e0da4df54d1e/run
352025-12-23T18:24:57.088ZINFOcrucible-agent (worker): applying SMF actions for region fc424d5b-62d8-4ab1-91c1-24be030983d8 running snapshot 179df76f-1165-411a-94d9-e0da4df54d1e (state Requested)...
362025-12-23T18:24:57.092ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-fc424d5b-62d8-4ab1-91c1-24be030983d8-179df76f-1165-411a-94d9-e0da4df54d1e
372025-12-23T18:24:57.100ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-fc424d5b-62d8-4ab1-91c1-24be030983d8-179df76f-1165-411a-94d9-e0da4df54d1e
382025-12-23T18:24:57.111ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-fc424d5b-62d8-4ab1-91c1-24be030983d8-179df76f-1165-411a-94d9-e0da4df54d1e
392025-12-23T18:24:57.112ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8/.zfs/snapshot/179df76f-1165-411a-94d9-e0da4df54d1e
402025-12-23T18:24:57.112ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-12-23T18:24:57.112ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-12-23T18:24:57.112ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432025-12-23T18:24:57.112ZINFOcrucible-agent (worker): commit
442025-12-23T18:24:57.116ZINFOcrucible-agent (worker): ok!
452025-12-23T18:24:57.120ZINFOcrucible-agent (worker): SMF ok!
462025-12-23T18:24:57.120ZINFOcrucible-agent (datafile): region fc424d5b-62d8-4ab1-91c1-24be030983d8 running snapshot 179df76f-1165-411a-94d9-e0da4df54d1e state: Requested -> Created
472025-12-23T18:24:57.237ZINFOcrucible-agent (datafile): path is "/data/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8"
482025-12-23T18:24:57.244ZINFOcrucible-agent (datafile): dataset is oxp_41ca4dbe-008f-40f6-b715-38966cfaa8ce/crucible/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8
492025-12-23T18:24:57.251ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_41ca4dbe-008f-40f6-b715-38966cfaa8ce/crucible/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8@179df76f-1165-411a-94d9-e0da4df54d1e
502025-12-23T18:24:57.251ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_41ca4dbe-008f-40f6-b715-38966cfaa8ce/crucible/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8", "179df76f-1165-411a-94d9-e0da4df54d1e"]
512025-12-23T18:24:57.257ZINFOcrucible-agent (dropshot): request completed latency_us = 19356 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34003 req_id = 3b2cd2c0-1ca4-426c-81f1-978f1651bb93 response_code = 200 uri = /crucible/0/regions/fc424d5b-62d8-4ab1-91c1-24be030983d8/snapshots/179df76f-1165-411a-94d9-e0da4df54d1e/run