Max level shown:
12025-03-25T22:40:06.251ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::c]:50165
22025-03-25T22:40:06.251ZINFOcrucible-agent (datafile): region fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb state: Requested
32025-03-25T22:40:06.252ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-03-25T22:40:06.252ZINFOcrucible-agent (dropshot): request completed latency_us = 656 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:50165 req_id = e71c6eed-80d5-4686-ad73-8ef85547e895 response_code = 200 uri = /crucible/0/regions
52025-03-25T22:40:06.265ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_db2f7830-be20-44d7-ae1f-5a4724d29a8b/crucible/regions/fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb
62025-03-25T22:40:06.265ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_db2f7830-be20-44d7-ae1f-5a4724d29a8b/crucible/regions/fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb
72025-03-25T22:40:06.327ZINFOcrucible-agent (worker): creating region Region { id: RegionId("fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb" region = fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb
82025-03-25T22:40:06.327ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb" "--data" "/data/regions/fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb
92025-03-25T22:40:06.429ZINFOcrucible-agent (worker): region files created ok region = fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb
102025-03-25T22:40:06.429ZINFOcrucible-agent (datafile): region fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb state: Requested -> Created
112025-03-25T22:40:06.430ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-03-25T22:40:06.436ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb
132025-03-25T22:40:06.449ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb
142025-03-25T22:40:06.458ZINFOcrucible-agent (worker): creating config property group
152025-03-25T22:40:06.463ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb
162025-03-25T22:40:06.463ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fd2f5f5f-2c48-4a84-baf6-6984cee2d6fb
172025-03-25T22:40:06.464ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-03-25T22:40:06.464ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
192025-03-25T22:40:06.464ZINFOcrucible-agent (worker): commit
202025-03-25T22:40:06.469ZINFOcrucible-agent (worker): ok!
212025-03-25T22:40:06.475ZINFOcrucible-agent (worker): SMF ok!
222025-03-25T22:40:06.575ZINFOcrucible-agent (dropshot): request completed latency_us = 76 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:50165 req_id = d7c6f287-72c7-4280-89cb-d82cb496f815 response_code = 200 uri = /crucible/0/regions
232025-03-25T22:40:08.031ZINFOcrucible-agent (dropshot): request completed latency_us = 46 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:50165 req_id = 0b34bc53-d858-4473-9af1-bbc71b82250b response_code = 200 uri = /crucible/0/regions/e7274c9e-8434-4084-8eea-7823f3327499
242025-03-25T22:40:08.038ZINFOcrucible-agent (datafile): path is "/data/regions/e7274c9e-8434-4084-8eea-7823f3327499"
252025-03-25T22:40:08.047ZINFOcrucible-agent (datafile): dataset is oxp_db2f7830-be20-44d7-ae1f-5a4724d29a8b/crucible/regions/e7274c9e-8434-4084-8eea-7823f3327499
262025-03-25T22:40:08.056ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_db2f7830-be20-44d7-ae1f-5a4724d29a8b/crucible/regions/e7274c9e-8434-4084-8eea-7823f3327499@52750cf1-0fce-4409-90e4-2bbf33898c96
272025-03-25T22:40:08.056ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_db2f7830-be20-44d7-ae1f-5a4724d29a8b/crucible/regions/e7274c9e-8434-4084-8eea-7823f3327499", "52750cf1-0fce-4409-90e4-2bbf33898c96"]
282025-03-25T22:40:08.067ZINFOcrucible-agent (dropshot): request completed latency_us = 32753 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:50165 req_id = 5c5cd240-7fb3-4462-b39d-302c97e9bb0f response_code = 200 uri = /crucible/0/regions/e7274c9e-8434-4084-8eea-7823f3327499/snapshots/52750cf1-0fce-4409-90e4-2bbf33898c96
292025-03-25T22:40:08.071ZINFOcrucible-agent (datafile): path is "/data/regions/e7274c9e-8434-4084-8eea-7823f3327499"
302025-03-25T22:40:08.079ZINFOcrucible-agent (datafile): dataset is oxp_db2f7830-be20-44d7-ae1f-5a4724d29a8b/crucible/regions/e7274c9e-8434-4084-8eea-7823f3327499
312025-03-25T22:40:08.088ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_db2f7830-be20-44d7-ae1f-5a4724d29a8b/crucible/regions/e7274c9e-8434-4084-8eea-7823f3327499@52750cf1-0fce-4409-90e4-2bbf33898c96
322025-03-25T22:40:08.088ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_db2f7830-be20-44d7-ae1f-5a4724d29a8b/crucible/regions/e7274c9e-8434-4084-8eea-7823f3327499", "52750cf1-0fce-4409-90e4-2bbf33898c96"]
332025-03-25T22:40:08.098ZINFOcrucible-agent (datafile): requesting running snapshot e7274c9e-8434-4084-8eea-7823f3327499-52750cf1-0fce-4409-90e4-2bbf33898c96 state: Requested
342025-03-25T22:40:08.098ZINFOcrucible-agent (worker): applying SMF actions for region e7274c9e-8434-4084-8eea-7823f3327499 running snapshot 52750cf1-0fce-4409-90e4-2bbf33898c96 (state Requested)...
352025-03-25T22:40:08.098ZINFOcrucible-agent (dropshot): request completed latency_us = 27140 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:50165 req_id = f5f91bdb-52dc-4a69-97ae-085419d30960 response_code = 200 uri = /crucible/0/regions/e7274c9e-8434-4084-8eea-7823f3327499/snapshots/52750cf1-0fce-4409-90e4-2bbf33898c96/run
362025-03-25T22:40:08.101ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-e7274c9e-8434-4084-8eea-7823f3327499-52750cf1-0fce-4409-90e4-2bbf33898c96
372025-03-25T22:40:08.103ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-e7274c9e-8434-4084-8eea-7823f3327499-52750cf1-0fce-4409-90e4-2bbf33898c96
382025-03-25T22:40:08.113ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-e7274c9e-8434-4084-8eea-7823f3327499-52750cf1-0fce-4409-90e4-2bbf33898c96
392025-03-25T22:40:08.113ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/e7274c9e-8434-4084-8eea-7823f3327499/.zfs/snapshot/52750cf1-0fce-4409-90e4-2bbf33898c96
402025-03-25T22:40:08.113ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-03-25T22:40:08.114ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-03-25T22:40:08.114ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
432025-03-25T22:40:08.114ZINFOcrucible-agent (worker): commit
442025-03-25T22:40:08.117ZINFOcrucible-agent (worker): ok!
452025-03-25T22:40:08.122ZINFOcrucible-agent (worker): SMF ok!
462025-03-25T22:40:08.122ZINFOcrucible-agent (datafile): region e7274c9e-8434-4084-8eea-7823f3327499 running snapshot 52750cf1-0fce-4409-90e4-2bbf33898c96 state: Requested -> Created
472025-03-25T22:40:08.429ZINFOcrucible-agent (datafile): path is "/data/regions/e7274c9e-8434-4084-8eea-7823f3327499"
482025-03-25T22:40:08.438ZINFOcrucible-agent (datafile): dataset is oxp_db2f7830-be20-44d7-ae1f-5a4724d29a8b/crucible/regions/e7274c9e-8434-4084-8eea-7823f3327499
492025-03-25T22:40:08.446ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_db2f7830-be20-44d7-ae1f-5a4724d29a8b/crucible/regions/e7274c9e-8434-4084-8eea-7823f3327499@52750cf1-0fce-4409-90e4-2bbf33898c96
502025-03-25T22:40:08.446ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_db2f7830-be20-44d7-ae1f-5a4724d29a8b/crucible/regions/e7274c9e-8434-4084-8eea-7823f3327499", "52750cf1-0fce-4409-90e4-2bbf33898c96"]
512025-03-25T22:40:08.455ZINFOcrucible-agent (dropshot): request completed latency_us = 26015 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:50165 req_id = 491391e6-6142-4841-9f68-97c34afb4df6 response_code = 200 uri = /crucible/0/regions/e7274c9e-8434-4084-8eea-7823f3327499/snapshots/52750cf1-0fce-4409-90e4-2bbf33898c96/run