Max level shown:
12025-03-25T01:30:39.284ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::c]:48539
22025-03-25T01:30:39.284ZINFOcrucible-agent (datafile): region bd05e82a-5608-4a0e-8bff-9eb4074ae6b8 state: Requested
32025-03-25T01:30:39.285ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-03-25T01:30:39.285ZINFOcrucible-agent (dropshot): request completed latency_us = 740 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:48539 req_id = 3159fc0a-0dd7-4168-848f-da70e2547c41 response_code = 200 uri = /crucible/0/regions
52025-03-25T01:30:39.298ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_d53e0b7b-ddc8-4a6a-beba-4645e24cf879/crucible/regions/bd05e82a-5608-4a0e-8bff-9eb4074ae6b8
62025-03-25T01:30:39.298ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_d53e0b7b-ddc8-4a6a-beba-4645e24cf879/crucible/regions/bd05e82a-5608-4a0e-8bff-9eb4074ae6b8
72025-03-25T01:30:39.362ZINFOcrucible-agent (worker): creating region Region { id: RegionId("bd05e82a-5608-4a0e-8bff-9eb4074ae6b8"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/bd05e82a-5608-4a0e-8bff-9eb4074ae6b8" region = bd05e82a-5608-4a0e-8bff-9eb4074ae6b8
82025-03-25T01:30:39.362ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "bd05e82a-5608-4a0e-8bff-9eb4074ae6b8" "--data" "/data/regions/bd05e82a-5608-4a0e-8bff-9eb4074ae6b8" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = bd05e82a-5608-4a0e-8bff-9eb4074ae6b8
92025-03-25T01:30:39.474ZINFOcrucible-agent (worker): region files created ok region = bd05e82a-5608-4a0e-8bff-9eb4074ae6b8
102025-03-25T01:30:39.474ZINFOcrucible-agent (datafile): region bd05e82a-5608-4a0e-8bff-9eb4074ae6b8 state: Requested -> Created
112025-03-25T01:30:39.474ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-03-25T01:30:39.475ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-bd05e82a-5608-4a0e-8bff-9eb4074ae6b8
132025-03-25T01:30:39.481ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-bd05e82a-5608-4a0e-8bff-9eb4074ae6b8
142025-03-25T01:30:39.484ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:48539 req_id = 7e516755-1116-47aa-97d9-29ee3bec616d response_code = 200 uri = /crucible/0/regions
152025-03-25T01:30:39.499ZINFOcrucible-agent (worker): creating config property group
162025-03-25T01:30:39.503ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-bd05e82a-5608-4a0e-8bff-9eb4074ae6b8
172025-03-25T01:30:39.503ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/bd05e82a-5608-4a0e-8bff-9eb4074ae6b8
182025-03-25T01:30:39.503ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-03-25T01:30:39.503ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
202025-03-25T01:30:39.503ZINFOcrucible-agent (worker): commit
212025-03-25T01:30:39.510ZINFOcrucible-agent (worker): ok!
222025-03-25T01:30:39.519ZINFOcrucible-agent (worker): SMF ok!
232025-03-25T01:30:41.574ZINFOcrucible-agent (dropshot): request completed latency_us = 3153 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:48539 req_id = bf9fd822-6fdd-44f5-8949-e5d06ec53fe5 response_code = 200 uri = /crucible/0/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242
242025-03-25T01:30:41.579ZINFOcrucible-agent (datafile): path is "/data/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242"
252025-03-25T01:30:41.588ZINFOcrucible-agent (datafile): dataset is oxp_d53e0b7b-ddc8-4a6a-beba-4645e24cf879/crucible/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242
262025-03-25T01:30:41.596ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d53e0b7b-ddc8-4a6a-beba-4645e24cf879/crucible/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242@66fb985c-9eba-4d11-93cb-897c0fc8f339
272025-03-25T01:30:41.596ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d53e0b7b-ddc8-4a6a-beba-4645e24cf879/crucible/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242", "66fb985c-9eba-4d11-93cb-897c0fc8f339"]
282025-03-25T01:30:41.605ZINFOcrucible-agent (dropshot): request completed latency_us = 28202 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:48539 req_id = 300e85b0-5bab-4403-a2c4-0786645732b4 response_code = 200 uri = /crucible/0/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242/snapshots/66fb985c-9eba-4d11-93cb-897c0fc8f339
292025-03-25T01:30:41.609ZINFOcrucible-agent (datafile): path is "/data/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242"
302025-03-25T01:30:41.617ZINFOcrucible-agent (datafile): dataset is oxp_d53e0b7b-ddc8-4a6a-beba-4645e24cf879/crucible/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242
312025-03-25T01:30:41.625ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d53e0b7b-ddc8-4a6a-beba-4645e24cf879/crucible/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242@66fb985c-9eba-4d11-93cb-897c0fc8f339
322025-03-25T01:30:41.625ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d53e0b7b-ddc8-4a6a-beba-4645e24cf879/crucible/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242", "66fb985c-9eba-4d11-93cb-897c0fc8f339"]
332025-03-25T01:30:41.633ZINFOcrucible-agent (datafile): requesting running snapshot e2c28b59-f235-4a2d-ad6c-abda2d72a242-66fb985c-9eba-4d11-93cb-897c0fc8f339 state: Requested
342025-03-25T01:30:41.633ZINFOcrucible-agent (worker): applying SMF actions for region e2c28b59-f235-4a2d-ad6c-abda2d72a242 running snapshot 66fb985c-9eba-4d11-93cb-897c0fc8f339 (state Requested)...
352025-03-25T01:30:41.634ZINFOcrucible-agent (dropshot): request completed latency_us = 24387 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:48539 req_id = fa3de049-8fbd-46b7-8274-b2cd6102d1e5 response_code = 200 uri = /crucible/0/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242/snapshots/66fb985c-9eba-4d11-93cb-897c0fc8f339/run
362025-03-25T01:30:41.636ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-e2c28b59-f235-4a2d-ad6c-abda2d72a242-66fb985c-9eba-4d11-93cb-897c0fc8f339
372025-03-25T01:30:41.644ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-e2c28b59-f235-4a2d-ad6c-abda2d72a242-66fb985c-9eba-4d11-93cb-897c0fc8f339
382025-03-25T01:30:41.661ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-e2c28b59-f235-4a2d-ad6c-abda2d72a242-66fb985c-9eba-4d11-93cb-897c0fc8f339
392025-03-25T01:30:41.662ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242/.zfs/snapshot/66fb985c-9eba-4d11-93cb-897c0fc8f339
402025-03-25T01:30:41.662ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-03-25T01:30:41.662ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-03-25T01:30:41.662ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
432025-03-25T01:30:41.662ZINFOcrucible-agent (worker): commit
442025-03-25T01:30:41.668ZINFOcrucible-agent (worker): ok!
452025-03-25T01:30:41.674ZINFOcrucible-agent (worker): SMF ok!
462025-03-25T01:30:41.674ZINFOcrucible-agent (datafile): region e2c28b59-f235-4a2d-ad6c-abda2d72a242 running snapshot 66fb985c-9eba-4d11-93cb-897c0fc8f339 state: Requested -> Created
472025-03-25T01:30:41.825ZINFOcrucible-agent (datafile): path is "/data/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242"
482025-03-25T01:30:41.835ZINFOcrucible-agent (datafile): dataset is oxp_d53e0b7b-ddc8-4a6a-beba-4645e24cf879/crucible/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242
492025-03-25T01:30:41.843ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d53e0b7b-ddc8-4a6a-beba-4645e24cf879/crucible/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242@66fb985c-9eba-4d11-93cb-897c0fc8f339
502025-03-25T01:30:41.843ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d53e0b7b-ddc8-4a6a-beba-4645e24cf879/crucible/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242", "66fb985c-9eba-4d11-93cb-897c0fc8f339"]
512025-03-25T01:30:41.850ZINFOcrucible-agent (dropshot): request completed latency_us = 24630 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:48539 req_id = 9ad1660d-6d4e-4847-b686-87227196a93e response_code = 200 uri = /crucible/0/regions/e2c28b59-f235-4a2d-ad6c-abda2d72a242/snapshots/66fb985c-9eba-4d11-93cb-897c0fc8f339/run