Max level shown:
12025-05-15T04:23:45.453ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::c]:63205
22025-05-15T04:23:45.453ZINFOcrucible-agent (datafile): region dc2c2000-0ec8-42ce-9828-17ad062cdb3c state: Requested
32025-05-15T04:23:45.454ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-05-15T04:23:45.454ZINFOcrucible-agent (dropshot): request completed latency_us = 634 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63205 req_id = c7f7313f-7989-41a6-a112-03116d77dfb9 response_code = 200 uri = /crucible/0/regions
52025-05-15T04:23:45.467ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_f9de2043-47b2-4be8-ac5e-48b3cba8acce/crucible/regions/dc2c2000-0ec8-42ce-9828-17ad062cdb3c
62025-05-15T04:23:45.467ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_f9de2043-47b2-4be8-ac5e-48b3cba8acce/crucible/regions/dc2c2000-0ec8-42ce-9828-17ad062cdb3c
72025-05-15T04:23:45.542ZINFOcrucible-agent (worker): creating region Region { id: RegionId("dc2c2000-0ec8-42ce-9828-17ad062cdb3c"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/dc2c2000-0ec8-42ce-9828-17ad062cdb3c" region = dc2c2000-0ec8-42ce-9828-17ad062cdb3c
82025-05-15T04:23:45.542ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "dc2c2000-0ec8-42ce-9828-17ad062cdb3c" "--data" "/data/regions/dc2c2000-0ec8-42ce-9828-17ad062cdb3c" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = dc2c2000-0ec8-42ce-9828-17ad062cdb3c
92025-05-15T04:23:45.663ZINFOcrucible-agent (dropshot): request completed latency_us = 135 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63205 req_id = adc55cc5-b7cb-44ba-b4a2-45d324284817 response_code = 200 uri = /crucible/0/regions
102025-05-15T04:23:45.683ZINFOcrucible-agent (worker): region files created ok region = dc2c2000-0ec8-42ce-9828-17ad062cdb3c
112025-05-15T04:23:45.683ZINFOcrucible-agent (datafile): region dc2c2000-0ec8-42ce-9828-17ad062cdb3c state: Requested -> Created
122025-05-15T04:23:45.683ZINFOcrucible-agent (worker): applying SMF actions post create...
132025-05-15T04:23:45.702ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-dc2c2000-0ec8-42ce-9828-17ad062cdb3c
142025-05-15T04:23:45.722ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-dc2c2000-0ec8-42ce-9828-17ad062cdb3c
152025-05-15T04:23:45.745ZINFOcrucible-agent (worker): creating config property group
162025-05-15T04:23:45.763ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-dc2c2000-0ec8-42ce-9828-17ad062cdb3c
172025-05-15T04:23:45.763ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/dc2c2000-0ec8-42ce-9828-17ad062cdb3c
182025-05-15T04:23:45.763ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-05-15T04:23:45.763ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
202025-05-15T04:23:45.763ZINFOcrucible-agent (worker): commit
212025-05-15T04:23:45.771ZINFOcrucible-agent (worker): ok!
222025-05-15T04:23:45.778ZINFOcrucible-agent (worker): SMF ok!
232025-05-15T04:23:45.999ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63205 req_id = 8668b887-e752-4c1d-a4be-ead77919e1b8 response_code = 200 uri = /crucible/0/regions
242025-05-15T04:23:46.857ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:63205 req_id = 5e61cda8-cf02-4578-a684-f5cda7493398 response_code = 200 uri = /crucible/0/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4
252025-05-15T04:23:46.859ZINFOcrucible-agent (datafile): path is "/data/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4"
262025-05-15T04:23:46.869ZINFOcrucible-agent (datafile): dataset is oxp_f9de2043-47b2-4be8-ac5e-48b3cba8acce/crucible/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4
272025-05-15T04:23:46.878ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f9de2043-47b2-4be8-ac5e-48b3cba8acce/crucible/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4@e2d1acd3-04b2-43d5-803f-142b0cba2a9e
282025-05-15T04:23:46.878ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f9de2043-47b2-4be8-ac5e-48b3cba8acce/crucible/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4", "e2d1acd3-04b2-43d5-803f-142b0cba2a9e"]
292025-05-15T04:23:46.890ZINFOcrucible-agent (dropshot): request completed latency_us = 30776 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:63205 req_id = de4f6e53-129e-4167-bc60-0b52ef9dd90b response_code = 200 uri = /crucible/0/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4/snapshots/e2d1acd3-04b2-43d5-803f-142b0cba2a9e
302025-05-15T04:23:46.894ZINFOcrucible-agent (datafile): path is "/data/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4"
312025-05-15T04:23:46.902ZINFOcrucible-agent (datafile): dataset is oxp_f9de2043-47b2-4be8-ac5e-48b3cba8acce/crucible/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4
322025-05-15T04:23:46.910ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f9de2043-47b2-4be8-ac5e-48b3cba8acce/crucible/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4@e2d1acd3-04b2-43d5-803f-142b0cba2a9e
332025-05-15T04:23:46.910ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f9de2043-47b2-4be8-ac5e-48b3cba8acce/crucible/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4", "e2d1acd3-04b2-43d5-803f-142b0cba2a9e"]
342025-05-15T04:23:46.920ZINFOcrucible-agent (datafile): requesting running snapshot 51aea369-a278-45d5-bb03-80a11e3d8ec4-e2d1acd3-04b2-43d5-803f-142b0cba2a9e state: Requested
352025-05-15T04:23:46.920ZINFOcrucible-agent (worker): applying SMF actions for region 51aea369-a278-45d5-bb03-80a11e3d8ec4 running snapshot e2d1acd3-04b2-43d5-803f-142b0cba2a9e (state Requested)...
362025-05-15T04:23:46.920ZINFOcrucible-agent (dropshot): request completed latency_us = 26836 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63205 req_id = 0f8c8a70-4b11-4aed-9b51-22b919ad27b2 response_code = 200 uri = /crucible/0/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4/snapshots/e2d1acd3-04b2-43d5-803f-142b0cba2a9e/run
372025-05-15T04:23:46.923ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-51aea369-a278-45d5-bb03-80a11e3d8ec4-e2d1acd3-04b2-43d5-803f-142b0cba2a9e
382025-05-15T04:23:46.927ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-51aea369-a278-45d5-bb03-80a11e3d8ec4-e2d1acd3-04b2-43d5-803f-142b0cba2a9e
392025-05-15T04:23:46.942ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-51aea369-a278-45d5-bb03-80a11e3d8ec4-e2d1acd3-04b2-43d5-803f-142b0cba2a9e
402025-05-15T04:23:46.943ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4/.zfs/snapshot/e2d1acd3-04b2-43d5-803f-142b0cba2a9e
412025-05-15T04:23:46.943ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
422025-05-15T04:23:46.943ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
432025-05-15T04:23:46.943ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
442025-05-15T04:23:46.943ZINFOcrucible-agent (worker): commit
452025-05-15T04:23:46.949ZINFOcrucible-agent (worker): ok!
462025-05-15T04:23:46.955ZINFOcrucible-agent (worker): SMF ok!
472025-05-15T04:23:46.955ZINFOcrucible-agent (datafile): region 51aea369-a278-45d5-bb03-80a11e3d8ec4 running snapshot e2d1acd3-04b2-43d5-803f-142b0cba2a9e state: Requested -> Created
482025-05-15T04:23:47.229ZINFOcrucible-agent (datafile): path is "/data/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4"
492025-05-15T04:23:47.237ZINFOcrucible-agent (datafile): dataset is oxp_f9de2043-47b2-4be8-ac5e-48b3cba8acce/crucible/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4
502025-05-15T04:23:47.245ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f9de2043-47b2-4be8-ac5e-48b3cba8acce/crucible/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4@e2d1acd3-04b2-43d5-803f-142b0cba2a9e
512025-05-15T04:23:47.245ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f9de2043-47b2-4be8-ac5e-48b3cba8acce/crucible/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4", "e2d1acd3-04b2-43d5-803f-142b0cba2a9e"]
522025-05-15T04:23:47.254ZINFOcrucible-agent (dropshot): request completed latency_us = 24992 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63205 req_id = dce64b56-5849-4ae7-bf31-90619fe2d40d response_code = 200 uri = /crucible/0/regions/51aea369-a278-45d5-bb03-80a11e3d8ec4/snapshots/e2d1acd3-04b2-43d5-803f-142b0cba2a9e/run