Max level shown:
12025-11-12T23:16:27.565ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::b]:43495
22025-11-12T23:16:27.565ZINFOcrucible-agent (datafile): region 9a93491d-e4e4-4e96-ba15-eb7f0c432f47 state: Requested
32025-11-12T23:16:27.565ZINFOcrucible-agent (dropshot): request completed latency_us = 258 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43495 req_id = 08dd87cb-e4fa-4f4e-9c5d-b347997d3cca response_code = 200 uri = /crucible/0/regions
42025-11-12T23:16:27.565ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52025-11-12T23:16:27.572ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_a3465915-c84b-4fd1-a0ba-e7fb55dac738/crucible/regions/9a93491d-e4e4-4e96-ba15-eb7f0c432f47
62025-11-12T23:16:27.572ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_a3465915-c84b-4fd1-a0ba-e7fb55dac738/crucible/regions/9a93491d-e4e4-4e96-ba15-eb7f0c432f47
72025-11-12T23:16:27.626ZINFOcrucible-agent (worker): creating region Region { id: RegionId("9a93491d-e4e4-4e96-ba15-eb7f0c432f47"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/9a93491d-e4e4-4e96-ba15-eb7f0c432f47" region = 9a93491d-e4e4-4e96-ba15-eb7f0c432f47
82025-11-12T23:16:27.626ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "9a93491d-e4e4-4e96-ba15-eb7f0c432f47" "--data" "/data/regions/9a93491d-e4e4-4e96-ba15-eb7f0c432f47" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 9a93491d-e4e4-4e96-ba15-eb7f0c432f47
92025-11-12T23:16:27.704ZINFOcrucible-agent (worker): region files created ok region = 9a93491d-e4e4-4e96-ba15-eb7f0c432f47
102025-11-12T23:16:27.704ZINFOcrucible-agent (datafile): region 9a93491d-e4e4-4e96-ba15-eb7f0c432f47 state: Requested -> Created
112025-11-12T23:16:27.704ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-11-12T23:16:27.720ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-9a93491d-e4e4-4e96-ba15-eb7f0c432f47
132025-11-12T23:16:27.730ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-9a93491d-e4e4-4e96-ba15-eb7f0c432f47
142025-11-12T23:16:27.743ZINFOcrucible-agent (worker): creating config property group
152025-11-12T23:16:27.746ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-9a93491d-e4e4-4e96-ba15-eb7f0c432f47
162025-11-12T23:16:27.746ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/9a93491d-e4e4-4e96-ba15-eb7f0c432f47
172025-11-12T23:16:27.746ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-11-12T23:16:27.746ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
192025-11-12T23:16:27.746ZINFOcrucible-agent (worker): commit
202025-11-12T23:16:27.749ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43495 req_id = 80112611-4458-4d43-90bf-f1dc47f9295e response_code = 200 uri = /crucible/0/regions
212025-11-12T23:16:27.750ZINFOcrucible-agent (worker): ok!
222025-11-12T23:16:27.754ZINFOcrucible-agent (worker): SMF ok!
232025-11-12T23:16:29.745ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43495 req_id = 400c14f8-916c-4b9e-9518-250c5ce9fbcb response_code = 200 uri = /crucible/0/regions/293270cf-9cf5-4218-afbc-542317af450d
242025-11-12T23:16:29.748ZINFOcrucible-agent (datafile): path is "/data/regions/293270cf-9cf5-4218-afbc-542317af450d"
252025-11-12T23:16:29.754ZINFOcrucible-agent (datafile): dataset is oxp_a3465915-c84b-4fd1-a0ba-e7fb55dac738/crucible/regions/293270cf-9cf5-4218-afbc-542317af450d
262025-11-12T23:16:29.759ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a3465915-c84b-4fd1-a0ba-e7fb55dac738/crucible/regions/293270cf-9cf5-4218-afbc-542317af450d@9139e272-d5a4-417a-9f57-cc036b321bee
272025-11-12T23:16:29.759ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a3465915-c84b-4fd1-a0ba-e7fb55dac738/crucible/regions/293270cf-9cf5-4218-afbc-542317af450d", "9139e272-d5a4-417a-9f57-cc036b321bee"]
282025-11-12T23:16:29.768ZINFOcrucible-agent (dropshot): request completed latency_us = 20378 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43495 req_id = b11ca98a-decc-457d-b809-cb861750a340 response_code = 200 uri = /crucible/0/regions/293270cf-9cf5-4218-afbc-542317af450d/snapshots/9139e272-d5a4-417a-9f57-cc036b321bee
292025-11-12T23:16:29.772ZINFOcrucible-agent (datafile): path is "/data/regions/293270cf-9cf5-4218-afbc-542317af450d"
302025-11-12T23:16:29.777ZINFOcrucible-agent (datafile): dataset is oxp_a3465915-c84b-4fd1-a0ba-e7fb55dac738/crucible/regions/293270cf-9cf5-4218-afbc-542317af450d
312025-11-12T23:16:29.781ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a3465915-c84b-4fd1-a0ba-e7fb55dac738/crucible/regions/293270cf-9cf5-4218-afbc-542317af450d@9139e272-d5a4-417a-9f57-cc036b321bee
322025-11-12T23:16:29.781ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a3465915-c84b-4fd1-a0ba-e7fb55dac738/crucible/regions/293270cf-9cf5-4218-afbc-542317af450d", "9139e272-d5a4-417a-9f57-cc036b321bee"]
332025-11-12T23:16:29.788ZINFOcrucible-agent (datafile): requesting running snapshot 293270cf-9cf5-4218-afbc-542317af450d-9139e272-d5a4-417a-9f57-cc036b321bee state: Requested
342025-11-12T23:16:29.788ZINFOcrucible-agent (dropshot): request completed latency_us = 16543 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43495 req_id = 6226cb06-302a-4560-9d17-c7cba51d33cd response_code = 200 uri = /crucible/0/regions/293270cf-9cf5-4218-afbc-542317af450d/snapshots/9139e272-d5a4-417a-9f57-cc036b321bee/run
352025-11-12T23:16:29.788ZINFOcrucible-agent (worker): applying SMF actions for region 293270cf-9cf5-4218-afbc-542317af450d running snapshot 9139e272-d5a4-417a-9f57-cc036b321bee (state Requested)...
362025-11-12T23:16:29.791ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-293270cf-9cf5-4218-afbc-542317af450d-9139e272-d5a4-417a-9f57-cc036b321bee
372025-11-12T23:16:29.794ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-293270cf-9cf5-4218-afbc-542317af450d-9139e272-d5a4-417a-9f57-cc036b321bee
382025-11-12T23:16:29.803ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-293270cf-9cf5-4218-afbc-542317af450d-9139e272-d5a4-417a-9f57-cc036b321bee
392025-11-12T23:16:29.803ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/293270cf-9cf5-4218-afbc-542317af450d/.zfs/snapshot/9139e272-d5a4-417a-9f57-cc036b321bee
402025-11-12T23:16:29.803ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-11-12T23:16:29.803ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-11-12T23:16:29.803ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
432025-11-12T23:16:29.803ZINFOcrucible-agent (worker): commit
442025-11-12T23:16:29.807ZINFOcrucible-agent (worker): ok!
452025-11-12T23:16:29.811ZINFOcrucible-agent (worker): SMF ok!
462025-11-12T23:16:29.811ZINFOcrucible-agent (datafile): region 293270cf-9cf5-4218-afbc-542317af450d running snapshot 9139e272-d5a4-417a-9f57-cc036b321bee state: Requested -> Created
472025-11-12T23:16:30.109ZINFOcrucible-agent (datafile): path is "/data/regions/293270cf-9cf5-4218-afbc-542317af450d"
482025-11-12T23:16:30.115ZINFOcrucible-agent (datafile): dataset is oxp_a3465915-c84b-4fd1-a0ba-e7fb55dac738/crucible/regions/293270cf-9cf5-4218-afbc-542317af450d
492025-11-12T23:16:30.122ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a3465915-c84b-4fd1-a0ba-e7fb55dac738/crucible/regions/293270cf-9cf5-4218-afbc-542317af450d@9139e272-d5a4-417a-9f57-cc036b321bee
502025-11-12T23:16:30.122ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a3465915-c84b-4fd1-a0ba-e7fb55dac738/crucible/regions/293270cf-9cf5-4218-afbc-542317af450d", "9139e272-d5a4-417a-9f57-cc036b321bee"]
512025-11-12T23:16:30.127ZINFOcrucible-agent (dropshot): request completed latency_us = 17603 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43495 req_id = fbe2d3cd-84df-464b-8c79-0d97189a6216 response_code = 200 uri = /crucible/0/regions/293270cf-9cf5-4218-afbc-542317af450d/snapshots/9139e272-d5a4-417a-9f57-cc036b321bee/run