Max level shown:
12025-01-17T07:39:56.604ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::b]:38640
22025-01-17T07:39:56.604ZINFOcrucible-agent (datafile): region 27fb74e9-c403-406a-97a0-a2af0c26ed91 state: Requested
32025-01-17T07:39:56.605ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-01-17T07:39:56.605ZINFOcrucible-agent (dropshot): request completed latency_us = 905 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38640 req_id = 50db1ca6-3702-47e1-b0c2-06e8bd4f9c97 response_code = 200 uri = /crucible/0/regions
52025-01-17T07:39:56.618ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_b5a87517-097c-4a2f-afbf-79d6c9f09841/crucible/regions/27fb74e9-c403-406a-97a0-a2af0c26ed91
62025-01-17T07:39:56.618ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_b5a87517-097c-4a2f-afbf-79d6c9f09841/crucible/regions/27fb74e9-c403-406a-97a0-a2af0c26ed91
72025-01-17T07:39:56.679ZINFOcrucible-agent (worker): creating region Region { id: RegionId("27fb74e9-c403-406a-97a0-a2af0c26ed91"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/27fb74e9-c403-406a-97a0-a2af0c26ed91" region = 27fb74e9-c403-406a-97a0-a2af0c26ed91
82025-01-17T07:39:56.679ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "27fb74e9-c403-406a-97a0-a2af0c26ed91" "--data" "/data/regions/27fb74e9-c403-406a-97a0-a2af0c26ed91" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 27fb74e9-c403-406a-97a0-a2af0c26ed91
92025-01-17T07:39:56.756ZINFOcrucible-agent (worker): region files created ok region = 27fb74e9-c403-406a-97a0-a2af0c26ed91
102025-01-17T07:39:56.757ZINFOcrucible-agent (datafile): region 27fb74e9-c403-406a-97a0-a2af0c26ed91 state: Requested -> Created
112025-01-17T07:39:56.757ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-01-17T07:39:56.758ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-27fb74e9-c403-406a-97a0-a2af0c26ed91
132025-01-17T07:39:56.771ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-27fb74e9-c403-406a-97a0-a2af0c26ed91
142025-01-17T07:39:56.788ZINFOcrucible-agent (worker): creating config property group
152025-01-17T07:39:56.791ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-27fb74e9-c403-406a-97a0-a2af0c26ed91
162025-01-17T07:39:56.791ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/27fb74e9-c403-406a-97a0-a2af0c26ed91
172025-01-17T07:39:56.791ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-01-17T07:39:56.791ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
192025-01-17T07:39:56.791ZINFOcrucible-agent (worker): commit
202025-01-17T07:39:56.795ZINFOcrucible-agent (worker): ok!
212025-01-17T07:39:56.800ZINFOcrucible-agent (dropshot): request completed latency_us = 106 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38640 req_id = 6d4164ce-d8b9-4911-9bbf-a301f673fcee response_code = 200 uri = /crucible/0/regions
222025-01-17T07:39:56.802ZINFOcrucible-agent (worker): SMF ok!
232025-01-17T07:39:58.149ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:38640 req_id = 85be4afb-28ca-4717-b632-b3d2f42a7d85 response_code = 200 uri = /crucible/0/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0
242025-01-17T07:39:58.151ZINFOcrucible-agent (datafile): path is "/data/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0"
252025-01-17T07:39:58.161ZINFOcrucible-agent (datafile): dataset is oxp_b5a87517-097c-4a2f-afbf-79d6c9f09841/crucible/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0
262025-01-17T07:39:58.170ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b5a87517-097c-4a2f-afbf-79d6c9f09841/crucible/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0@86450f84-3eee-4115-949b-68c03f6303f2
272025-01-17T07:39:58.170ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b5a87517-097c-4a2f-afbf-79d6c9f09841/crucible/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0", "86450f84-3eee-4115-949b-68c03f6303f2"]
282025-01-17T07:39:58.181ZINFOcrucible-agent (dropshot): request completed latency_us = 29913 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:38640 req_id = 0832ff8f-84e8-4c57-b078-59a2b85da2ca response_code = 200 uri = /crucible/0/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0/snapshots/86450f84-3eee-4115-949b-68c03f6303f2
292025-01-17T07:39:58.185ZINFOcrucible-agent (datafile): path is "/data/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0"
302025-01-17T07:39:58.193ZINFOcrucible-agent (datafile): dataset is oxp_b5a87517-097c-4a2f-afbf-79d6c9f09841/crucible/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0
312025-01-17T07:39:58.201ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b5a87517-097c-4a2f-afbf-79d6c9f09841/crucible/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0@86450f84-3eee-4115-949b-68c03f6303f2
322025-01-17T07:39:58.201ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b5a87517-097c-4a2f-afbf-79d6c9f09841/crucible/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0", "86450f84-3eee-4115-949b-68c03f6303f2"]
332025-01-17T07:39:58.209ZINFOcrucible-agent (datafile): requesting running snapshot f85a7a06-0135-4672-89cf-bac17a02cfa0-86450f84-3eee-4115-949b-68c03f6303f2 state: Requested
342025-01-17T07:39:58.210ZINFOcrucible-agent (worker): applying SMF actions for region f85a7a06-0135-4672-89cf-bac17a02cfa0 running snapshot 86450f84-3eee-4115-949b-68c03f6303f2 (state Requested)...
352025-01-17T07:39:58.210ZINFOcrucible-agent (dropshot): request completed latency_us = 24861 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38640 req_id = 106e813b-03b6-4c1a-8b3f-459f65ea745c response_code = 200 uri = /crucible/0/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0/snapshots/86450f84-3eee-4115-949b-68c03f6303f2/run
362025-01-17T07:39:58.212ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-f85a7a06-0135-4672-89cf-bac17a02cfa0-86450f84-3eee-4115-949b-68c03f6303f2
372025-01-17T07:39:58.215ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-f85a7a06-0135-4672-89cf-bac17a02cfa0-86450f84-3eee-4115-949b-68c03f6303f2
382025-01-17T07:39:58.225ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-f85a7a06-0135-4672-89cf-bac17a02cfa0-86450f84-3eee-4115-949b-68c03f6303f2
392025-01-17T07:39:58.226ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0/.zfs/snapshot/86450f84-3eee-4115-949b-68c03f6303f2
402025-01-17T07:39:58.226ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-01-17T07:39:58.226ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-01-17T07:39:58.226ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
432025-01-17T07:39:58.226ZINFOcrucible-agent (worker): commit
442025-01-17T07:39:58.229ZINFOcrucible-agent (worker): ok!
452025-01-17T07:39:58.234ZINFOcrucible-agent (worker): SMF ok!
462025-01-17T07:39:58.234ZINFOcrucible-agent (datafile): region f85a7a06-0135-4672-89cf-bac17a02cfa0 running snapshot 86450f84-3eee-4115-949b-68c03f6303f2 state: Requested -> Created
472025-01-17T07:39:58.383ZINFOcrucible-agent (datafile): path is "/data/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0"
482025-01-17T07:39:58.396ZINFOcrucible-agent (datafile): dataset is oxp_b5a87517-097c-4a2f-afbf-79d6c9f09841/crucible/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0
492025-01-17T07:39:58.407ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b5a87517-097c-4a2f-afbf-79d6c9f09841/crucible/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0@86450f84-3eee-4115-949b-68c03f6303f2
502025-01-17T07:39:58.407ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b5a87517-097c-4a2f-afbf-79d6c9f09841/crucible/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0", "86450f84-3eee-4115-949b-68c03f6303f2"]
512025-01-17T07:39:58.419ZINFOcrucible-agent (dropshot): request completed latency_us = 35768 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38640 req_id = 979c9dec-18f2-42d5-8b60-6aafa798de4c response_code = 200 uri = /crucible/0/regions/f85a7a06-0135-4672-89cf-bac17a02cfa0/snapshots/86450f84-3eee-4115-949b-68c03f6303f2/run