Max level shown:
12025-12-23T01:49:00.210ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::b]:57078
22025-12-23T01:49:00.210ZINFOcrucible-agent (datafile): region dba67bce-1e88-4321-b663-814b4f9b061e state: Requested
32025-12-23T01:49:00.211ZINFOcrucible-agent (dropshot): request completed latency_us = 635 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:57078 req_id = 27e40150-a0dd-4244-b52f-bfdf43a2e4d8 response_code = 200 uri = /crucible/0/regions
42025-12-23T01:49:00.211ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52025-12-23T01:49:00.221ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_3bf40781-e915-4295-81f7-a73109183637/crucible/regions/dba67bce-1e88-4321-b663-814b4f9b061e
62025-12-23T01:49:00.221ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_3bf40781-e915-4295-81f7-a73109183637/crucible/regions/dba67bce-1e88-4321-b663-814b4f9b061e
72025-12-23T01:49:00.271ZINFOcrucible-agent (worker): creating region Region { id: RegionId("dba67bce-1e88-4321-b663-814b4f9b061e"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/dba67bce-1e88-4321-b663-814b4f9b061e" region = dba67bce-1e88-4321-b663-814b4f9b061e
82025-12-23T01:49:00.271ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "dba67bce-1e88-4321-b663-814b4f9b061e" "--data" "/data/regions/dba67bce-1e88-4321-b663-814b4f9b061e" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = dba67bce-1e88-4321-b663-814b4f9b061e
92025-12-23T01:49:00.368ZINFOcrucible-agent (worker): region files created ok region = dba67bce-1e88-4321-b663-814b4f9b061e
102025-12-23T01:49:00.369ZINFOcrucible-agent (datafile): region dba67bce-1e88-4321-b663-814b4f9b061e state: Requested -> Created
112025-12-23T01:49:00.369ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-12-23T01:49:00.388ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-dba67bce-1e88-4321-b663-814b4f9b061e
132025-12-23T01:49:00.394ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-dba67bce-1e88-4321-b663-814b4f9b061e
142025-12-23T01:49:00.406ZINFOcrucible-agent (worker): creating config property group
152025-12-23T01:49:00.409ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-dba67bce-1e88-4321-b663-814b4f9b061e
162025-12-23T01:49:00.409ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/dba67bce-1e88-4321-b663-814b4f9b061e
172025-12-23T01:49:00.409ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-12-23T01:49:00.409ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
192025-12-23T01:49:00.409ZINFOcrucible-agent (worker): commit
202025-12-23T01:49:00.413ZINFOcrucible-agent (worker): ok!
212025-12-23T01:49:00.417ZINFOcrucible-agent (worker): SMF ok!
222025-12-23T01:49:00.553ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:57078 req_id = d3e5c165-ffc3-47d2-8672-f3f6ed38d59e response_code = 200 uri = /crucible/0/regions
232025-12-23T01:49:00.933ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:57078 req_id = 208f5044-d8c0-40f7-a108-a2f4c86caefe response_code = 200 uri = /crucible/0/regions/31642235-185f-4c85-8a88-d4bce6b7918a
242025-12-23T01:49:00.936ZINFOcrucible-agent (datafile): path is "/data/regions/31642235-185f-4c85-8a88-d4bce6b7918a"
252025-12-23T01:49:00.944ZINFOcrucible-agent (datafile): dataset is oxp_3bf40781-e915-4295-81f7-a73109183637/crucible/regions/31642235-185f-4c85-8a88-d4bce6b7918a
262025-12-23T01:49:00.950ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3bf40781-e915-4295-81f7-a73109183637/crucible/regions/31642235-185f-4c85-8a88-d4bce6b7918a@45477e06-0bdf-482e-973b-3eb2b7281fea
272025-12-23T01:49:00.950ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3bf40781-e915-4295-81f7-a73109183637/crucible/regions/31642235-185f-4c85-8a88-d4bce6b7918a", "45477e06-0bdf-482e-973b-3eb2b7281fea"]
282025-12-23T01:49:00.959ZINFOcrucible-agent (dropshot): request completed latency_us = 22064 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:57078 req_id = bccce3ac-7139-44a4-86f0-961be4ba1d65 response_code = 200 uri = /crucible/0/regions/31642235-185f-4c85-8a88-d4bce6b7918a/snapshots/45477e06-0bdf-482e-973b-3eb2b7281fea
292025-12-23T01:49:00.966ZINFOcrucible-agent (datafile): path is "/data/regions/31642235-185f-4c85-8a88-d4bce6b7918a"
302025-12-23T01:49:00.973ZINFOcrucible-agent (datafile): dataset is oxp_3bf40781-e915-4295-81f7-a73109183637/crucible/regions/31642235-185f-4c85-8a88-d4bce6b7918a
312025-12-23T01:49:00.980ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3bf40781-e915-4295-81f7-a73109183637/crucible/regions/31642235-185f-4c85-8a88-d4bce6b7918a@45477e06-0bdf-482e-973b-3eb2b7281fea
322025-12-23T01:49:00.980ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3bf40781-e915-4295-81f7-a73109183637/crucible/regions/31642235-185f-4c85-8a88-d4bce6b7918a", "45477e06-0bdf-482e-973b-3eb2b7281fea"]
332025-12-23T01:49:00.988ZINFOcrucible-agent (datafile): requesting running snapshot 31642235-185f-4c85-8a88-d4bce6b7918a-45477e06-0bdf-482e-973b-3eb2b7281fea state: Requested
342025-12-23T01:49:00.988ZINFOcrucible-agent (dropshot): request completed latency_us = 22616 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:57078 req_id = 96ffe72a-212c-432d-9cd3-9c978b3e775e response_code = 200 uri = /crucible/0/regions/31642235-185f-4c85-8a88-d4bce6b7918a/snapshots/45477e06-0bdf-482e-973b-3eb2b7281fea/run
352025-12-23T01:49:00.988ZINFOcrucible-agent (worker): applying SMF actions for region 31642235-185f-4c85-8a88-d4bce6b7918a running snapshot 45477e06-0bdf-482e-973b-3eb2b7281fea (state Requested)...
362025-12-23T01:49:00.993ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-31642235-185f-4c85-8a88-d4bce6b7918a-45477e06-0bdf-482e-973b-3eb2b7281fea
372025-12-23T01:49:01.001ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-31642235-185f-4c85-8a88-d4bce6b7918a-45477e06-0bdf-482e-973b-3eb2b7281fea
382025-12-23T01:49:01.011ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-31642235-185f-4c85-8a88-d4bce6b7918a-45477e06-0bdf-482e-973b-3eb2b7281fea
392025-12-23T01:49:01.011ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/31642235-185f-4c85-8a88-d4bce6b7918a/.zfs/snapshot/45477e06-0bdf-482e-973b-3eb2b7281fea
402025-12-23T01:49:01.011ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-12-23T01:49:01.011ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-12-23T01:49:01.011ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
432025-12-23T01:49:01.011ZINFOcrucible-agent (worker): commit
442025-12-23T01:49:01.015ZINFOcrucible-agent (worker): ok!
452025-12-23T01:49:01.019ZINFOcrucible-agent (worker): SMF ok!
462025-12-23T01:49:01.019ZINFOcrucible-agent (datafile): region 31642235-185f-4c85-8a88-d4bce6b7918a running snapshot 45477e06-0bdf-482e-973b-3eb2b7281fea state: Requested -> Created
472025-12-23T01:49:01.155ZINFOcrucible-agent (datafile): path is "/data/regions/31642235-185f-4c85-8a88-d4bce6b7918a"
482025-12-23T01:49:01.162ZINFOcrucible-agent (datafile): dataset is oxp_3bf40781-e915-4295-81f7-a73109183637/crucible/regions/31642235-185f-4c85-8a88-d4bce6b7918a
492025-12-23T01:49:01.168ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3bf40781-e915-4295-81f7-a73109183637/crucible/regions/31642235-185f-4c85-8a88-d4bce6b7918a@45477e06-0bdf-482e-973b-3eb2b7281fea
502025-12-23T01:49:01.168ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3bf40781-e915-4295-81f7-a73109183637/crucible/regions/31642235-185f-4c85-8a88-d4bce6b7918a", "45477e06-0bdf-482e-973b-3eb2b7281fea"]
512025-12-23T01:49:01.174ZINFOcrucible-agent (dropshot): request completed latency_us = 18727 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:57078 req_id = 75774db4-a500-4375-8f1b-96a0304ba1e8 response_code = 200 uri = /crucible/0/regions/31642235-185f-4c85-8a88-d4bce6b7918a/snapshots/45477e06-0bdf-482e-973b-3eb2b7281fea/run