Max level shown:
12026-11-07T01:30:55.570ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:50226
22026-11-07T01:30:55.601ZINFOcrucible-agent (datafile): region d0139a65-ec5c-46ca-a00b-d483d5722e60 state: Requested
32026-11-07T01:30:55.608ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-11-07T01:30:55.612ZINFOcrucible-agent (dropshot): request completed latency_us = 17339 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50226 req_id = 353f0bd2-a83d-473a-84e0-aefa4113f81d response_code = 200 uri = /crucible/0/regions
52026-11-07T01:30:55.614ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_70a3ec24-ad83-4e86-8549-f962422b08c6/crucible/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60
62026-11-07T01:30:55.614ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_70a3ec24-ad83-4e86-8549-f962422b08c6/crucible/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60
72026-11-07T01:30:55.677ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d0139a65-ec5c-46ca-a00b-d483d5722e60"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60" region = d0139a65-ec5c-46ca-a00b-d483d5722e60
82026-11-07T01:30:55.677ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d0139a65-ec5c-46ca-a00b-d483d5722e60" "--data" "/data/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = d0139a65-ec5c-46ca-a00b-d483d5722e60
92026-11-07T01:30:55.797ZINFOcrucible-agent (worker): region files created ok region = d0139a65-ec5c-46ca-a00b-d483d5722e60
102026-11-07T01:30:55.799ZINFOcrucible-agent (datafile): region d0139a65-ec5c-46ca-a00b-d483d5722e60 state: Requested -> Created
112026-11-07T01:30:55.799ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-11-07T01:30:55.800ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d0139a65-ec5c-46ca-a00b-d483d5722e60
132026-11-07T01:30:55.806ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d0139a65-ec5c-46ca-a00b-d483d5722e60
142026-11-07T01:30:55.814ZINFOcrucible-agent (worker): creating config property group
152026-11-07T01:30:55.816ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d0139a65-ec5c-46ca-a00b-d483d5722e60
162026-11-07T01:30:55.817ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60
172026-11-07T01:30:55.817ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
182026-11-07T01:30:55.817ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
192026-11-07T01:30:55.817ZINFOcrucible-agent (worker): commit
202026-11-07T01:30:55.819ZINFOcrucible-agent (worker): ok!
212026-11-07T01:30:55.822ZINFOcrucible-agent (worker): SMF ok!
222026-11-07T01:30:55.902ZINFOcrucible-agent (dropshot): request completed latency_us = 79 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50226 req_id = 394ceff6-b3b2-4b12-a558-8b3f7bc4139b response_code = 200 uri = /crucible/0/regions
232026-11-07T01:41:51.552ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:65493
242026-11-07T01:41:51.552ZINFOcrucible-agent (dropshot): request completed latency_us = 43 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:65493 req_id = 803d70f1-dbb0-4371-b1a9-c559bdde3deb response_code = 200 uri = /crucible/0/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60
252026-11-07T01:41:51.559ZINFOcrucible-agent (datafile): path is "/data/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60"
262026-11-07T01:41:51.564ZINFOcrucible-agent (datafile): dataset is oxp_70a3ec24-ad83-4e86-8549-f962422b08c6/crucible/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60
272026-11-07T01:41:51.570ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_70a3ec24-ad83-4e86-8549-f962422b08c6/crucible/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60@55984e54-d07b-45fc-a09d-6bd039a59994
282026-11-07T01:41:51.570ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_70a3ec24-ad83-4e86-8549-f962422b08c6/crucible/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60", "55984e54-d07b-45fc-a09d-6bd039a59994"]
292026-11-07T01:41:51.581ZINFOcrucible-agent (dropshot): request completed latency_us = 25431 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:65493 req_id = 03743c93-8d0e-489b-9f97-ee835d97ef5d response_code = 200 uri = /crucible/0/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60/snapshots/55984e54-d07b-45fc-a09d-6bd039a59994
302026-11-07T01:41:51.585ZINFOcrucible-agent (datafile): path is "/data/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60"
312026-11-07T01:41:51.591ZINFOcrucible-agent (datafile): dataset is oxp_70a3ec24-ad83-4e86-8549-f962422b08c6/crucible/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60
322026-11-07T01:41:51.595ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_70a3ec24-ad83-4e86-8549-f962422b08c6/crucible/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60@55984e54-d07b-45fc-a09d-6bd039a59994
332026-11-07T01:41:51.595ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_70a3ec24-ad83-4e86-8549-f962422b08c6/crucible/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60", "55984e54-d07b-45fc-a09d-6bd039a59994"]
342026-11-07T01:41:51.601ZINFOcrucible-agent (datafile): requesting running snapshot d0139a65-ec5c-46ca-a00b-d483d5722e60-55984e54-d07b-45fc-a09d-6bd039a59994 state: Requested
352026-11-07T01:41:51.602ZINFOcrucible-agent (dropshot): request completed latency_us = 16816 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:65493 req_id = a0d48b29-7bf6-4547-a91d-57c519509e7f response_code = 200 uri = /crucible/0/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60/snapshots/55984e54-d07b-45fc-a09d-6bd039a59994/run
362026-11-07T01:41:51.602ZINFOcrucible-agent (worker): applying SMF actions for region d0139a65-ec5c-46ca-a00b-d483d5722e60 running snapshot 55984e54-d07b-45fc-a09d-6bd039a59994 (state Requested)...
372026-11-07T01:41:51.606ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-d0139a65-ec5c-46ca-a00b-d483d5722e60-55984e54-d07b-45fc-a09d-6bd039a59994
382026-11-07T01:41:51.608ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-d0139a65-ec5c-46ca-a00b-d483d5722e60-55984e54-d07b-45fc-a09d-6bd039a59994
392026-11-07T01:41:51.613ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-d0139a65-ec5c-46ca-a00b-d483d5722e60-55984e54-d07b-45fc-a09d-6bd039a59994
402026-11-07T01:41:51.613ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60/.zfs/snapshot/55984e54-d07b-45fc-a09d-6bd039a59994
412026-11-07T01:41:51.613ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
422026-11-07T01:41:51.613ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
432026-11-07T01:41:51.613ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
442026-11-07T01:41:51.613ZINFOcrucible-agent (worker): commit
452026-11-07T01:41:51.616ZINFOcrucible-agent (worker): ok!
462026-11-07T01:41:51.619ZINFOcrucible-agent (worker): SMF ok!
472026-11-07T01:41:51.619ZINFOcrucible-agent (datafile): region d0139a65-ec5c-46ca-a00b-d483d5722e60 running snapshot 55984e54-d07b-45fc-a09d-6bd039a59994 state: Requested -> Created
482026-11-07T01:41:51.767ZINFOcrucible-agent (datafile): path is "/data/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60"
492026-11-07T01:41:51.773ZINFOcrucible-agent (datafile): dataset is oxp_70a3ec24-ad83-4e86-8549-f962422b08c6/crucible/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60
502026-11-07T01:41:51.779ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_70a3ec24-ad83-4e86-8549-f962422b08c6/crucible/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60@55984e54-d07b-45fc-a09d-6bd039a59994
512026-11-07T01:41:51.779ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_70a3ec24-ad83-4e86-8549-f962422b08c6/crucible/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60", "55984e54-d07b-45fc-a09d-6bd039a59994"]
522026-11-07T01:41:51.785ZINFOcrucible-agent (dropshot): request completed latency_us = 18643 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:65493 req_id = c550331b-90af-4ce4-9b3d-86e780f1ff32 response_code = 200 uri = /crucible/0/regions/d0139a65-ec5c-46ca-a00b-d483d5722e60/snapshots/55984e54-d07b-45fc-a09d-6bd039a59994/run