Max level shown:
1 [ Apr 16 03:53:54 Enabled. ]
2 [ Apr 16 03:53:54 Rereading configuration. ]
3 [ Apr 16 03:53:57 Rereading configuration. ]
4 [ Apr 16 03:54:06 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_9e21f20e-6490-442c-8054-e7093cd793d5/crucible -l [fd00:1122:3344:101::12]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-04-16T03:54:06.184ZINFOcrucible-agent: dataset: "oxp_9e21f20e-6490-442c-8054-e7093cd793d5/crucible"
72025-04-16T03:54:06.187ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::12]:32345
82025-04-16T03:54:06.187ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-04-16T03:54:06.205ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-04-16T03:54:06.519ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::12]:32345
112025-04-16T03:54:29.316ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:51743
122025-04-16T03:54:29.342ZINFOcrucible-agent (datafile): region d226aff5-7ef1-4460-b237-3baef566d006 state: Requested
132025-04-16T03:54:29.343ZINFOcrucible-agent (dropshot): request completed latency_us = 10146 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51743 req_id = 8ed66f80-53c0-447e-981b-e20fba51abce response_code = 200 uri = /crucible/0/regions
142025-04-16T03:54:29.345ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152025-04-16T03:54:29.356ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_9e21f20e-6490-442c-8054-e7093cd793d5/crucible/regions/d226aff5-7ef1-4460-b237-3baef566d006
162025-04-16T03:54:29.356ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_9e21f20e-6490-442c-8054-e7093cd793d5/crucible/regions/d226aff5-7ef1-4460-b237-3baef566d006
172025-04-16T03:54:29.468ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d226aff5-7ef1-4460-b237-3baef566d006"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d226aff5-7ef1-4460-b237-3baef566d006" region = d226aff5-7ef1-4460-b237-3baef566d006
182025-04-16T03:54:29.471ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d226aff5-7ef1-4460-b237-3baef566d006" "--data" "/data/regions/d226aff5-7ef1-4460-b237-3baef566d006" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = d226aff5-7ef1-4460-b237-3baef566d006
192025-04-16T03:54:29.626ZINFOcrucible-agent (worker): region files created ok region = d226aff5-7ef1-4460-b237-3baef566d006
202025-04-16T03:54:29.626ZINFOcrucible-agent (datafile): region d226aff5-7ef1-4460-b237-3baef566d006 state: Requested -> Created
212025-04-16T03:54:29.626ZINFOcrucible-agent (worker): applying SMF actions post create...
222025-04-16T03:54:29.626ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d226aff5-7ef1-4460-b237-3baef566d006
232025-04-16T03:54:29.633ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d226aff5-7ef1-4460-b237-3baef566d006
242025-04-16T03:54:29.654ZINFOcrucible-agent (worker): creating config property group
252025-04-16T03:54:29.661ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d226aff5-7ef1-4460-b237-3baef566d006
262025-04-16T03:54:29.672ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d226aff5-7ef1-4460-b237-3baef566d006
272025-04-16T03:54:29.672ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
282025-04-16T03:54:29.672ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
292025-04-16T03:54:29.672ZINFOcrucible-agent (worker): commit
302025-04-16T03:54:29.677ZINFOcrucible-agent (worker): ok!
312025-04-16T03:54:29.682ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51743 req_id = 17831df1-ba21-4ab9-a68a-39fd5ef5588b response_code = 200 uri = /crucible/0/regions
322025-04-16T03:54:29.685ZINFOcrucible-agent (worker): SMF ok!
332025-04-16T03:54:30.690ZINFOcrucible-agent (dropshot): request completed latency_us = 75 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:51743 req_id = c66db0b6-85eb-441d-bef9-920751b6a081 response_code = 200 uri = /crucible/0/regions/d226aff5-7ef1-4460-b237-3baef566d006
342025-04-16T03:54:30.693ZINFOcrucible-agent (datafile): path is "/data/regions/d226aff5-7ef1-4460-b237-3baef566d006"
352025-04-16T03:54:30.707ZINFOcrucible-agent (datafile): dataset is oxp_9e21f20e-6490-442c-8054-e7093cd793d5/crucible/regions/d226aff5-7ef1-4460-b237-3baef566d006
362025-04-16T03:54:30.719ZINFOcrucible-agent (datafile): region d226aff5-7ef1-4460-b237-3baef566d006 state: Created -> Tombstoned
372025-04-16T03:54:30.719ZINFOcrucible-agent (worker): applying SMF actions before removal...
382025-04-16T03:54:30.719ZINFOcrucible-agent (dropshot): request completed latency_us = 25663 local_addr = [fd00:1122:3344:101::12]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:51743 req_id = 2bb4c3ae-6cae-4356-85da-9b09b0cdbf5c response_code = 204 uri = /crucible/0/regions/d226aff5-7ef1-4460-b237-3baef566d006
392025-04-16T03:54:30.720ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-d226aff5-7ef1-4460-b237-3baef566d006 (instance states: (Some(Online), None))
402025-04-16T03:54:30.724ZINFOcrucible-agent (dropshot): request completed latency_us = 35 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:51743 req_id = 117a4125-8f50-4d76-896b-ecce149e6540 response_code = 200 uri = /crucible/0/regions/d226aff5-7ef1-4460-b237-3baef566d006
412025-04-16T03:54:30.725ZINFOcrucible-agent (worker): SMF ok!
422025-04-16T03:54:30.738ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_9e21f20e-6490-442c-8054-e7093cd793d5/crucible/regions/d226aff5-7ef1-4460-b237-3baef566d006" region = d226aff5-7ef1-4460-b237-3baef566d006
432025-04-16T03:54:30.837ZINFOcrucible-agent (datafile): region d226aff5-7ef1-4460-b237-3baef566d006 state: Tombstoned -> Destroyed
442025-04-16T03:54:30.876ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:51743 req_id = 490c35d9-2127-4f9f-aa41-a977dfe8f61f response_code = 200 uri = /crucible/0/regions/d226aff5-7ef1-4460-b237-3baef566d006
452025-04-16T03:54:33.310ZINFOcrucible-agent (datafile): region d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f state: Requested
462025-04-16T03:54:33.310ZINFOcrucible-agent (dropshot): request completed latency_us = 278 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51743 req_id = df3309d4-c542-4382-90ed-6fee7d2b6ef3 response_code = 200 uri = /crucible/0/regions
472025-04-16T03:54:33.310ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
482025-04-16T03:54:33.320ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_9e21f20e-6490-442c-8054-e7093cd793d5/crucible/regions/d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f
492025-04-16T03:54:33.320ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_9e21f20e-6490-442c-8054-e7093cd793d5/crucible/regions/d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f
502025-04-16T03:54:33.439ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f" region = d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f
512025-04-16T03:54:33.442ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f" "--data" "/data/regions/d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f
522025-04-16T03:54:33.588ZINFOcrucible-agent (dropshot): request completed latency_us = 76 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51743 req_id = 5df17c7d-24b6-4693-bd44-8926410a1d4c response_code = 200 uri = /crucible/0/regions
532025-04-16T03:54:33.684ZINFOcrucible-agent (worker): region files created ok region = d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f
542025-04-16T03:54:33.684ZINFOcrucible-agent (datafile): region d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f state: Requested -> Created
552025-04-16T03:54:33.684ZINFOcrucible-agent (worker): applying SMF actions post create...
562025-04-16T03:54:33.686ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f
572025-04-16T03:54:33.693ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f
582025-04-16T03:54:33.708ZINFOcrucible-agent (worker): creating config property group
592025-04-16T03:54:33.712ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f
602025-04-16T03:54:33.712ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d5dbda5e-ebf0-4a6d-8fd6-a2f32016fe1f
612025-04-16T03:54:33.712ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
622025-04-16T03:54:33.712ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
632025-04-16T03:54:33.712ZINFOcrucible-agent (worker): commit
642025-04-16T03:54:33.717ZINFOcrucible-agent (worker): ok!
652025-04-16T03:54:33.723ZINFOcrucible-agent (worker): SMF ok!
662025-04-16T03:54:33.997ZINFOcrucible-agent (dropshot): request completed latency_us = 51 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51743 req_id = 3c862fe6-736a-472c-b80a-e43eebfe90b5 response_code = 200 uri = /crucible/0/regions