Max level shown:
1 [ May 8 21:58:25 Enabled. ]
2 [ May 8 21:58:25 Rereading configuration. ]
3 [ May 8 21:58:28 Rereading configuration. ]
4 [ May 8 21:58:31 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_919c09e7-2acf-4d38-a80c-d10b0d488f35/crucible -l [fd00:1122:3344:101::14]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-05-08T21:58:31.679ZINFOcrucible-agent: dataset: "oxp_919c09e7-2acf-4d38-a80c-d10b0d488f35/crucible"
72025-05-08T21:58:31.683ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::14]:32345
82025-05-08T21:58:31.683ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-05-08T21:58:31.701ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-05-08T21:58:31.772ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::14]:32345
112025-05-08T21:59:27.993ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:55699
122025-05-08T21:59:28.015ZINFOcrucible-agent (datafile): region 491f684d-ae0d-47dd-bd73-0fe2f980c709 state: Requested
132025-05-08T21:59:28.016ZINFOcrucible-agent (dropshot): request completed latency_us = 12180 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55699 req_id = 1499bf8b-97b5-4df9-8ca2-c1e206c0b41d response_code = 200 uri = /crucible/0/regions
142025-05-08T21:59:28.017ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152025-05-08T21:59:28.027ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_919c09e7-2acf-4d38-a80c-d10b0d488f35/crucible/regions/491f684d-ae0d-47dd-bd73-0fe2f980c709
162025-05-08T21:59:28.028ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_919c09e7-2acf-4d38-a80c-d10b0d488f35/crucible/regions/491f684d-ae0d-47dd-bd73-0fe2f980c709
172025-05-08T21:59:28.084ZINFOcrucible-agent (worker): creating region Region { id: RegionId("491f684d-ae0d-47dd-bd73-0fe2f980c709"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/491f684d-ae0d-47dd-bd73-0fe2f980c709" region = 491f684d-ae0d-47dd-bd73-0fe2f980c709
182025-05-08T21:59:28.086ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "491f684d-ae0d-47dd-bd73-0fe2f980c709" "--data" "/data/regions/491f684d-ae0d-47dd-bd73-0fe2f980c709" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 491f684d-ae0d-47dd-bd73-0fe2f980c709
192025-05-08T21:59:28.212ZINFOcrucible-agent (worker): region files created ok region = 491f684d-ae0d-47dd-bd73-0fe2f980c709
202025-05-08T21:59:28.212ZINFOcrucible-agent (datafile): region 491f684d-ae0d-47dd-bd73-0fe2f980c709 state: Requested -> Created
212025-05-08T21:59:28.212ZINFOcrucible-agent (worker): applying SMF actions post create...
222025-05-08T21:59:28.213ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-491f684d-ae0d-47dd-bd73-0fe2f980c709
232025-05-08T21:59:28.220ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-491f684d-ae0d-47dd-bd73-0fe2f980c709
242025-05-08T21:59:28.237ZINFOcrucible-agent (worker): creating config property group
252025-05-08T21:59:28.240ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-491f684d-ae0d-47dd-bd73-0fe2f980c709
262025-05-08T21:59:28.241ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/491f684d-ae0d-47dd-bd73-0fe2f980c709
272025-05-08T21:59:28.241ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
282025-05-08T21:59:28.241ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
292025-05-08T21:59:28.241ZINFOcrucible-agent (worker): commit
302025-05-08T21:59:28.245ZINFOcrucible-agent (worker): ok!
312025-05-08T21:59:28.250ZINFOcrucible-agent (worker): SMF ok!
322025-05-08T21:59:28.295ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55699 req_id = 46657dd6-0de9-44bc-87d8-4168634337e2 response_code = 200 uri = /crucible/0/regions
332025-05-08T21:59:29.376ZINFOcrucible-agent (dropshot): request completed latency_us = 70 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:55699 req_id = 54dd03b8-1472-46e4-b51a-747939787b87 response_code = 200 uri = /crucible/0/regions/491f684d-ae0d-47dd-bd73-0fe2f980c709
342025-05-08T21:59:29.380ZINFOcrucible-agent (datafile): path is "/data/regions/491f684d-ae0d-47dd-bd73-0fe2f980c709"
352025-05-08T21:59:29.391ZINFOcrucible-agent (datafile): dataset is oxp_919c09e7-2acf-4d38-a80c-d10b0d488f35/crucible/regions/491f684d-ae0d-47dd-bd73-0fe2f980c709
362025-05-08T21:59:29.402ZINFOcrucible-agent (datafile): region 491f684d-ae0d-47dd-bd73-0fe2f980c709 state: Created -> Tombstoned
372025-05-08T21:59:29.403ZINFOcrucible-agent (worker): applying SMF actions before removal...
382025-05-08T21:59:29.403ZINFOcrucible-agent (dropshot): request completed latency_us = 22924 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:55699 req_id = 5ad612fb-c075-4a41-bc69-e83cd306e8f2 response_code = 204 uri = /crucible/0/regions/491f684d-ae0d-47dd-bd73-0fe2f980c709
392025-05-08T21:59:29.404ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-491f684d-ae0d-47dd-bd73-0fe2f980c709 (instance states: (Some(Online), None))
402025-05-08T21:59:29.406ZINFOcrucible-agent (dropshot): request completed latency_us = 31 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:55699 req_id = bf7b6c15-0e3a-419a-9dec-46f22139bf0c response_code = 200 uri = /crucible/0/regions/491f684d-ae0d-47dd-bd73-0fe2f980c709
412025-05-08T21:59:29.408ZINFOcrucible-agent (worker): SMF ok!
422025-05-08T21:59:29.422ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_919c09e7-2acf-4d38-a80c-d10b0d488f35/crucible/regions/491f684d-ae0d-47dd-bd73-0fe2f980c709" region = 491f684d-ae0d-47dd-bd73-0fe2f980c709
432025-05-08T21:59:29.496ZINFOcrucible-agent (datafile): region 491f684d-ae0d-47dd-bd73-0fe2f980c709 state: Tombstoned -> Destroyed
442025-05-08T21:59:29.538ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:55699 req_id = 2a21a3d5-8623-4ae4-9447-9cd83c38924a response_code = 200 uri = /crucible/0/regions/491f684d-ae0d-47dd-bd73-0fe2f980c709
452025-05-08T21:59:31.800ZINFOcrucible-agent (datafile): region 20e18ce3-e7e1-4e9d-b507-f56ca6b7b307 state: Requested
462025-05-08T21:59:31.800ZINFOcrucible-agent (dropshot): request completed latency_us = 255 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55699 req_id = 0293be9f-0568-4a9c-991f-bf4f4a398ca5 response_code = 200 uri = /crucible/0/regions
472025-05-08T21:59:31.800ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
482025-05-08T21:59:31.810ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_919c09e7-2acf-4d38-a80c-d10b0d488f35/crucible/regions/20e18ce3-e7e1-4e9d-b507-f56ca6b7b307
492025-05-08T21:59:31.810ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_919c09e7-2acf-4d38-a80c-d10b0d488f35/crucible/regions/20e18ce3-e7e1-4e9d-b507-f56ca6b7b307
502025-05-08T21:59:31.897ZINFOcrucible-agent (worker): creating region Region { id: RegionId("20e18ce3-e7e1-4e9d-b507-f56ca6b7b307"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/20e18ce3-e7e1-4e9d-b507-f56ca6b7b307" region = 20e18ce3-e7e1-4e9d-b507-f56ca6b7b307
512025-05-08T21:59:31.899ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "20e18ce3-e7e1-4e9d-b507-f56ca6b7b307" "--data" "/data/regions/20e18ce3-e7e1-4e9d-b507-f56ca6b7b307" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 20e18ce3-e7e1-4e9d-b507-f56ca6b7b307
522025-05-08T21:59:31.995ZINFOcrucible-agent (worker): region files created ok region = 20e18ce3-e7e1-4e9d-b507-f56ca6b7b307
532025-05-08T21:59:31.995ZINFOcrucible-agent (datafile): region 20e18ce3-e7e1-4e9d-b507-f56ca6b7b307 state: Requested -> Created
542025-05-08T21:59:31.995ZINFOcrucible-agent (worker): applying SMF actions post create...
552025-05-08T21:59:31.996ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-20e18ce3-e7e1-4e9d-b507-f56ca6b7b307
562025-05-08T21:59:31.999ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-20e18ce3-e7e1-4e9d-b507-f56ca6b7b307
572025-05-08T21:59:32.007ZINFOcrucible-agent (worker): creating config property group
582025-05-08T21:59:32.011ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-20e18ce3-e7e1-4e9d-b507-f56ca6b7b307
592025-05-08T21:59:32.012ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/20e18ce3-e7e1-4e9d-b507-f56ca6b7b307
602025-05-08T21:59:32.012ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
612025-05-08T21:59:32.012ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
622025-05-08T21:59:32.012ZINFOcrucible-agent (worker): commit
632025-05-08T21:59:32.018ZINFOcrucible-agent (worker): ok!
642025-05-08T21:59:32.025ZINFOcrucible-agent (worker): SMF ok!
652025-05-08T21:59:32.146ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55699 req_id = 8f85aa2a-745e-4811-adb0-3ce9186a7109 response_code = 200 uri = /crucible/0/regions