Max level shown:
1 [ Sep 17 20:36:42 Enabled. ]
2 [ Sep 17 20:36:43 Rereading configuration. ]
3 [ Sep 17 20:36:47 Rereading configuration. ]
4 [ Sep 17 20:36:54 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_e19d73aa-c08b-435c-ac10-1c882c11ec61/crucible -l [fd00:1122:3344:101::14]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-09-17T20:36:54.566ZINFOcrucible-agent: Crucible Version: Crucible Version: 0.0.1 Commit SHA: 779775d5130ff7a4836f52f48b7e64d1479ee104 Commit timestamp: 2025-08-09T00:27:59.000000000Z branch: main rustc: 1.86.0 stable x86_64-unknown-illumos Cargo: x86_64-unknown-illumos Debug: false Opt level: 3
72025-09-17T20:36:54.570ZINFOcrucible-agent: dataset: "oxp_e19d73aa-c08b-435c-ac10-1c882c11ec61/crucible"
82025-09-17T20:36:54.570ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::14]:32345
92025-09-17T20:36:54.570ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
102025-09-17T20:36:54.588ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
112025-09-17T20:36:54.694ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::14]:32345
122025-09-17T20:37:31.391ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:49997
132025-09-17T20:37:31.439ZINFOcrucible-agent (datafile): region 217fd9ff-d8fd-453d-8796-c05a46a26c25 state: Requested
142025-09-17T20:37:31.446ZINFOcrucible-agent (dropshot): request completed latency_us = 26890 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49997 req_id = 2f38071a-9350-4a1f-b548-c80e4ba97c3d response_code = 200 uri = /crucible/0/regions
152025-09-17T20:37:31.451ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
162025-09-17T20:37:31.461ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_e19d73aa-c08b-435c-ac10-1c882c11ec61/crucible/regions/217fd9ff-d8fd-453d-8796-c05a46a26c25
172025-09-17T20:37:31.461ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_e19d73aa-c08b-435c-ac10-1c882c11ec61/crucible/regions/217fd9ff-d8fd-453d-8796-c05a46a26c25
182025-09-17T20:37:31.515ZINFOcrucible-agent (worker): creating region Region { id: RegionId("217fd9ff-d8fd-453d-8796-c05a46a26c25"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/217fd9ff-d8fd-453d-8796-c05a46a26c25" region = 217fd9ff-d8fd-453d-8796-c05a46a26c25
192025-09-17T20:37:31.517ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "217fd9ff-d8fd-453d-8796-c05a46a26c25" "--data" "/data/regions/217fd9ff-d8fd-453d-8796-c05a46a26c25" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 217fd9ff-d8fd-453d-8796-c05a46a26c25
202025-09-17T20:37:31.712ZINFOcrucible-agent (dropshot): request completed latency_us = 101 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49997 req_id = 5c46d676-9b77-4f36-afa6-2ac9fda33a45 response_code = 200 uri = /crucible/0/regions
212025-09-17T20:37:31.869ZINFOcrucible-agent (worker): region files created ok region = 217fd9ff-d8fd-453d-8796-c05a46a26c25
222025-09-17T20:37:31.870ZINFOcrucible-agent (datafile): region 217fd9ff-d8fd-453d-8796-c05a46a26c25 state: Requested -> Created
232025-09-17T20:37:31.870ZINFOcrucible-agent (worker): applying SMF actions post create...
242025-09-17T20:37:31.870ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-217fd9ff-d8fd-453d-8796-c05a46a26c25
252025-09-17T20:37:31.877ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-217fd9ff-d8fd-453d-8796-c05a46a26c25
262025-09-17T20:37:31.890ZINFOcrucible-agent (worker): creating config property group
272025-09-17T20:37:31.894ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-217fd9ff-d8fd-453d-8796-c05a46a26c25
282025-09-17T20:37:31.895ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/217fd9ff-d8fd-453d-8796-c05a46a26c25
292025-09-17T20:37:31.904ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
302025-09-17T20:37:31.904ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
312025-09-17T20:37:31.904ZINFOcrucible-agent (worker): commit
322025-09-17T20:37:31.908ZINFOcrucible-agent (worker): ok!
332025-09-17T20:37:31.914ZINFOcrucible-agent (worker): SMF ok!
342025-09-17T20:37:32.140ZINFOcrucible-agent (dropshot): request completed latency_us = 48 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49997 req_id = e963831f-ab4f-42ef-886b-432264838f27 response_code = 200 uri = /crucible/0/regions
352025-09-17T20:37:32.762ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:49997 req_id = f616d1a7-14cf-4f39-bf59-847702621eac response_code = 200 uri = /crucible/0/regions/217fd9ff-d8fd-453d-8796-c05a46a26c25
362025-09-17T20:37:32.765ZINFOcrucible-agent (datafile): path is "/data/regions/217fd9ff-d8fd-453d-8796-c05a46a26c25"
372025-09-17T20:37:32.776ZINFOcrucible-agent (datafile): dataset is oxp_e19d73aa-c08b-435c-ac10-1c882c11ec61/crucible/regions/217fd9ff-d8fd-453d-8796-c05a46a26c25
382025-09-17T20:37:32.788ZINFOcrucible-agent (datafile): region 217fd9ff-d8fd-453d-8796-c05a46a26c25 state: Created -> Tombstoned
392025-09-17T20:37:32.789ZINFOcrucible-agent (worker): applying SMF actions before removal...
402025-09-17T20:37:32.789ZINFOcrucible-agent (dropshot): request completed latency_us = 24075 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:49997 req_id = a04e59c9-b26b-46a5-ad55-a00be190eaa4 response_code = 204 uri = /crucible/0/regions/217fd9ff-d8fd-453d-8796-c05a46a26c25
412025-09-17T20:37:32.790ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-217fd9ff-d8fd-453d-8796-c05a46a26c25 (instance states: (Some(Online), None))
422025-09-17T20:37:32.795ZINFOcrucible-agent (dropshot): request completed latency_us = 31 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:49997 req_id = 7af219f7-cb6a-4f15-88b9-17e85b2f6120 response_code = 200 uri = /crucible/0/regions/217fd9ff-d8fd-453d-8796-c05a46a26c25
432025-09-17T20:37:32.796ZINFOcrucible-agent (worker): SMF ok!
442025-09-17T20:37:32.809ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_e19d73aa-c08b-435c-ac10-1c882c11ec61/crucible/regions/217fd9ff-d8fd-453d-8796-c05a46a26c25" region = 217fd9ff-d8fd-453d-8796-c05a46a26c25
452025-09-17T20:37:32.878ZINFOcrucible-agent (datafile): region 217fd9ff-d8fd-453d-8796-c05a46a26c25 state: Tombstoned -> Destroyed
462025-09-17T20:37:32.891ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:49997 req_id = a0d2375e-c3d6-4177-8fe3-0449adf2b0b3 response_code = 200 uri = /crucible/0/regions/217fd9ff-d8fd-453d-8796-c05a46a26c25
472025-09-17T20:37:34.712ZINFOcrucible-agent (datafile): region 717917ac-6ea4-4786-bee0-719f3d3c3071 state: Requested
482025-09-17T20:37:34.712ZINFOcrucible-agent (dropshot): request completed latency_us = 223 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49997 req_id = d92f5d1e-1241-493e-a551-a688412430d9 response_code = 200 uri = /crucible/0/regions
492025-09-17T20:37:34.712ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
502025-09-17T20:37:34.722ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_e19d73aa-c08b-435c-ac10-1c882c11ec61/crucible/regions/717917ac-6ea4-4786-bee0-719f3d3c3071
512025-09-17T20:37:34.722ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_e19d73aa-c08b-435c-ac10-1c882c11ec61/crucible/regions/717917ac-6ea4-4786-bee0-719f3d3c3071
522025-09-17T20:37:34.821ZINFOcrucible-agent (worker): creating region Region { id: RegionId("717917ac-6ea4-4786-bee0-719f3d3c3071"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/717917ac-6ea4-4786-bee0-719f3d3c3071" region = 717917ac-6ea4-4786-bee0-719f3d3c3071
532025-09-17T20:37:34.821ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "717917ac-6ea4-4786-bee0-719f3d3c3071" "--data" "/data/regions/717917ac-6ea4-4786-bee0-719f3d3c3071" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 717917ac-6ea4-4786-bee0-719f3d3c3071
542025-09-17T20:37:34.943ZINFOcrucible-agent (worker): region files created ok region = 717917ac-6ea4-4786-bee0-719f3d3c3071
552025-09-17T20:37:34.943ZINFOcrucible-agent (datafile): region 717917ac-6ea4-4786-bee0-719f3d3c3071 state: Requested -> Created
562025-09-17T20:37:34.943ZINFOcrucible-agent (worker): applying SMF actions post create...
572025-09-17T20:37:34.945ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-717917ac-6ea4-4786-bee0-719f3d3c3071
582025-09-17T20:37:34.950ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-717917ac-6ea4-4786-bee0-719f3d3c3071
592025-09-17T20:37:34.962ZINFOcrucible-agent (worker): creating config property group
602025-09-17T20:37:34.966ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-717917ac-6ea4-4786-bee0-719f3d3c3071
612025-09-17T20:37:34.966ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/717917ac-6ea4-4786-bee0-719f3d3c3071
622025-09-17T20:37:34.966ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632025-09-17T20:37:34.966ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
642025-09-17T20:37:34.967ZINFOcrucible-agent (worker): commit
652025-09-17T20:37:34.971ZINFOcrucible-agent (worker): ok!
662025-09-17T20:37:34.977ZINFOcrucible-agent (worker): SMF ok!
672025-09-17T20:37:34.990ZINFOcrucible-agent (dropshot): request completed latency_us = 81 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49997 req_id = ce07ba3b-f26e-493a-a2f5-27d6333c7f10 response_code = 200 uri = /crucible/0/regions