Max level shown:
1 [ Dec 22 16:22:10 Enabled. ]
2 [ Dec 22 16:22:10 Rereading configuration. ]
3 [ Dec 22 16:22:12 Rereading configuration. ]
4 [ Dec 22 16:22:21 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_713fed97-1fef-4f49-b90b-b20eaba7d366/crucible -l [fd00:1122:3344:101::16]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-12-22T16:22:21.492ZINFOcrucible-agent: Crucible Version: Crucible Version: 0.0.1 Commit SHA: 7103cd3a3d7b0112d2949dd135db06fef0c156bb Commit timestamp: 2025-11-10T22:24:53.000000000Z branch: main rustc: 1.90.0 stable x86_64-unknown-illumos Cargo: x86_64-unknown-illumos Debug: false Opt level: 3
72025-12-22T16:22:21.492ZINFOcrucible-agent: dataset: "oxp_713fed97-1fef-4f49-b90b-b20eaba7d366/crucible"
82025-12-22T16:22:21.492ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::16]:32345
92025-12-22T16:22:21.492ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
102025-12-22T16:22:21.505ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
112025-12-22T16:22:21.604ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::16]:32345
122025-12-22T16:23:35.195ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::c]:53943
132025-12-22T16:23:35.221ZINFOcrucible-agent (datafile): region f82b7048-533b-46a7-9cdb-b5009fa31748 state: Requested
142025-12-22T16:23:35.224ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152025-12-22T16:23:35.224ZINFOcrucible-agent (dropshot): request completed latency_us = 14734 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:53943 req_id = bd17d95c-8d44-4137-98f3-d6a468cc1acc response_code = 200 uri = /crucible/0/regions
162025-12-22T16:23:35.233ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_713fed97-1fef-4f49-b90b-b20eaba7d366/crucible/regions/f82b7048-533b-46a7-9cdb-b5009fa31748
172025-12-22T16:23:35.233ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_713fed97-1fef-4f49-b90b-b20eaba7d366/crucible/regions/f82b7048-533b-46a7-9cdb-b5009fa31748
182025-12-22T16:23:35.300ZINFOcrucible-agent (worker): creating region Region { id: RegionId("f82b7048-533b-46a7-9cdb-b5009fa31748"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/f82b7048-533b-46a7-9cdb-b5009fa31748" region = f82b7048-533b-46a7-9cdb-b5009fa31748
192025-12-22T16:23:35.302ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "f82b7048-533b-46a7-9cdb-b5009fa31748" "--data" "/data/regions/f82b7048-533b-46a7-9cdb-b5009fa31748" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = f82b7048-533b-46a7-9cdb-b5009fa31748
202025-12-22T16:23:35.469ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:53943 req_id = 2220b904-c17e-4794-9eda-5867d3f80a70 response_code = 200 uri = /crucible/0/regions
212025-12-22T16:23:35.633ZINFOcrucible-agent (worker): region files created ok region = f82b7048-533b-46a7-9cdb-b5009fa31748
222025-12-22T16:23:35.633ZINFOcrucible-agent (datafile): region f82b7048-533b-46a7-9cdb-b5009fa31748 state: Requested -> Created
232025-12-22T16:23:35.633ZINFOcrucible-agent (worker): applying SMF actions post create...
242025-12-22T16:23:35.633ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-f82b7048-533b-46a7-9cdb-b5009fa31748
252025-12-22T16:23:35.640ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-f82b7048-533b-46a7-9cdb-b5009fa31748
262025-12-22T16:23:35.657ZINFOcrucible-agent (worker): creating config property group
272025-12-22T16:23:35.660ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-f82b7048-533b-46a7-9cdb-b5009fa31748
282025-12-22T16:23:35.661ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f82b7048-533b-46a7-9cdb-b5009fa31748
292025-12-22T16:23:35.661ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
302025-12-22T16:23:35.661ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
312025-12-22T16:23:35.661ZINFOcrucible-agent (worker): commit
322025-12-22T16:23:35.666ZINFOcrucible-agent (worker): ok!
332025-12-22T16:23:35.671ZINFOcrucible-agent (worker): SMF ok!
342025-12-22T16:23:35.779ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:53943 req_id = 9e86c230-a000-4b59-a206-6b21f2ed0f7c response_code = 200 uri = /crucible/0/regions
352025-12-22T16:23:36.592ZINFOcrucible-agent (dropshot): request completed latency_us = 79 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:53943 req_id = 0190f9b6-cc1a-4e55-bb8a-2bf45009536a response_code = 200 uri = /crucible/0/regions/f82b7048-533b-46a7-9cdb-b5009fa31748
362025-12-22T16:23:36.595ZINFOcrucible-agent (datafile): path is "/data/regions/f82b7048-533b-46a7-9cdb-b5009fa31748"
372025-12-22T16:23:36.603ZINFOcrucible-agent (datafile): dataset is oxp_713fed97-1fef-4f49-b90b-b20eaba7d366/crucible/regions/f82b7048-533b-46a7-9cdb-b5009fa31748
382025-12-22T16:23:36.610ZINFOcrucible-agent (datafile): region f82b7048-533b-46a7-9cdb-b5009fa31748 state: Created -> Tombstoned
392025-12-22T16:23:36.610ZINFOcrucible-agent (dropshot): request completed latency_us = 15177 local_addr = [fd00:1122:3344:101::16]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:53943 req_id = 0452c392-6d60-44e6-b8dd-f700218f0069 response_code = 204 uri = /crucible/0/regions/f82b7048-533b-46a7-9cdb-b5009fa31748
402025-12-22T16:23:36.610ZINFOcrucible-agent (worker): applying SMF actions before removal...
412025-12-22T16:23:36.611ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-f82b7048-533b-46a7-9cdb-b5009fa31748 (instance states: (Some(Online), None))
422025-12-22T16:23:36.613ZINFOcrucible-agent (dropshot): request completed latency_us = 27 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:53943 req_id = 351345de-e921-454f-9357-b6cf3a67f2c0 response_code = 200 uri = /crucible/0/regions/f82b7048-533b-46a7-9cdb-b5009fa31748
432025-12-22T16:23:36.616ZINFOcrucible-agent (worker): SMF ok!
442025-12-22T16:23:36.623ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_713fed97-1fef-4f49-b90b-b20eaba7d366/crucible/regions/f82b7048-533b-46a7-9cdb-b5009fa31748" region = f82b7048-533b-46a7-9cdb-b5009fa31748
452025-12-22T16:23:36.691ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:53943 req_id = 7922248a-432e-436c-a594-b6b739c06c35 response_code = 200 uri = /crucible/0/regions/f82b7048-533b-46a7-9cdb-b5009fa31748
462025-12-22T16:23:36.694ZINFOcrucible-agent (datafile): region f82b7048-533b-46a7-9cdb-b5009fa31748 state: Tombstoned -> Destroyed
472025-12-22T16:23:36.824ZINFOcrucible-agent (dropshot): request completed latency_us = 33 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:53943 req_id = db03495e-0032-4639-9eaa-ddb571e0bdc3 response_code = 200 uri = /crucible/0/regions/f82b7048-533b-46a7-9cdb-b5009fa31748