Max level shown:
1 [ Mar 30 23:26:37 Enabled. ]
2 [ Mar 30 23:26:37 Rereading configuration. ]
3 [ Mar 30 23:26:38 Rereading configuration. ]
4 [ Mar 30 23:26:41 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_6797e705-a9d8-4257-8abd-3a6b647ed68b/crucible -l [fd00:1122:3344:101::16]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62026-03-30T23:26:41.525ZINFOcrucible-agent: Crucible Version: Crucible Version: 0.0.1 Commit SHA: 7103cd3a3d7b0112d2949dd135db06fef0c156bb Commit timestamp: 2025-11-10T22:24:53.000000000Z branch: main rustc: 1.90.0 stable x86_64-unknown-illumos Cargo: x86_64-unknown-illumos Debug: false Opt level: 3
72026-03-30T23:26:41.526ZINFOcrucible-agent: dataset: "oxp_6797e705-a9d8-4257-8abd-3a6b647ed68b/crucible"
82026-03-30T23:26:41.526ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::16]:32345
92026-03-30T23:26:41.526ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
102026-03-30T23:26:41.544ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
112026-03-30T23:26:41.634ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::16]:32345
122026-03-30T23:27:22.023ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:33416
132026-03-30T23:27:22.049ZINFOcrucible-agent (datafile): region a568476c-d977-461a-b6af-9a9f43cb1367 state: Requested
142026-03-30T23:27:22.052ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152026-03-30T23:27:22.052ZINFOcrucible-agent (dropshot): request completed latency_us = 14591 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33416 req_id = eb042984-f8fb-4ff7-b2da-a11cfe3a2c13 response_code = 200 uri = /crucible/0/regions
162026-03-30T23:27:22.063ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_6797e705-a9d8-4257-8abd-3a6b647ed68b/crucible/regions/a568476c-d977-461a-b6af-9a9f43cb1367
172026-03-30T23:27:22.063ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_6797e705-a9d8-4257-8abd-3a6b647ed68b/crucible/regions/a568476c-d977-461a-b6af-9a9f43cb1367
182026-03-30T23:27:22.149ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a568476c-d977-461a-b6af-9a9f43cb1367"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/a568476c-d977-461a-b6af-9a9f43cb1367" region = a568476c-d977-461a-b6af-9a9f43cb1367
192026-03-30T23:27:22.151ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "a568476c-d977-461a-b6af-9a9f43cb1367" "--data" "/data/regions/a568476c-d977-461a-b6af-9a9f43cb1367" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = a568476c-d977-461a-b6af-9a9f43cb1367
202026-03-30T23:27:22.317ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33416 req_id = a9f99998-cf6c-4e3a-bd9a-0da6c3b8771a response_code = 200 uri = /crucible/0/regions
212026-03-30T23:27:22.546ZINFOcrucible-agent (worker): region files created ok region = a568476c-d977-461a-b6af-9a9f43cb1367
222026-03-30T23:27:22.547ZINFOcrucible-agent (datafile): region a568476c-d977-461a-b6af-9a9f43cb1367 state: Requested -> Created
232026-03-30T23:27:22.547ZINFOcrucible-agent (worker): applying SMF actions post create...
242026-03-30T23:27:22.547ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a568476c-d977-461a-b6af-9a9f43cb1367
252026-03-30T23:27:22.557ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a568476c-d977-461a-b6af-9a9f43cb1367
262026-03-30T23:27:22.583ZINFOcrucible-agent (worker): creating config property group
272026-03-30T23:27:22.591ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a568476c-d977-461a-b6af-9a9f43cb1367
282026-03-30T23:27:22.591ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a568476c-d977-461a-b6af-9a9f43cb1367
292026-03-30T23:27:22.591ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
302026-03-30T23:27:22.591ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
312026-03-30T23:27:22.591ZINFOcrucible-agent (worker): commit
322026-03-30T23:27:22.597ZINFOcrucible-agent (worker): ok!
332026-03-30T23:27:22.603ZINFOcrucible-agent (worker): SMF ok!
342026-03-30T23:27:23.050ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33416 req_id = a78bdbc6-09ef-4e86-8811-e0de0634c856 response_code = 200 uri = /crucible/0/regions
352026-03-30T23:27:24.223ZINFOcrucible-agent (dropshot): request completed latency_us = 95 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33416 req_id = 447f20c9-460d-47b1-8a49-890a0abae1cc response_code = 200 uri = /crucible/0/regions/a568476c-d977-461a-b6af-9a9f43cb1367
362026-03-30T23:27:24.226ZINFOcrucible-agent (datafile): path is "/data/regions/a568476c-d977-461a-b6af-9a9f43cb1367"
372026-03-30T23:27:24.236ZINFOcrucible-agent (datafile): dataset is oxp_6797e705-a9d8-4257-8abd-3a6b647ed68b/crucible/regions/a568476c-d977-461a-b6af-9a9f43cb1367
382026-03-30T23:27:24.245ZINFOcrucible-agent (datafile): region a568476c-d977-461a-b6af-9a9f43cb1367 state: Created -> Tombstoned
392026-03-30T23:27:24.245ZINFOcrucible-agent (worker): applying SMF actions before removal...
402026-03-30T23:27:24.245ZINFOcrucible-agent (dropshot): request completed latency_us = 18602 local_addr = [fd00:1122:3344:101::16]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:33416 req_id = 90bcdd90-a3cb-4c9e-8e81-970ed08a6cbf response_code = 204 uri = /crucible/0/regions/a568476c-d977-461a-b6af-9a9f43cb1367
412026-03-30T23:27:24.246ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-a568476c-d977-461a-b6af-9a9f43cb1367 (instance states: (Some(Online), None))
422026-03-30T23:27:24.250ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33416 req_id = a766ac93-b2be-4c19-827a-de9c18f9fa1f response_code = 200 uri = /crucible/0/regions/a568476c-d977-461a-b6af-9a9f43cb1367
432026-03-30T23:27:24.256ZINFOcrucible-agent (worker): SMF ok!
442026-03-30T23:27:24.265ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_6797e705-a9d8-4257-8abd-3a6b647ed68b/crucible/regions/a568476c-d977-461a-b6af-9a9f43cb1367" region = a568476c-d977-461a-b6af-9a9f43cb1367
452026-03-30T23:27:24.353ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33416 req_id = 8b499c75-30a2-441f-89af-2549966d216a response_code = 200 uri = /crucible/0/regions/a568476c-d977-461a-b6af-9a9f43cb1367
462026-03-30T23:27:24.385ZINFOcrucible-agent (datafile): region a568476c-d977-461a-b6af-9a9f43cb1367 state: Tombstoned -> Destroyed
472026-03-30T23:27:24.505ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33416 req_id = 3da9199e-4462-42be-a0af-8f16c7014cc8 response_code = 200 uri = /crucible/0/regions/a568476c-d977-461a-b6af-9a9f43cb1367