Max level shown:
1 [ Mar 19 07:54:17 Enabled. ]
2 [ Mar 19 07:54:17 Rereading configuration. ]
3 [ Mar 19 07:54:18 Rereading configuration. ]
4 [ Mar 19 07:54:22 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_6e86e10b-c123-443c-9e72-3ece6cd8c62b/crucible -l [fd00:1122:3344:101::19]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62026-03-19T07:54:22.213ZINFOcrucible-agent: Crucible Version: Crucible Version: 0.0.1 Commit SHA: 7103cd3a3d7b0112d2949dd135db06fef0c156bb Commit timestamp: 2025-11-10T22:24:53.000000000Z branch: main rustc: 1.90.0 stable x86_64-unknown-illumos Cargo: x86_64-unknown-illumos Debug: false Opt level: 3
72026-03-19T07:54:22.214ZINFOcrucible-agent: dataset: "oxp_6e86e10b-c123-443c-9e72-3ece6cd8c62b/crucible"
82026-03-19T07:54:22.214ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::19]:32345
92026-03-19T07:54:22.214ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
102026-03-19T07:54:22.230ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
112026-03-19T07:54:22.298ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::19]:32345
122026-03-19T07:55:31.700ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:42641
132026-03-19T07:55:31.736ZINFOcrucible-agent (datafile): region d3c72131-a807-42ba-b225-764adcba2ac7 state: Requested
142026-03-19T07:55:31.739ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152026-03-19T07:55:31.739ZINFOcrucible-agent (dropshot): request completed latency_us = 12066 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42641 req_id = e9548760-956b-4261-812f-1f588700c6b2 response_code = 200 uri = /crucible/0/regions
162026-03-19T07:55:31.748ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_6e86e10b-c123-443c-9e72-3ece6cd8c62b/crucible/regions/d3c72131-a807-42ba-b225-764adcba2ac7
172026-03-19T07:55:31.748ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_6e86e10b-c123-443c-9e72-3ece6cd8c62b/crucible/regions/d3c72131-a807-42ba-b225-764adcba2ac7
182026-03-19T07:55:31.829ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d3c72131-a807-42ba-b225-764adcba2ac7"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d3c72131-a807-42ba-b225-764adcba2ac7" region = d3c72131-a807-42ba-b225-764adcba2ac7
192026-03-19T07:55:31.832ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d3c72131-a807-42ba-b225-764adcba2ac7" "--data" "/data/regions/d3c72131-a807-42ba-b225-764adcba2ac7" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = d3c72131-a807-42ba-b225-764adcba2ac7
202026-03-19T07:55:31.878ZINFOcrucible-agent (dropshot): request completed latency_us = 83 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42641 req_id = f52e5d1e-dea8-4794-a3a5-870c0afc0676 response_code = 200 uri = /crucible/0/regions
212026-03-19T07:55:32.007ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::b]:61217
222026-03-19T07:55:32.008ZINFOcrucible-agent (dropshot): request completed latency_us = 43 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:61217 req_id = 608402f6-1fb8-4b4d-bcf1-9e025af49f7c response_code = 200 uri = /crucible/0/regions/d3c72131-a807-42ba-b225-764adcba2ac7
232026-03-19T07:55:32.195ZINFOcrucible-agent (worker): region files created ok region = d3c72131-a807-42ba-b225-764adcba2ac7
242026-03-19T07:55:32.195ZINFOcrucible-agent (datafile): region d3c72131-a807-42ba-b225-764adcba2ac7 state: Requested -> Created
252026-03-19T07:55:32.195ZINFOcrucible-agent (worker): applying SMF actions post create...
262026-03-19T07:55:32.196ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d3c72131-a807-42ba-b225-764adcba2ac7
272026-03-19T07:55:32.203ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d3c72131-a807-42ba-b225-764adcba2ac7
282026-03-19T07:55:32.218ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42641 req_id = 8ffada3f-5b7a-46cc-a5b9-78ce31e30aca response_code = 200 uri = /crucible/0/regions
292026-03-19T07:55:32.226ZINFOcrucible-agent (worker): creating config property group
302026-03-19T07:55:32.231ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d3c72131-a807-42ba-b225-764adcba2ac7
312026-03-19T07:55:32.232ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d3c72131-a807-42ba-b225-764adcba2ac7
322026-03-19T07:55:32.232ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
332026-03-19T07:55:32.232ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
342026-03-19T07:55:32.232ZINFOcrucible-agent (worker): commit
352026-03-19T07:55:32.239ZINFOcrucible-agent (worker): ok!
362026-03-19T07:55:32.243ZINFOcrucible-agent (worker): SMF ok!
372026-03-19T07:55:33.355ZINFOcrucible-agent (dropshot): request completed latency_us = 70 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42641 req_id = cfd1e098-333e-4d9c-a590-a166018c6476 response_code = 200 uri = /crucible/0/regions/d3c72131-a807-42ba-b225-764adcba2ac7
382026-03-19T07:55:33.364ZINFOcrucible-agent (datafile): path is "/data/regions/d3c72131-a807-42ba-b225-764adcba2ac7"
392026-03-19T07:55:33.374ZINFOcrucible-agent (datafile): dataset is oxp_6e86e10b-c123-443c-9e72-3ece6cd8c62b/crucible/regions/d3c72131-a807-42ba-b225-764adcba2ac7
402026-03-19T07:55:33.384ZINFOcrucible-agent (datafile): region d3c72131-a807-42ba-b225-764adcba2ac7 state: Created -> Tombstoned
412026-03-19T07:55:33.384ZINFOcrucible-agent (worker): applying SMF actions before removal...
422026-03-19T07:55:33.384ZINFOcrucible-agent (dropshot): request completed latency_us = 20456 local_addr = [fd00:1122:3344:101::19]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:42641 req_id = ac67f139-d659-4b6d-8943-b439553f0e2f response_code = 204 uri = /crucible/0/regions/d3c72131-a807-42ba-b225-764adcba2ac7
432026-03-19T07:55:33.385ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-d3c72131-a807-42ba-b225-764adcba2ac7 (instance states: (Some(Online), None))
442026-03-19T07:55:33.388ZINFOcrucible-agent (dropshot): request completed latency_us = 46 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42641 req_id = 49173b34-a9dc-423d-9e3f-63bbe33a9328 response_code = 200 uri = /crucible/0/regions/d3c72131-a807-42ba-b225-764adcba2ac7
452026-03-19T07:55:33.395ZINFOcrucible-agent (worker): SMF ok!
462026-03-19T07:55:33.406ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_6e86e10b-c123-443c-9e72-3ece6cd8c62b/crucible/regions/d3c72131-a807-42ba-b225-764adcba2ac7" region = d3c72131-a807-42ba-b225-764adcba2ac7
472026-03-19T07:55:33.484ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42641 req_id = 18e4f450-19f6-47f5-91d5-ec9612a12807 response_code = 200 uri = /crucible/0/regions/d3c72131-a807-42ba-b225-764adcba2ac7
482026-03-19T07:55:33.499ZINFOcrucible-agent (datafile): region d3c72131-a807-42ba-b225-764adcba2ac7 state: Tombstoned -> Destroyed
492026-03-19T07:55:33.660ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42641 req_id = d7c8b832-b68e-40f3-a676-c41686b57a38 response_code = 200 uri = /crucible/0/regions/d3c72131-a807-42ba-b225-764adcba2ac7