Max level shown:
1 [ May 23 04:50:04 Enabled. ]
2 [ May 23 04:50:04 Rereading configuration. ]
3 [ May 23 04:50:07 Rereading configuration. ]
4 [ May 23 04:50:12 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_eeceeb18-81e3-48cf-b5ca-b99b22951510/crucible -l [fd00:1122:3344:101::17]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62026-05-23T04:50:12.758ZINFOcrucible-agent: Crucible Version: Crucible Version: 0.0.1 Commit SHA: 3c1708d86e10f0370807388a1efe092edd99d431 Commit timestamp: 2026-04-27T19:14:23.000000000Z branch: main rustc: 1.90.0 stable x86_64-unknown-illumos Cargo: x86_64-unknown-illumos Debug: false Opt level: 3
72026-05-23T04:50:12.759ZINFOcrucible-agent: dataset: "oxp_eeceeb18-81e3-48cf-b5ca-b99b22951510/crucible"
82026-05-23T04:50:12.759ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::17]:32345
92026-05-23T04:50:12.759ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
102026-05-23T04:50:12.769ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
112026-05-23T04:50:13.003ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::17]:32345
122026-05-23T04:50:39.444ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::c]:65240
132026-05-23T04:50:39.490ZINFOcrucible-agent (datafile): region b873cacc-2e2c-4227-bb55-19d8c2cdd553 state: Requested
142026-05-23T04:50:39.491ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152026-05-23T04:50:39.491ZINFOcrucible-agent (dropshot): request completed latency_us = 29171 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:65240 req_id = 041b6f40-af23-4761-a25b-5a462f192b23 response_code = 200 uri = /crucible/0/regions
162026-05-23T04:50:39.498ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_eeceeb18-81e3-48cf-b5ca-b99b22951510/crucible/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553
172026-05-23T04:50:39.498ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_eeceeb18-81e3-48cf-b5ca-b99b22951510/crucible/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553
182026-05-23T04:50:39.581ZINFOcrucible-agent (worker): creating region Region { id: RegionId("b873cacc-2e2c-4227-bb55-19d8c2cdd553"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553" region = b873cacc-2e2c-4227-bb55-19d8c2cdd553
192026-05-23T04:50:39.581ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "b873cacc-2e2c-4227-bb55-19d8c2cdd553" "--data" "/data/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = b873cacc-2e2c-4227-bb55-19d8c2cdd553
202026-05-23T04:50:39.824ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:65240 req_id = c7be6526-ce9e-49be-a1af-c882d7d4d7b5 response_code = 200 uri = /crucible/0/regions
212026-05-23T04:50:39.887ZINFOcrucible-agent (worker): region files created ok region = b873cacc-2e2c-4227-bb55-19d8c2cdd553
222026-05-23T04:50:39.887ZINFOcrucible-agent (datafile): region b873cacc-2e2c-4227-bb55-19d8c2cdd553 state: Requested -> Created
232026-05-23T04:50:39.887ZINFOcrucible-agent (worker): applying SMF actions post create...
242026-05-23T04:50:39.890ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-b873cacc-2e2c-4227-bb55-19d8c2cdd553
252026-05-23T04:50:39.895ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-b873cacc-2e2c-4227-bb55-19d8c2cdd553
262026-05-23T04:50:39.910ZINFOcrucible-agent (worker): creating config property group
272026-05-23T04:50:39.917ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-b873cacc-2e2c-4227-bb55-19d8c2cdd553
282026-05-23T04:50:39.918ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553
292026-05-23T04:50:39.920ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
302026-05-23T04:50:39.920ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
312026-05-23T04:50:39.921ZINFOcrucible-agent (worker): commit
322026-05-23T04:50:39.934ZINFOcrucible-agent (worker): ok!
332026-05-23T04:50:39.938ZINFOcrucible-agent (worker): SMF ok!
342026-05-23T04:50:40.510ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:65240 req_id = c6f4b384-7b99-4772-8614-c0d37e377bab response_code = 200 uri = /crucible/0/regions
352026-05-23T04:50:41.169ZINFOcrucible-agent (dropshot): request completed latency_us = 2417 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:65240 req_id = 5a0901e5-142a-49a3-8648-8c0741386f9d response_code = 200 uri = /crucible/0/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553
362026-05-23T04:50:41.173ZINFOcrucible-agent (datafile): path is "/data/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553"
372026-05-23T04:50:41.181ZINFOcrucible-agent (datafile): dataset is oxp_eeceeb18-81e3-48cf-b5ca-b99b22951510/crucible/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553
382026-05-23T04:50:41.189ZINFOcrucible-agent (datafile): region b873cacc-2e2c-4227-bb55-19d8c2cdd553 state: Created -> Tombstoned
392026-05-23T04:50:41.189ZINFOcrucible-agent (worker): applying SMF actions before removal...
402026-05-23T04:50:41.189ZINFOcrucible-agent (dropshot): request completed latency_us = 15967 local_addr = [fd00:1122:3344:101::17]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:65240 req_id = 5fd3f45a-dc2b-4c6f-a0a5-90f6b32f520b response_code = 204 uri = /crucible/0/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553
412026-05-23T04:50:41.190ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-b873cacc-2e2c-4227-bb55-19d8c2cdd553 (instance states: (Some(Online), None))
422026-05-23T04:50:41.195ZINFOcrucible-agent (dropshot): request completed latency_us = 23 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:65240 req_id = fccb0d19-0156-4587-a607-e46efc52d7f0 response_code = 200 uri = /crucible/0/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553
432026-05-23T04:50:41.202ZINFOcrucible-agent (worker): SMF ok!
442026-05-23T04:50:41.215ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_eeceeb18-81e3-48cf-b5ca-b99b22951510/crucible/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553" region = b873cacc-2e2c-4227-bb55-19d8c2cdd553
452026-05-23T04:50:41.296ZINFOcrucible-agent (dropshot): request completed latency_us = 34 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:65240 req_id = 7382792b-c960-47c8-b21e-06d3f802c168 response_code = 200 uri = /crucible/0/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553
462026-05-23T04:50:41.319ZINFOcrucible-agent (datafile): region b873cacc-2e2c-4227-bb55-19d8c2cdd553 state: Tombstoned -> Destroyed
472026-05-23T04:50:41.438ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:65240 req_id = c3d138ee-60ce-4d2b-8a21-3832411628cd response_code = 200 uri = /crucible/0/regions/b873cacc-2e2c-4227-bb55-19d8c2cdd553