Max level shown:
1 [ Mar 19 13:02:40 Enabled. ]
2 [ Mar 19 13:02:40 Rereading configuration. ]
3 [ Mar 19 13:02:41 Rereading configuration. ]
4 [ Mar 19 13:02:53 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_678e42d3-fa8e-4fcb-b96a-776196a44be7/crucible -l [fd00:1122:3344:101::1a]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62026-03-19T13:02:53.771ZINFOcrucible-agent: Crucible Version: Crucible Version: 0.0.1 Commit SHA: 7103cd3a3d7b0112d2949dd135db06fef0c156bb Commit timestamp: 2025-11-10T22:24:53.000000000Z branch: main rustc: 1.90.0 stable x86_64-unknown-illumos Cargo: x86_64-unknown-illumos Debug: false Opt level: 3
72026-03-19T13:02:53.771ZINFOcrucible-agent: dataset: "oxp_678e42d3-fa8e-4fcb-b96a-776196a44be7/crucible"
82026-03-19T13:02:53.771ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::1a]:32345
92026-03-19T13:02:53.771ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
102026-03-19T13:02:53.784ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
112026-03-19T13:02:53.949ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::1a]:32345
122026-03-19T13:13:45.740ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::b]:63639
132026-03-19T13:13:45.791ZINFOcrucible-agent (datafile): region 2dfded42-59ea-48c8-8644-96d7d1905641 state: Requested
142026-03-19T13:13:45.794ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
152026-03-19T13:13:45.794ZINFOcrucible-agent (dropshot): request completed latency_us = 20311 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:63639 req_id = c1575154-6880-42be-adb1-ff6a5f848c13 response_code = 200 uri = /crucible/0/regions
162026-03-19T13:13:45.807ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_678e42d3-fa8e-4fcb-b96a-776196a44be7/crucible/regions/2dfded42-59ea-48c8-8644-96d7d1905641
172026-03-19T13:13:45.807ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_678e42d3-fa8e-4fcb-b96a-776196a44be7/crucible/regions/2dfded42-59ea-48c8-8644-96d7d1905641
182026-03-19T13:13:45.861ZINFOcrucible-agent (worker): creating region Region { id: RegionId("2dfded42-59ea-48c8-8644-96d7d1905641"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/2dfded42-59ea-48c8-8644-96d7d1905641" region = 2dfded42-59ea-48c8-8644-96d7d1905641
192026-03-19T13:13:45.863ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "2dfded42-59ea-48c8-8644-96d7d1905641" "--data" "/data/regions/2dfded42-59ea-48c8-8644-96d7d1905641" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 2dfded42-59ea-48c8-8644-96d7d1905641
202026-03-19T13:13:46.047ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:63639 req_id = f2abb245-fd0e-447c-848d-1b023c586666 response_code = 200 uri = /crucible/0/regions
212026-03-19T13:13:46.222ZINFOcrucible-agent (worker): region files created ok region = 2dfded42-59ea-48c8-8644-96d7d1905641
222026-03-19T13:13:46.222ZINFOcrucible-agent (datafile): region 2dfded42-59ea-48c8-8644-96d7d1905641 state: Requested -> Created
232026-03-19T13:13:46.222ZINFOcrucible-agent (worker): applying SMF actions post create...
242026-03-19T13:13:46.223ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-2dfded42-59ea-48c8-8644-96d7d1905641
252026-03-19T13:13:46.236ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-2dfded42-59ea-48c8-8644-96d7d1905641
262026-03-19T13:13:46.258ZINFOcrucible-agent (worker): creating config property group
272026-03-19T13:13:46.262ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-2dfded42-59ea-48c8-8644-96d7d1905641
282026-03-19T13:13:46.263ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/2dfded42-59ea-48c8-8644-96d7d1905641
292026-03-19T13:13:46.263ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
302026-03-19T13:13:46.263ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
312026-03-19T13:13:46.263ZINFOcrucible-agent (worker): commit
322026-03-19T13:13:46.267ZINFOcrucible-agent (worker): ok!
332026-03-19T13:13:46.272ZINFOcrucible-agent (worker): SMF ok!
342026-03-19T13:13:46.370ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:63639 req_id = c89aa74e-ceb7-4ae0-9f8a-a39c2f4a8f7d response_code = 200 uri = /crucible/0/regions
352026-03-19T13:14:11.938ZINFOcrucible-agent (dropshot): request completed latency_us = 96 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:63639 req_id = fb074dab-e2e4-4a4f-9b54-1a5d3dd8d11f response_code = 200 uri = /crucible/0/regions/2dfded42-59ea-48c8-8644-96d7d1905641
362026-03-19T13:14:11.941ZINFOcrucible-agent (datafile): path is "/data/regions/2dfded42-59ea-48c8-8644-96d7d1905641"
372026-03-19T13:14:11.953ZINFOcrucible-agent (datafile): dataset is oxp_678e42d3-fa8e-4fcb-b96a-776196a44be7/crucible/regions/2dfded42-59ea-48c8-8644-96d7d1905641
382026-03-19T13:14:11.960ZINFOcrucible-agent (datafile): region 2dfded42-59ea-48c8-8644-96d7d1905641 state: Created -> Tombstoned
392026-03-19T13:14:11.960ZINFOcrucible-agent (worker): applying SMF actions before removal...
402026-03-19T13:14:11.960ZINFOcrucible-agent (dropshot): request completed latency_us = 19080 local_addr = [fd00:1122:3344:101::1a]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:63639 req_id = 4e46ee55-a991-4d77-8a95-b4fdbc82b45b response_code = 204 uri = /crucible/0/regions/2dfded42-59ea-48c8-8644-96d7d1905641
412026-03-19T13:14:11.961ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-2dfded42-59ea-48c8-8644-96d7d1905641 (instance states: (Some(Online), None))
422026-03-19T13:14:11.963ZINFOcrucible-agent (dropshot): request completed latency_us = 35 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:63639 req_id = 9d57a172-b3f4-47e2-9116-140230591d0f response_code = 200 uri = /crucible/0/regions/2dfded42-59ea-48c8-8644-96d7d1905641
432026-03-19T13:14:12.019ZINFOcrucible-agent (worker): SMF ok!
442026-03-19T13:14:12.027ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_678e42d3-fa8e-4fcb-b96a-776196a44be7/crucible/regions/2dfded42-59ea-48c8-8644-96d7d1905641" region = 2dfded42-59ea-48c8-8644-96d7d1905641
452026-03-19T13:14:12.028ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:63639 req_id = e83f7f50-40bd-4dc3-8be1-54d93a01133a response_code = 200 uri = /crucible/0/regions/2dfded42-59ea-48c8-8644-96d7d1905641
462026-03-19T13:14:12.122ZINFOcrucible-agent (datafile): region 2dfded42-59ea-48c8-8644-96d7d1905641 state: Tombstoned -> Destroyed
472026-03-19T13:14:12.146ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:63639 req_id = 6161e447-cc2b-487f-8a63-f8e5261ecfb8 response_code = 200 uri = /crucible/0/regions/2dfded42-59ea-48c8-8644-96d7d1905641