Max level shown:
1 [ Jul 4 15:31:18 Enabled. ]
2 [ Jul 4 15:31:18 Rereading configuration. ]
3 [ Jul 4 15:31:22 Rereading configuration. ]
4 [ Jul 4 15:31:28 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_087771a7-5c8b-4cbb-a799-90cb06830f64/crucible -l [fd00:1122:3344:101::13]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62026-07-04T15:31:28.352ZINFOcrucible-agent: Crucible Version: Crucible Version: 0.0.1 Commit SHA: 2bfe090eb5318ec8c467157018db2429d4df535b Commit timestamp: 2026-06-25T19:33:01.000000000Z branch: main rustc: 1.90.0 stable x86_64-unknown-illumos Cargo: x86_64-unknown-illumos Debug: false Opt level: 3
72026-07-04T15:31:28.352ZINFOcrucible-agent: dataset: "oxp_087771a7-5c8b-4cbb-a799-90cb06830f64/crucible"
82026-07-04T15:31:28.352ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::13]:32345
92026-07-04T15:31:28.352ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
102026-07-04T15:31:28.368ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
112026-07-04T15:31:28.558ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::13]:32345
122026-07-04T15:43:18.730ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::c]:51313
132026-07-04T15:43:18.803ZINFOcrucible-agent (datafile): region 932ecd61-c07b-475f-ac6c-e15001b0ee1e state: Requested
142026-07-04T15:43:18.828ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
152026-07-04T15:43:18.831ZINFOcrucible-agent (dropshot): request completed latency_us = 43680 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:51313 req_id = 91376c48-1f47-4e57-a9f3-0493f14db140 response_code = 200 uri = /crucible/0/regions
162026-07-04T15:43:18.838ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_087771a7-5c8b-4cbb-a799-90cb06830f64/crucible/regions/932ecd61-c07b-475f-ac6c-e15001b0ee1e
172026-07-04T15:43:18.838ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_087771a7-5c8b-4cbb-a799-90cb06830f64/crucible/regions/932ecd61-c07b-475f-ac6c-e15001b0ee1e
182026-07-04T15:43:18.983ZINFOcrucible-agent (worker): creating region Region { id: RegionId("932ecd61-c07b-475f-ac6c-e15001b0ee1e"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/932ecd61-c07b-475f-ac6c-e15001b0ee1e" region = 932ecd61-c07b-475f-ac6c-e15001b0ee1e
192026-07-04T15:43:18.985ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "932ecd61-c07b-475f-ac6c-e15001b0ee1e" "--data" "/data/regions/932ecd61-c07b-475f-ac6c-e15001b0ee1e" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 932ecd61-c07b-475f-ac6c-e15001b0ee1e
202026-07-04T15:43:19.102ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:51313 req_id = 91743d79-e46e-4766-a94a-389598c1edd0 response_code = 200 uri = /crucible/0/regions
212026-07-04T15:43:19.757ZINFOcrucible-agent (dropshot): request completed latency_us = 139 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:51313 req_id = 93a43b9c-5dd3-4f18-b280-8e31570a14e9 response_code = 200 uri = /crucible/0/regions
222026-07-04T15:43:20.010ZINFOcrucible-agent (worker): region files created ok region = 932ecd61-c07b-475f-ac6c-e15001b0ee1e
232026-07-04T15:43:20.012ZINFOcrucible-agent (datafile): region 932ecd61-c07b-475f-ac6c-e15001b0ee1e state: Requested -> Created
242026-07-04T15:43:20.012ZINFOcrucible-agent (worker): applying SMF actions post create...
252026-07-04T15:43:20.015ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-932ecd61-c07b-475f-ac6c-e15001b0ee1e
262026-07-04T15:43:20.062ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-932ecd61-c07b-475f-ac6c-e15001b0ee1e
272026-07-04T15:43:20.141ZINFOcrucible-agent (worker): creating config property group
282026-07-04T15:43:20.200ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-932ecd61-c07b-475f-ac6c-e15001b0ee1e
292026-07-04T15:43:20.201ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/932ecd61-c07b-475f-ac6c-e15001b0ee1e
302026-07-04T15:43:20.203ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
312026-07-04T15:43:20.203ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
322026-07-04T15:43:20.203ZINFOcrucible-agent (worker): commit
332026-07-04T15:43:20.277ZINFOcrucible-agent (worker): ok!
342026-07-04T15:43:20.299ZINFOcrucible-agent (worker): SMF ok!
352026-07-04T15:43:20.734ZINFOcrucible-agent (dropshot): request completed latency_us = 88 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:51313 req_id = 28373c77-c12f-4fe2-a847-d72c2cf6c16b response_code = 200 uri = /crucible/0/regions
362026-07-04T15:43:51.684ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::c]:59546
372026-07-04T15:43:51.684ZINFOcrucible-agent (dropshot): request completed latency_us = 42 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:59546 req_id = 6506f2cb-e9d6-4a14-8313-0221f9435464 response_code = 200 uri = /crucible/0/regions/932ecd61-c07b-475f-ac6c-e15001b0ee1e
382026-07-04T15:43:51.688ZINFOcrucible-agent (datafile): path is "/data/regions/932ecd61-c07b-475f-ac6c-e15001b0ee1e"
392026-07-04T15:43:51.698ZINFOcrucible-agent (datafile): dataset is oxp_087771a7-5c8b-4cbb-a799-90cb06830f64/crucible/regions/932ecd61-c07b-475f-ac6c-e15001b0ee1e
402026-07-04T15:43:51.705ZINFOcrucible-agent (datafile): region 932ecd61-c07b-475f-ac6c-e15001b0ee1e state: Created -> Tombstoned
412026-07-04T15:43:51.705ZINFOcrucible-agent (worker): applying SMF actions before removal...
422026-07-04T15:43:51.705ZINFOcrucible-agent (dropshot): request completed latency_us = 16841 local_addr = [fd00:1122:3344:101::13]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:59546 req_id = cd1ac0ee-29b2-471c-a0f3-bfc587c77f9f response_code = 204 uri = /crucible/0/regions/932ecd61-c07b-475f-ac6c-e15001b0ee1e
432026-07-04T15:43:51.705ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-932ecd61-c07b-475f-ac6c-e15001b0ee1e (instance states: (Some(Online), None))
442026-07-04T15:43:51.709ZINFOcrucible-agent (dropshot): request completed latency_us = 52 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:59546 req_id = cc8955a6-9ff4-4ed4-abf1-4ac7120fb861 response_code = 200 uri = /crucible/0/regions/932ecd61-c07b-475f-ac6c-e15001b0ee1e
452026-07-04T15:43:51.711ZINFOcrucible-agent (worker): SMF ok!
462026-07-04T15:43:51.719ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_087771a7-5c8b-4cbb-a799-90cb06830f64/crucible/regions/932ecd61-c07b-475f-ac6c-e15001b0ee1e" region = 932ecd61-c07b-475f-ac6c-e15001b0ee1e
472026-07-04T15:43:51.776ZINFOcrucible-agent (datafile): region 932ecd61-c07b-475f-ac6c-e15001b0ee1e state: Tombstoned -> Destroyed
482026-07-04T15:43:51.843ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:59546 req_id = d1d3e3a6-bdeb-41bf-b87b-f0be05e5c203 response_code = 200 uri = /crucible/0/regions/932ecd61-c07b-475f-ac6c-e15001b0ee1e