Max level shown:
1 [ Jan 10 02:48:59 Enabled. ]
2 [ Jan 10 02:48:59 Rereading configuration. ]
3 [ Jan 10 02:49:00 Rereading configuration. ]
4 [ Jan 10 02:49:05 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_1aaf211b-cfc3-4059-83dc-7ba3dd5121ff/crucible -l [fd00:1122:3344:101::1a]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62026-01-10T02:49:05.956ZINFOcrucible-agent: Crucible Version: Crucible Version: 0.0.1 Commit SHA: 7103cd3a3d7b0112d2949dd135db06fef0c156bb Commit timestamp: 2025-11-10T22:24:53.000000000Z branch: main rustc: 1.90.0 stable x86_64-unknown-illumos Cargo: x86_64-unknown-illumos Debug: false Opt level: 3
72026-01-10T02:49:05.957ZINFOcrucible-agent: dataset: "oxp_1aaf211b-cfc3-4059-83dc-7ba3dd5121ff/crucible"
82026-01-10T02:49:05.957ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::1a]:32345
92026-01-10T02:49:05.957ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
102026-01-10T02:49:05.970ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
112026-01-10T02:49:06.119ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::1a]:32345
122026-01-10T02:49:50.267ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::c]:41548
132026-01-10T02:49:50.295ZINFOcrucible-agent (datafile): region b0f88ffc-9d28-4142-849e-8361a6abeb73 state: Requested
142026-01-10T02:49:50.298ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152026-01-10T02:49:50.298ZINFOcrucible-agent (dropshot): request completed latency_us = 16137 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:41548 req_id = 556768de-8cef-4d5f-ac16-f1c3818a2977 response_code = 200 uri = /crucible/0/regions
162026-01-10T02:49:50.308ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_1aaf211b-cfc3-4059-83dc-7ba3dd5121ff/crucible/regions/b0f88ffc-9d28-4142-849e-8361a6abeb73
172026-01-10T02:49:50.308ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_1aaf211b-cfc3-4059-83dc-7ba3dd5121ff/crucible/regions/b0f88ffc-9d28-4142-849e-8361a6abeb73
182026-01-10T02:49:50.387ZINFOcrucible-agent (worker): creating region Region { id: RegionId("b0f88ffc-9d28-4142-849e-8361a6abeb73"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/b0f88ffc-9d28-4142-849e-8361a6abeb73" region = b0f88ffc-9d28-4142-849e-8361a6abeb73
192026-01-10T02:49:50.389ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "b0f88ffc-9d28-4142-849e-8361a6abeb73" "--data" "/data/regions/b0f88ffc-9d28-4142-849e-8361a6abeb73" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = b0f88ffc-9d28-4142-849e-8361a6abeb73
202026-01-10T02:49:50.653ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:41548 req_id = 6267665d-cb42-4db4-960b-54034b5bc1ea response_code = 200 uri = /crucible/0/regions
212026-01-10T02:49:50.735ZINFOcrucible-agent (worker): region files created ok region = b0f88ffc-9d28-4142-849e-8361a6abeb73
222026-01-10T02:49:50.735ZINFOcrucible-agent (datafile): region b0f88ffc-9d28-4142-849e-8361a6abeb73 state: Requested -> Created
232026-01-10T02:49:50.735ZINFOcrucible-agent (worker): applying SMF actions post create...
242026-01-10T02:49:50.735ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-b0f88ffc-9d28-4142-849e-8361a6abeb73
252026-01-10T02:49:50.742ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-b0f88ffc-9d28-4142-849e-8361a6abeb73
262026-01-10T02:49:50.759ZINFOcrucible-agent (worker): creating config property group
272026-01-10T02:49:50.763ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-b0f88ffc-9d28-4142-849e-8361a6abeb73
282026-01-10T02:49:50.763ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b0f88ffc-9d28-4142-849e-8361a6abeb73
292026-01-10T02:49:50.763ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
302026-01-10T02:49:50.763ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
312026-01-10T02:49:50.763ZINFOcrucible-agent (worker): commit
322026-01-10T02:49:50.767ZINFOcrucible-agent (worker): ok!
332026-01-10T02:49:50.772ZINFOcrucible-agent (worker): SMF ok!
342026-01-10T02:49:51.312ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:41548 req_id = 7bc174ae-2a34-4d7b-bb7a-fb3e60b94501 response_code = 200 uri = /crucible/0/regions
352026-01-10T02:49:51.806ZINFOcrucible-agent (dropshot): request completed latency_us = 76 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:41548 req_id = 718e318a-e39a-4a52-b7ac-ed5a1082cd74 response_code = 200 uri = /crucible/0/regions/b0f88ffc-9d28-4142-849e-8361a6abeb73
362026-01-10T02:49:51.809ZINFOcrucible-agent (datafile): path is "/data/regions/b0f88ffc-9d28-4142-849e-8361a6abeb73"
372026-01-10T02:49:51.818ZINFOcrucible-agent (datafile): dataset is oxp_1aaf211b-cfc3-4059-83dc-7ba3dd5121ff/crucible/regions/b0f88ffc-9d28-4142-849e-8361a6abeb73
382026-01-10T02:49:51.826ZINFOcrucible-agent (datafile): region b0f88ffc-9d28-4142-849e-8361a6abeb73 state: Created -> Tombstoned
392026-01-10T02:49:51.826ZINFOcrucible-agent (dropshot): request completed latency_us = 16718 local_addr = [fd00:1122:3344:101::1a]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:41548 req_id = a28b6569-163a-4dae-a0bf-70afddeb0378 response_code = 204 uri = /crucible/0/regions/b0f88ffc-9d28-4142-849e-8361a6abeb73
402026-01-10T02:49:51.826ZINFOcrucible-agent (worker): applying SMF actions before removal...
412026-01-10T02:49:51.827ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-b0f88ffc-9d28-4142-849e-8361a6abeb73 (instance states: (Some(Online), None))
422026-01-10T02:49:51.831ZINFOcrucible-agent (dropshot): request completed latency_us = 26 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:41548 req_id = 03ade096-b097-4885-96fd-ee83bd58568d response_code = 200 uri = /crucible/0/regions/b0f88ffc-9d28-4142-849e-8361a6abeb73
432026-01-10T02:49:51.832ZINFOcrucible-agent (worker): SMF ok!
442026-01-10T02:49:51.841ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_1aaf211b-cfc3-4059-83dc-7ba3dd5121ff/crucible/regions/b0f88ffc-9d28-4142-849e-8361a6abeb73" region = b0f88ffc-9d28-4142-849e-8361a6abeb73
452026-01-10T02:49:51.911ZINFOcrucible-agent (datafile): region b0f88ffc-9d28-4142-849e-8361a6abeb73 state: Tombstoned -> Destroyed
462026-01-10T02:49:51.979ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:41548 req_id = dcb87948-a0e8-4da1-b2b9-6039ea332dbc response_code = 200 uri = /crucible/0/regions/b0f88ffc-9d28-4142-849e-8361a6abeb73