Max level shown:
1 [ Jun 29 20:30:11 Enabled. ]
2 [ Jun 29 20:30:11 Rereading configuration. ]
3 [ Jun 29 20:30:14 Rereading configuration. ]
4 [ Jun 29 20:30:18 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_3fff747b-18d8-4456-8204-c83c507de802/crucible -l [fd00:1122:3344:101::13]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62026-06-29T20:30:18.862ZINFOcrucible-agent: Crucible Version: Crucible Version: 0.0.1 Commit SHA: 2bfe090eb5318ec8c467157018db2429d4df535b Commit timestamp: 2026-06-25T19:33:01.000000000Z branch: main rustc: 1.90.0 stable x86_64-unknown-illumos Cargo: x86_64-unknown-illumos Debug: false Opt level: 3
72026-06-29T20:30:18.862ZINFOcrucible-agent: dataset: "oxp_3fff747b-18d8-4456-8204-c83c507de802/crucible"
82026-06-29T20:30:18.862ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::13]:32345
92026-06-29T20:30:18.862ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
102026-06-29T20:30:18.877ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
112026-06-29T20:30:18.978ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::13]:32345
122026-06-29T20:42:34.673ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::c]:40827
132026-06-29T20:42:34.752ZINFOcrucible-agent (datafile): region 731d8c7b-bfe5-4e10-84f5-819e14e5bfbc state: Requested
142026-06-29T20:42:34.765ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
152026-06-29T20:42:34.767ZINFOcrucible-agent (dropshot): request completed latency_us = 25809 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:40827 req_id = 27594bc0-477b-43e6-8abd-e3031bf4fabb response_code = 200 uri = /crucible/0/regions
162026-06-29T20:42:34.788ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_3fff747b-18d8-4456-8204-c83c507de802/crucible/regions/731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
172026-06-29T20:42:34.788ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_3fff747b-18d8-4456-8204-c83c507de802/crucible/regions/731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
182026-06-29T20:42:34.916ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:40827 req_id = b9056e67-5500-460e-b0b5-c12a89123d87 response_code = 200 uri = /crucible/0/regions
192026-06-29T20:42:34.932ZINFOcrucible-agent (worker): creating region Region { id: RegionId("731d8c7b-bfe5-4e10-84f5-819e14e5bfbc"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/731d8c7b-bfe5-4e10-84f5-819e14e5bfbc" region = 731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
202026-06-29T20:42:34.934ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "731d8c7b-bfe5-4e10-84f5-819e14e5bfbc" "--data" "/data/regions/731d8c7b-bfe5-4e10-84f5-819e14e5bfbc" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
212026-06-29T20:42:35.629ZINFOcrucible-agent (dropshot): request completed latency_us = 69 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:40827 req_id = 1f20c0d8-1a18-4a30-8b01-efab8d4a09cb response_code = 200 uri = /crucible/0/regions
222026-06-29T20:42:35.723ZINFOcrucible-agent (worker): region files created ok region = 731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
232026-06-29T20:42:35.725ZINFOcrucible-agent (datafile): region 731d8c7b-bfe5-4e10-84f5-819e14e5bfbc state: Requested -> Created
242026-06-29T20:42:35.725ZINFOcrucible-agent (worker): applying SMF actions post create...
252026-06-29T20:42:35.728ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
262026-06-29T20:42:35.749ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
272026-06-29T20:42:35.776ZINFOcrucible-agent (worker): creating config property group
282026-06-29T20:42:35.788ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
292026-06-29T20:42:35.789ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
302026-06-29T20:42:35.798ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
312026-06-29T20:42:35.798ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
322026-06-29T20:42:35.798ZINFOcrucible-agent (worker): commit
332026-06-29T20:42:35.831ZINFOcrucible-agent (worker): ok!
342026-06-29T20:42:35.851ZINFOcrucible-agent (worker): SMF ok!
352026-06-29T20:42:36.248ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:40827 req_id = e90f0816-28aa-4ec2-a1a4-6213318013f5 response_code = 200 uri = /crucible/0/regions
362026-06-29T20:43:07.823ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::c]:55313
372026-06-29T20:43:07.823ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:55313 req_id = a3cb1984-9aad-478c-bd56-3176748b15d7 response_code = 200 uri = /crucible/0/regions/731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
382026-06-29T20:43:07.827ZINFOcrucible-agent (datafile): path is "/data/regions/731d8c7b-bfe5-4e10-84f5-819e14e5bfbc"
392026-06-29T20:43:07.838ZINFOcrucible-agent (datafile): dataset is oxp_3fff747b-18d8-4456-8204-c83c507de802/crucible/regions/731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
402026-06-29T20:43:07.845ZINFOcrucible-agent (datafile): region 731d8c7b-bfe5-4e10-84f5-819e14e5bfbc state: Created -> Tombstoned
412026-06-29T20:43:07.845ZINFOcrucible-agent (worker): applying SMF actions before removal...
422026-06-29T20:43:07.845ZINFOcrucible-agent (dropshot): request completed latency_us = 17663 local_addr = [fd00:1122:3344:101::13]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:55313 req_id = e335253b-f6fd-4806-9cab-e64c4dcf9715 response_code = 204 uri = /crucible/0/regions/731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
432026-06-29T20:43:07.846ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-731d8c7b-bfe5-4e10-84f5-819e14e5bfbc (instance states: (Some(Online), None))
442026-06-29T20:43:07.849ZINFOcrucible-agent (dropshot): request completed latency_us = 38 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:55313 req_id = feec9b86-9028-4781-8479-06d2d8d43ad1 response_code = 200 uri = /crucible/0/regions/731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
452026-06-29T20:43:07.850ZINFOcrucible-agent (worker): SMF ok!
462026-06-29T20:43:07.858ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_3fff747b-18d8-4456-8204-c83c507de802/crucible/regions/731d8c7b-bfe5-4e10-84f5-819e14e5bfbc" region = 731d8c7b-bfe5-4e10-84f5-819e14e5bfbc
472026-06-29T20:43:07.923ZINFOcrucible-agent (datafile): region 731d8c7b-bfe5-4e10-84f5-819e14e5bfbc state: Tombstoned -> Destroyed
482026-06-29T20:43:07.977ZINFOcrucible-agent (dropshot): request completed latency_us = 58 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:55313 req_id = 894ca426-61f2-4449-84f8-1f1c29492ac2 response_code = 200 uri = /crucible/0/regions/731d8c7b-bfe5-4e10-84f5-819e14e5bfbc