Max level shown:
1 [ Nov 1 22:25:21 Enabled. ]
2 [ Nov 1 22:25:21 Rereading configuration. ]
3 [ Nov 1 22:25:23 Rereading configuration. ]
4 [ Nov 1 22:25:25 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_768cd5c9-c954-41f2-a65d-7a379e2ab256/crucible -l [fd00:1122:3344:101::14]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-11-01T22:25:25.528ZINFOcrucible-agent: dataset: "oxp_768cd5c9-c954-41f2-a65d-7a379e2ab256/crucible"
72024-11-01T22:25:25.530ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::14]:32345
82024-11-01T22:25:25.530ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-11-01T22:25:25.569ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-11-01T22:25:25.766ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::14]:32345
112024-11-01T22:26:14.491ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::b]:49716
122024-11-01T22:26:14.532ZINFOcrucible-agent (datafile): region 27e4ccf3-ca9d-4b9f-ac5f-689714a549ce state: Requested
132024-11-01T22:26:14.537ZINFOcrucible-agent (dropshot): request completed latency_us = 18195 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:49716 req_id = c9f2c8f4-7c5f-4c30-9976-a1286d035d99 response_code = 200 uri = /crucible/0/regions
142024-11-01T22:26:14.539ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152024-11-01T22:26:14.548ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_768cd5c9-c954-41f2-a65d-7a379e2ab256/crucible/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
162024-11-01T22:26:14.548ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_768cd5c9-c954-41f2-a65d-7a379e2ab256/crucible/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
172024-11-01T22:26:14.600ZINFOcrucible-agent (worker): creating region Region { id: RegionId("27e4ccf3-ca9d-4b9f-ac5f-689714a549ce"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce" region = 27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
182024-11-01T22:26:14.604ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "27e4ccf3-ca9d-4b9f-ac5f-689714a549ce" "--data" "/data/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
192024-11-01T22:26:14.875ZINFOcrucible-agent (worker): region files created ok region = 27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
202024-11-01T22:26:14.875ZINFOcrucible-agent (datafile): region 27e4ccf3-ca9d-4b9f-ac5f-689714a549ce state: Requested -> Created
212024-11-01T22:26:14.875ZINFOcrucible-agent (worker): applying SMF actions post create...
222024-11-01T22:26:14.876ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
232024-11-01T22:26:14.886ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
242024-11-01T22:26:14.893ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:49716 req_id = 254ea21c-ec89-4468-b152-c7c2124b9481 response_code = 200 uri = /crucible/0/regions
252024-11-01T22:26:14.905ZINFOcrucible-agent (worker): creating config property group
262024-11-01T22:26:14.908ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
272024-11-01T22:26:14.909ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
282024-11-01T22:26:14.911ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-11-01T22:26:14.911ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
302024-11-01T22:26:14.911ZINFOcrucible-agent (worker): commit
312024-11-01T22:26:14.921ZINFOcrucible-agent (worker): ok!
322024-11-01T22:26:14.927ZINFOcrucible-agent (worker): SMF ok!
332024-11-01T22:26:15.887ZINFOcrucible-agent (dropshot): request completed latency_us = 97 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:49716 req_id = 5df1cc32-a8f3-462b-9679-aee445445482 response_code = 200 uri = /crucible/0/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
342024-11-01T22:26:15.890ZINFOcrucible-agent (datafile): path is "/data/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce"
352024-11-01T22:26:15.901ZINFOcrucible-agent (datafile): dataset is oxp_768cd5c9-c954-41f2-a65d-7a379e2ab256/crucible/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
362024-11-01T22:26:15.912ZINFOcrucible-agent (datafile): region 27e4ccf3-ca9d-4b9f-ac5f-689714a549ce state: Created -> Tombstoned
372024-11-01T22:26:15.912ZINFOcrucible-agent (worker): applying SMF actions before removal...
382024-11-01T22:26:15.913ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-27e4ccf3-ca9d-4b9f-ac5f-689714a549ce (instance states: (Some(Online), None))
392024-11-01T22:26:15.914ZINFOcrucible-agent (dropshot): request completed latency_us = 23815 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:49716 req_id = 382d964d-d5ba-4386-a9e1-b5679788893a response_code = 204 uri = /crucible/0/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
402024-11-01T22:26:15.917ZINFOcrucible-agent (worker): SMF ok!
412024-11-01T22:26:15.920ZINFOcrucible-agent (dropshot): request completed latency_us = 720 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:49716 req_id = 2139ff8e-507b-414c-8618-939a196bf09a response_code = 200 uri = /crucible/0/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
422024-11-01T22:26:15.929ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_768cd5c9-c954-41f2-a65d-7a379e2ab256/crucible/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce" region = 27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
432024-11-01T22:26:15.991ZINFOcrucible-agent (dropshot): request completed latency_us = 87 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:49716 req_id = faf90bc9-d914-415f-a249-db039efd4b51 response_code = 200 uri = /crucible/0/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
442024-11-01T22:26:15.995ZINFOcrucible-agent (datafile): region 27e4ccf3-ca9d-4b9f-ac5f-689714a549ce state: Tombstoned -> Destroyed
452024-11-01T22:26:16.076ZINFOcrucible-agent (dropshot): request completed latency_us = 44 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:49716 req_id = ffe0942c-6474-481f-a4ed-985488f18c5c response_code = 200 uri = /crucible/0/regions/27e4ccf3-ca9d-4b9f-ac5f-689714a549ce
462024-11-01T22:26:40.823ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:60547
472024-11-01T22:26:40.824ZINFOcrucible-agent (datafile): region 266eaba1-867e-4321-b2bd-4ba8b5a7ffa0 state: Requested
482024-11-01T22:26:40.824ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-11-01T22:26:40.824ZINFOcrucible-agent (dropshot): request completed latency_us = 213 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60547 req_id = fd9a3226-4fb0-4f48-aa7c-587ec74f1d20 response_code = 200 uri = /crucible/0/regions
502024-11-01T22:26:40.834ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_768cd5c9-c954-41f2-a65d-7a379e2ab256/crucible/regions/266eaba1-867e-4321-b2bd-4ba8b5a7ffa0
512024-11-01T22:26:40.834ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_768cd5c9-c954-41f2-a65d-7a379e2ab256/crucible/regions/266eaba1-867e-4321-b2bd-4ba8b5a7ffa0
522024-11-01T22:26:40.886ZINFOcrucible-agent (worker): creating region Region { id: RegionId("266eaba1-867e-4321-b2bd-4ba8b5a7ffa0"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/266eaba1-867e-4321-b2bd-4ba8b5a7ffa0" region = 266eaba1-867e-4321-b2bd-4ba8b5a7ffa0
532024-11-01T22:26:40.888ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "266eaba1-867e-4321-b2bd-4ba8b5a7ffa0" "--data" "/data/regions/266eaba1-867e-4321-b2bd-4ba8b5a7ffa0" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 266eaba1-867e-4321-b2bd-4ba8b5a7ffa0
542024-11-01T22:26:40.947ZINFOcrucible-agent (worker): region files created ok region = 266eaba1-867e-4321-b2bd-4ba8b5a7ffa0
552024-11-01T22:26:40.947ZINFOcrucible-agent (datafile): region 266eaba1-867e-4321-b2bd-4ba8b5a7ffa0 state: Requested -> Created
562024-11-01T22:26:40.947ZINFOcrucible-agent (worker): applying SMF actions post create...
572024-11-01T22:26:40.949ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-266eaba1-867e-4321-b2bd-4ba8b5a7ffa0
582024-11-01T22:26:40.952ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-266eaba1-867e-4321-b2bd-4ba8b5a7ffa0
592024-11-01T22:26:40.959ZINFOcrucible-agent (worker): creating config property group
602024-11-01T22:26:40.964ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-266eaba1-867e-4321-b2bd-4ba8b5a7ffa0
612024-11-01T22:26:40.965ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/266eaba1-867e-4321-b2bd-4ba8b5a7ffa0
622024-11-01T22:26:40.965ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632024-11-01T22:26:40.965ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
642024-11-01T22:26:40.965ZINFOcrucible-agent (worker): commit
652024-11-01T22:26:40.968ZINFOcrucible-agent (worker): ok!
662024-11-01T22:26:40.972ZINFOcrucible-agent (worker): SMF ok!
672024-11-01T22:26:41.163ZINFOcrucible-agent (dropshot): request completed latency_us = 58 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60547 req_id = db1e5618-35ae-4779-a1d2-5da9af1ab427 response_code = 200 uri = /crucible/0/regions