Max level shown:
1 [ Mar 7 22:47:02 Enabled. ]
2 [ Mar 7 22:47:02 Rereading configuration. ]
3 [ Mar 7 22:47:05 Rereading configuration. ]
4 [ Mar 7 22:47:09 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible -l [fd00:1122:3344:101::15]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-03-07T22:47:10.388ZINFOcrucible-agent: dataset: "oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible"
72024-03-07T22:47:10.391ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::15]:32345
82024-03-07T22:47:10.391ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-03-07T22:47:10.565ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::15]:32345
102024-03-07T22:47:30.323ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::b]:43671
112024-03-07T22:47:30.331ZINFOcrucible-agent (datafile): region a23adedc-786d-4178-aefb-f4995413f178 state: Requested
122024-03-07T22:47:30.331ZINFOcrucible-agent (dropshot): request completed latency_us = 3116 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43671 req_id = cd3475fd-470b-4930-a129-bce05e092641 response_code = 200 uri = /crucible/0/regions
132024-03-07T22:47:30.334ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
142024-03-07T22:47:30.344ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible/regions/a23adedc-786d-4178-aefb-f4995413f178
152024-03-07T22:47:30.344ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible/regions/a23adedc-786d-4178-aefb-f4995413f178
162024-03-07T22:47:30.440ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a23adedc-786d-4178-aefb-f4995413f178"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None } at "/data/regions/a23adedc-786d-4178-aefb-f4995413f178" region = a23adedc-786d-4178-aefb-f4995413f178
172024-03-07T22:47:30.556ZINFOcrucible-agent (dropshot): request completed latency_us = 576 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43671 req_id = 76562fdd-0c9a-41ce-96ac-592df36fd7c7 response_code = 200 uri = /crucible/0/regions
182024-03-07T22:47:30.757ZINFOcrucible-agent (worker): region files created ok region = a23adedc-786d-4178-aefb-f4995413f178
192024-03-07T22:47:30.757ZINFOcrucible-agent (datafile): region a23adedc-786d-4178-aefb-f4995413f178 state: Requested -> Created
202024-03-07T22:47:30.757ZINFOcrucible-agent (worker): applying SMF actions post create...
212024-03-07T22:47:30.758ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a23adedc-786d-4178-aefb-f4995413f178
222024-03-07T22:47:30.765ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a23adedc-786d-4178-aefb-f4995413f178
232024-03-07T22:47:30.781ZINFOcrucible-agent (worker): creating config property group
242024-03-07T22:47:30.790ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a23adedc-786d-4178-aefb-f4995413f178
252024-03-07T22:47:30.791ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a23adedc-786d-4178-aefb-f4995413f178
262024-03-07T22:47:30.791ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
272024-03-07T22:47:30.791ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
282024-03-07T22:47:30.791ZINFOcrucible-agent (worker): commit
292024-03-07T22:47:30.799ZINFOcrucible-agent (worker): ok!
302024-03-07T22:47:30.805ZINFOcrucible-agent (worker): SMF ok!
312024-03-07T22:47:31.063ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43671 req_id = 169f9b3e-a364-4331-b2b5-22a11ff03b9a response_code = 200 uri = /crucible/0/regions
322024-03-07T22:47:31.495ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::b]:40964
332024-03-07T22:47:31.496ZINFOcrucible-agent (dropshot): request completed latency_us = 516 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:40964 req_id = e5e63c02-5321-4083-9989-bb5f3a4beb6f response_code = 200 uri = /crucible/0/regions/a23adedc-786d-4178-aefb-f4995413f178
342024-03-07T22:47:31.496ZINFOcrucible-agent (datafile): path is "/data/regions/a23adedc-786d-4178-aefb-f4995413f178"
352024-03-07T22:47:31.506ZINFOcrucible-agent (datafile): dataset is oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible/regions/a23adedc-786d-4178-aefb-f4995413f178
362024-03-07T22:47:31.515ZINFOcrucible-agent (datafile): region a23adedc-786d-4178-aefb-f4995413f178 state: Created -> Tombstoned
372024-03-07T22:47:31.515ZINFOcrucible-agent (worker): applying SMF actions before removal...
382024-03-07T22:47:31.515ZINFOcrucible-agent (dropshot): request completed latency_us = 19271 local_addr = [fd00:1122:3344:101::15]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:40964 req_id = 9ed8ad2c-3978-4e47-bd51-271d5689a113 response_code = 204 uri = /crucible/0/regions/a23adedc-786d-4178-aefb-f4995413f178
392024-03-07T22:47:31.516ZINFOcrucible-agent (dropshot): request completed latency_us = 22 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:40964 req_id = 00da6f90-45b3-45a4-a733-a3caf2a76a10 response_code = 200 uri = /crucible/0/regions/a23adedc-786d-4178-aefb-f4995413f178
402024-03-07T22:47:31.516ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-a23adedc-786d-4178-aefb-f4995413f178 (instance states: (Some(Online), None))
412024-03-07T22:47:31.525ZINFOcrucible-agent (worker): SMF ok!
422024-03-07T22:47:31.535ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible/regions/a23adedc-786d-4178-aefb-f4995413f178" region = a23adedc-786d-4178-aefb-f4995413f178
432024-03-07T22:47:31.610ZINFOcrucible-agent (datafile): region a23adedc-786d-4178-aefb-f4995413f178 state: Tombstoned -> Destroyed
442024-03-07T22:47:31.651ZINFOcrucible-agent (dropshot): request completed latency_us = 593 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:40964 req_id = 1d6a95ce-55ad-4de2-93d3-d5ac38bc80a2 response_code = 200 uri = /crucible/0/regions/a23adedc-786d-4178-aefb-f4995413f178
452024-03-07T22:53:28.871ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:38314
462024-03-07T22:53:28.872ZINFOcrucible-agent (datafile): region 9688050b-0834-4f7a-a981-40ecaa636974 state: Requested
472024-03-07T22:53:28.872ZINFOcrucible-agent (dropshot): request completed latency_us = 710 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:38314 req_id = 5860d40c-7a38-4e0f-99f4-ca25b7694298 response_code = 200 uri = /crucible/0/regions
482024-03-07T22:53:28.872ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-03-07T22:53:28.882ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible/regions/9688050b-0834-4f7a-a981-40ecaa636974
502024-03-07T22:53:28.882ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible/regions/9688050b-0834-4f7a-a981-40ecaa636974
512024-03-07T22:53:28.960ZINFOcrucible-agent (worker): creating region Region { id: RegionId("9688050b-0834-4f7a-a981-40ecaa636974"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None } at "/data/regions/9688050b-0834-4f7a-a981-40ecaa636974" region = 9688050b-0834-4f7a-a981-40ecaa636974
522024-03-07T22:53:29.125ZINFOcrucible-agent (dropshot): request completed latency_us = 89 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:38314 req_id = 0240e3c2-255e-4b15-bb39-32f8f3813cda response_code = 200 uri = /crucible/0/regions
532024-03-07T22:53:29.149ZINFOcrucible-agent (worker): region files created ok region = 9688050b-0834-4f7a-a981-40ecaa636974
542024-03-07T22:53:29.149ZINFOcrucible-agent (datafile): region 9688050b-0834-4f7a-a981-40ecaa636974 state: Requested -> Created
552024-03-07T22:53:29.149ZINFOcrucible-agent (worker): applying SMF actions post create...
562024-03-07T22:53:29.151ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-9688050b-0834-4f7a-a981-40ecaa636974
572024-03-07T22:53:29.160ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-9688050b-0834-4f7a-a981-40ecaa636974
582024-03-07T22:53:29.184ZINFOcrucible-agent (worker): creating config property group
592024-03-07T22:53:29.188ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-9688050b-0834-4f7a-a981-40ecaa636974
602024-03-07T22:53:29.188ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/9688050b-0834-4f7a-a981-40ecaa636974
612024-03-07T22:53:29.188ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
622024-03-07T22:53:29.188ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
632024-03-07T22:53:29.188ZINFOcrucible-agent (worker): commit
642024-03-07T22:53:29.196ZINFOcrucible-agent (worker): ok!
652024-03-07T22:53:29.204ZINFOcrucible-agent (worker): SMF ok!
662024-03-07T22:53:29.661ZINFOcrucible-agent (dropshot): request completed latency_us = 83 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:38314 req_id = 6f3b3196-3904-4431-9854-3da1cb67fa83 response_code = 200 uri = /crucible/0/regions