Max level shown:
1 [ Aug 22 17:49:12 Enabled. ]
2 [ Aug 22 17:49:12 Rereading configuration. ]
3 [ Aug 22 17:49:15 Rereading configuration. ]
4 [ Aug 22 17:49:22 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_47979b26-9dbe-4c82-b4bc-993a7ac617e6/crucible -l [fd00:1122:3344:101::12]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-08-22T17:49:22.655ZINFOcrucible-agent: dataset: "oxp_47979b26-9dbe-4c82-b4bc-993a7ac617e6/crucible"
72024-08-22T17:49:22.656ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::12]:32345
82024-08-22T17:49:22.656ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-08-22T17:49:22.674ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-08-22T17:49:22.814ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::12]:32345
112024-08-22T17:50:02.452ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:33213
122024-08-22T17:50:02.469ZINFOcrucible-agent (datafile): region 1352ea10-fe5b-49ee-9c82-6f59976fa2bb state: Requested
132024-08-22T17:50:02.474ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
142024-08-22T17:50:02.478ZINFOcrucible-agent (dropshot): request completed latency_us = 13133 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33213 req_id = 27993043-9869-480d-bbeb-27033ef1860c response_code = 200 uri = /crucible/0/regions
152024-08-22T17:50:02.484ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_47979b26-9dbe-4c82-b4bc-993a7ac617e6/crucible/regions/1352ea10-fe5b-49ee-9c82-6f59976fa2bb
162024-08-22T17:50:02.484ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_47979b26-9dbe-4c82-b4bc-993a7ac617e6/crucible/regions/1352ea10-fe5b-49ee-9c82-6f59976fa2bb
172024-08-22T17:50:02.561ZINFOcrucible-agent (worker): creating region Region { id: RegionId("1352ea10-fe5b-49ee-9c82-6f59976fa2bb"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/1352ea10-fe5b-49ee-9c82-6f59976fa2bb" region = 1352ea10-fe5b-49ee-9c82-6f59976fa2bb
182024-08-22T17:50:02.563ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "1352ea10-fe5b-49ee-9c82-6f59976fa2bb" "--data" "/data/regions/1352ea10-fe5b-49ee-9c82-6f59976fa2bb" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 1352ea10-fe5b-49ee-9c82-6f59976fa2bb
192024-08-22T17:50:02.686ZINFOcrucible-agent (worker): region files created ok region = 1352ea10-fe5b-49ee-9c82-6f59976fa2bb
202024-08-22T17:50:02.687ZINFOcrucible-agent (datafile): region 1352ea10-fe5b-49ee-9c82-6f59976fa2bb state: Requested -> Created
212024-08-22T17:50:02.687ZINFOcrucible-agent (worker): applying SMF actions post create...
222024-08-22T17:50:02.687ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-1352ea10-fe5b-49ee-9c82-6f59976fa2bb
232024-08-22T17:50:02.692ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-1352ea10-fe5b-49ee-9c82-6f59976fa2bb
242024-08-22T17:50:02.713ZINFOcrucible-agent (worker): creating config property group
252024-08-22T17:50:02.716ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-1352ea10-fe5b-49ee-9c82-6f59976fa2bb
262024-08-22T17:50:02.723ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/1352ea10-fe5b-49ee-9c82-6f59976fa2bb
272024-08-22T17:50:02.726ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
282024-08-22T17:50:02.726ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
292024-08-22T17:50:02.726ZINFOcrucible-agent (worker): commit
302024-08-22T17:50:02.730ZINFOcrucible-agent (worker): ok!
312024-08-22T17:50:02.734ZINFOcrucible-agent (worker): SMF ok!
322024-08-22T17:50:02.832ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33213 req_id = 349c3bf4-a500-4a42-abe0-63f0c6a4849f response_code = 200 uri = /crucible/0/regions
332024-08-22T17:50:03.455ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33213 req_id = ecf8fb17-7036-4b05-bff7-277db76bde25 response_code = 200 uri = /crucible/0/regions/1352ea10-fe5b-49ee-9c82-6f59976fa2bb
342024-08-22T17:50:03.458ZINFOcrucible-agent (datafile): path is "/data/regions/1352ea10-fe5b-49ee-9c82-6f59976fa2bb"
352024-08-22T17:50:03.467ZINFOcrucible-agent (datafile): dataset is oxp_47979b26-9dbe-4c82-b4bc-993a7ac617e6/crucible/regions/1352ea10-fe5b-49ee-9c82-6f59976fa2bb
362024-08-22T17:50:03.476ZINFOcrucible-agent (datafile): region 1352ea10-fe5b-49ee-9c82-6f59976fa2bb state: Created -> Tombstoned
372024-08-22T17:50:03.477ZINFOcrucible-agent (worker): applying SMF actions before removal...
382024-08-22T17:50:03.477ZINFOcrucible-agent (dropshot): request completed latency_us = 18841 local_addr = [fd00:1122:3344:101::12]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:33213 req_id = 7cb4a76f-aadf-424b-9401-5a334c8453a2 response_code = 204 uri = /crucible/0/regions/1352ea10-fe5b-49ee-9c82-6f59976fa2bb
392024-08-22T17:50:03.478ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-1352ea10-fe5b-49ee-9c82-6f59976fa2bb (instance states: (Some(Online), None))
402024-08-22T17:50:03.480ZINFOcrucible-agent (dropshot): request completed latency_us = 27 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33213 req_id = 0a4567e0-860c-4152-a149-a5f7035f1800 response_code = 200 uri = /crucible/0/regions/1352ea10-fe5b-49ee-9c82-6f59976fa2bb
412024-08-22T17:50:03.482ZINFOcrucible-agent (worker): SMF ok!
422024-08-22T17:50:03.492ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_47979b26-9dbe-4c82-b4bc-993a7ac617e6/crucible/regions/1352ea10-fe5b-49ee-9c82-6f59976fa2bb" region = 1352ea10-fe5b-49ee-9c82-6f59976fa2bb
432024-08-22T17:50:03.562ZINFOcrucible-agent (datafile): region 1352ea10-fe5b-49ee-9c82-6f59976fa2bb state: Tombstoned -> Destroyed
442024-08-22T17:50:03.591ZINFOcrucible-agent (dropshot): request completed latency_us = 81 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33213 req_id = bdff606c-233b-498a-9056-e60a83340259 response_code = 200 uri = /crucible/0/regions/1352ea10-fe5b-49ee-9c82-6f59976fa2bb
452024-08-22T17:55:25.026ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:63754
462024-08-22T17:55:25.026ZINFOcrucible-agent (datafile): region 4721c7ee-0ca0-41d9-80e0-0939f59d041a state: Requested
472024-08-22T17:55:25.027ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
482024-08-22T17:55:25.027ZINFOcrucible-agent (dropshot): request completed latency_us = 660 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63754 req_id = 6b63bfa6-c6bf-46b7-92f8-a811fa11fc6c response_code = 200 uri = /crucible/0/regions
492024-08-22T17:55:25.038ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_47979b26-9dbe-4c82-b4bc-993a7ac617e6/crucible/regions/4721c7ee-0ca0-41d9-80e0-0939f59d041a
502024-08-22T17:55:25.038ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_47979b26-9dbe-4c82-b4bc-993a7ac617e6/crucible/regions/4721c7ee-0ca0-41d9-80e0-0939f59d041a
512024-08-22T17:55:25.097ZINFOcrucible-agent (worker): creating region Region { id: RegionId("4721c7ee-0ca0-41d9-80e0-0939f59d041a"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/4721c7ee-0ca0-41d9-80e0-0939f59d041a" region = 4721c7ee-0ca0-41d9-80e0-0939f59d041a
522024-08-22T17:55:25.097ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "4721c7ee-0ca0-41d9-80e0-0939f59d041a" "--data" "/data/regions/4721c7ee-0ca0-41d9-80e0-0939f59d041a" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 4721c7ee-0ca0-41d9-80e0-0939f59d041a
532024-08-22T17:55:25.163ZINFOcrucible-agent (dropshot): request completed latency_us = 85 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63754 req_id = a161448b-1fe0-4bf0-9c91-7d450d6dfbe7 response_code = 200 uri = /crucible/0/regions
542024-08-22T17:55:25.399ZINFOcrucible-agent (worker): region files created ok region = 4721c7ee-0ca0-41d9-80e0-0939f59d041a
552024-08-22T17:55:25.399ZINFOcrucible-agent (datafile): region 4721c7ee-0ca0-41d9-80e0-0939f59d041a state: Requested -> Created
562024-08-22T17:55:25.399ZINFOcrucible-agent (worker): applying SMF actions post create...
572024-08-22T17:55:25.401ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-4721c7ee-0ca0-41d9-80e0-0939f59d041a
582024-08-22T17:55:25.406ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-4721c7ee-0ca0-41d9-80e0-0939f59d041a
592024-08-22T17:55:25.419ZINFOcrucible-agent (worker): creating config property group
602024-08-22T17:55:25.423ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-4721c7ee-0ca0-41d9-80e0-0939f59d041a
612024-08-22T17:55:25.424ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4721c7ee-0ca0-41d9-80e0-0939f59d041a
622024-08-22T17:55:25.424ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632024-08-22T17:55:25.424ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
642024-08-22T17:55:25.424ZINFOcrucible-agent (worker): commit
652024-08-22T17:55:25.428ZINFOcrucible-agent (worker): ok!
662024-08-22T17:55:25.433ZINFOcrucible-agent (worker): SMF ok!
672024-08-22T17:55:25.552ZINFOcrucible-agent (dropshot): request completed latency_us = 58 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63754 req_id = 3d055723-8c4a-4b3c-91e5-0bdbbc7f6370 response_code = 200 uri = /crucible/0/regions