Max level shown:
1 [ Oct 14 10:09:54 Enabled. ]
2 [ Oct 14 10:09:54 Rereading configuration. ]
3 [ Oct 14 10:09:55 Rereading configuration. ]
4 [ Oct 14 10:10:04 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_8c517ed8-9134-4300-bd98-54891db0f94c/crucible -l [fd00:1122:3344:101::17]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-10-14T10:10:04.152ZINFOcrucible-agent: dataset: "oxp_8c517ed8-9134-4300-bd98-54891db0f94c/crucible"
72024-10-14T10:10:04.154ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::17]:32345
82024-10-14T10:10:04.154ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-10-14T10:10:04.167ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-10-14T10:10:04.434ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::17]:32345
112024-10-14T10:11:02.068ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:46010
122024-10-14T10:11:02.114ZINFOcrucible-agent (datafile): region 5eb199b7-ea34-4df9-a595-3930a3837791 state: Requested
132024-10-14T10:11:02.124ZINFOcrucible-agent (dropshot): request completed latency_us = 18906 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:46010 req_id = 80aadf76-7552-4e2b-a3e5-2ac7343cf556 response_code = 200 uri = /crucible/0/regions
142024-10-14T10:11:02.124ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152024-10-14T10:11:02.134ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_8c517ed8-9134-4300-bd98-54891db0f94c/crucible/regions/5eb199b7-ea34-4df9-a595-3930a3837791
162024-10-14T10:11:02.134ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_8c517ed8-9134-4300-bd98-54891db0f94c/crucible/regions/5eb199b7-ea34-4df9-a595-3930a3837791
172024-10-14T10:11:02.192ZINFOcrucible-agent (worker): creating region Region { id: RegionId("5eb199b7-ea34-4df9-a595-3930a3837791"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/5eb199b7-ea34-4df9-a595-3930a3837791" region = 5eb199b7-ea34-4df9-a595-3930a3837791
182024-10-14T10:11:02.197ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "5eb199b7-ea34-4df9-a595-3930a3837791" "--data" "/data/regions/5eb199b7-ea34-4df9-a595-3930a3837791" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 5eb199b7-ea34-4df9-a595-3930a3837791
192024-10-14T10:11:02.399ZINFOcrucible-agent (dropshot): request completed latency_us = 85 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:46010 req_id = 4f506ca8-e667-4cc3-aa5a-1fc3ed51c883 response_code = 200 uri = /crucible/0/regions
202024-10-14T10:11:02.523ZINFOcrucible-agent (worker): region files created ok region = 5eb199b7-ea34-4df9-a595-3930a3837791
212024-10-14T10:11:02.523ZINFOcrucible-agent (datafile): region 5eb199b7-ea34-4df9-a595-3930a3837791 state: Requested -> Created
222024-10-14T10:11:02.523ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-10-14T10:11:02.528ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-5eb199b7-ea34-4df9-a595-3930a3837791
242024-10-14T10:11:02.536ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-5eb199b7-ea34-4df9-a595-3930a3837791
252024-10-14T10:11:02.565ZINFOcrucible-agent (worker): creating config property group
262024-10-14T10:11:02.611ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-5eb199b7-ea34-4df9-a595-3930a3837791
272024-10-14T10:11:02.612ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/5eb199b7-ea34-4df9-a595-3930a3837791
282024-10-14T10:11:02.612ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-10-14T10:11:02.612ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
302024-10-14T10:11:02.612ZINFOcrucible-agent (worker): commit
312024-10-14T10:11:02.615ZINFOcrucible-agent (worker): ok!
322024-10-14T10:11:02.622ZINFOcrucible-agent (worker): SMF ok!
332024-10-14T10:11:02.867ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:46010 req_id = e2ea89d2-3144-4a21-bdf8-15c81d39268a response_code = 200 uri = /crucible/0/regions
342024-10-14T10:11:03.531ZINFOcrucible-agent (dropshot): request completed latency_us = 100 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:46010 req_id = e5a35255-e38b-4562-ac20-fce860bdd46f response_code = 200 uri = /crucible/0/regions/5eb199b7-ea34-4df9-a595-3930a3837791
352024-10-14T10:11:03.534ZINFOcrucible-agent (datafile): path is "/data/regions/5eb199b7-ea34-4df9-a595-3930a3837791"
362024-10-14T10:11:03.544ZINFOcrucible-agent (datafile): dataset is oxp_8c517ed8-9134-4300-bd98-54891db0f94c/crucible/regions/5eb199b7-ea34-4df9-a595-3930a3837791
372024-10-14T10:11:03.555ZINFOcrucible-agent (datafile): region 5eb199b7-ea34-4df9-a595-3930a3837791 state: Created -> Tombstoned
382024-10-14T10:11:03.555ZINFOcrucible-agent (worker): applying SMF actions before removal...
392024-10-14T10:11:03.555ZINFOcrucible-agent (dropshot): request completed latency_us = 21435 local_addr = [fd00:1122:3344:101::17]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:46010 req_id = 974ad17a-da75-45b7-97e0-2ea792624bbe response_code = 204 uri = /crucible/0/regions/5eb199b7-ea34-4df9-a595-3930a3837791
402024-10-14T10:11:03.556ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-5eb199b7-ea34-4df9-a595-3930a3837791 (instance states: (Some(Online), None))
412024-10-14T10:11:03.559ZINFOcrucible-agent (dropshot): request completed latency_us = 32 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:46010 req_id = 4a63f3d8-a69a-49f6-8334-0df33159d6d1 response_code = 200 uri = /crucible/0/regions/5eb199b7-ea34-4df9-a595-3930a3837791
422024-10-14T10:11:03.565ZINFOcrucible-agent (worker): SMF ok!
432024-10-14T10:11:03.575ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_8c517ed8-9134-4300-bd98-54891db0f94c/crucible/regions/5eb199b7-ea34-4df9-a595-3930a3837791" region = 5eb199b7-ea34-4df9-a595-3930a3837791
442024-10-14T10:11:03.667ZINFOcrucible-agent (datafile): region 5eb199b7-ea34-4df9-a595-3930a3837791 state: Tombstoned -> Destroyed
452024-10-14T10:11:03.674ZINFOcrucible-agent (dropshot): request completed latency_us = 88 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:46010 req_id = 1dcd3ac8-b747-4831-985b-77b3067f9a79 response_code = 200 uri = /crucible/0/regions/5eb199b7-ea34-4df9-a595-3930a3837791
462024-10-14T10:14:15.436ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:62670
472024-10-14T10:14:15.437ZINFOcrucible-agent (datafile): region dd5dbd58-c3f3-438d-8292-723cbb4e6668 state: Requested
482024-10-14T10:14:15.439ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-10-14T10:14:15.439ZINFOcrucible-agent (dropshot): request completed latency_us = 2344 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62670 req_id = 2aadb17d-0dc0-4fbe-b99a-5c470687e84d response_code = 200 uri = /crucible/0/regions
502024-10-14T10:14:15.447ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_8c517ed8-9134-4300-bd98-54891db0f94c/crucible/regions/dd5dbd58-c3f3-438d-8292-723cbb4e6668
512024-10-14T10:14:15.447ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_8c517ed8-9134-4300-bd98-54891db0f94c/crucible/regions/dd5dbd58-c3f3-438d-8292-723cbb4e6668
522024-10-14T10:14:15.503ZINFOcrucible-agent (worker): creating region Region { id: RegionId("dd5dbd58-c3f3-438d-8292-723cbb4e6668"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/dd5dbd58-c3f3-438d-8292-723cbb4e6668" region = dd5dbd58-c3f3-438d-8292-723cbb4e6668
532024-10-14T10:14:15.505ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "dd5dbd58-c3f3-438d-8292-723cbb4e6668" "--data" "/data/regions/dd5dbd58-c3f3-438d-8292-723cbb4e6668" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = dd5dbd58-c3f3-438d-8292-723cbb4e6668
542024-10-14T10:14:15.679ZINFOcrucible-agent (dropshot): request completed latency_us = 81 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62670 req_id = 0b6503a3-158a-49d6-bb2c-7b91fedc89ef response_code = 200 uri = /crucible/0/regions
552024-10-14T10:14:15.712ZINFOcrucible-agent (worker): region files created ok region = dd5dbd58-c3f3-438d-8292-723cbb4e6668
562024-10-14T10:14:15.712ZINFOcrucible-agent (datafile): region dd5dbd58-c3f3-438d-8292-723cbb4e6668 state: Requested -> Created
572024-10-14T10:14:15.712ZINFOcrucible-agent (worker): applying SMF actions post create...
582024-10-14T10:14:15.713ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-dd5dbd58-c3f3-438d-8292-723cbb4e6668
592024-10-14T10:14:15.721ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-dd5dbd58-c3f3-438d-8292-723cbb4e6668
602024-10-14T10:14:15.740ZINFOcrucible-agent (worker): creating config property group
612024-10-14T10:14:15.748ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-dd5dbd58-c3f3-438d-8292-723cbb4e6668
622024-10-14T10:14:15.748ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/dd5dbd58-c3f3-438d-8292-723cbb4e6668
632024-10-14T10:14:15.748ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
642024-10-14T10:14:15.748ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
652024-10-14T10:14:15.748ZINFOcrucible-agent (worker): commit
662024-10-14T10:14:15.755ZINFOcrucible-agent (worker): ok!
672024-10-14T10:14:15.763ZINFOcrucible-agent (worker): SMF ok!
682024-10-14T10:14:16.169ZINFOcrucible-agent (dropshot): request completed latency_us = 788 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62670 req_id = 56b699c0-10f0-42fa-85a8-bce411c18fa6 response_code = 200 uri = /crucible/0/regions