Max level shown:
12024-10-09T22:15:01.435ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
22024-10-09T22:15:01.722ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::19]:32345
32024-10-09T22:15:44.215ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::c]:58032
42024-10-09T22:15:44.248ZINFOcrucible-agent (datafile): region a5106e57-15be-454a-8d3f-cdf42af8474b state: Requested
52024-10-09T22:15:44.254ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
62024-10-09T22:15:44.254ZINFOcrucible-agent (dropshot): request completed latency_us = 13317 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:58032 req_id = c7b26561-6a63-4911-bffe-13913b2c051f response_code = 200 uri = /crucible/0/regions
72024-10-09T22:15:44.264ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_cca70482-eb31-4867-9ee1-c52adad83c57/crucible/regions/a5106e57-15be-454a-8d3f-cdf42af8474b
82024-10-09T22:15:44.264ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_cca70482-eb31-4867-9ee1-c52adad83c57/crucible/regions/a5106e57-15be-454a-8d3f-cdf42af8474b
92024-10-09T22:15:44.321ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a5106e57-15be-454a-8d3f-cdf42af8474b"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/a5106e57-15be-454a-8d3f-cdf42af8474b" region = a5106e57-15be-454a-8d3f-cdf42af8474b
102024-10-09T22:15:44.325ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "a5106e57-15be-454a-8d3f-cdf42af8474b" "--data" "/data/regions/a5106e57-15be-454a-8d3f-cdf42af8474b" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = a5106e57-15be-454a-8d3f-cdf42af8474b
112024-10-09T22:15:44.449ZINFOcrucible-agent (dropshot): request completed latency_us = 84 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:58032 req_id = 94622428-bbbb-48e6-866d-9964b0437cc4 response_code = 200 uri = /crucible/0/regions
122024-10-09T22:15:44.544ZINFOcrucible-agent (worker): region files created ok region = a5106e57-15be-454a-8d3f-cdf42af8474b
132024-10-09T22:15:44.544ZINFOcrucible-agent (datafile): region a5106e57-15be-454a-8d3f-cdf42af8474b state: Requested -> Created
142024-10-09T22:15:44.544ZINFOcrucible-agent (worker): applying SMF actions post create...
152024-10-09T22:15:44.547ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a5106e57-15be-454a-8d3f-cdf42af8474b
162024-10-09T22:15:44.551ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a5106e57-15be-454a-8d3f-cdf42af8474b
172024-10-09T22:15:44.565ZINFOcrucible-agent (worker): creating config property group
182024-10-09T22:15:44.568ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a5106e57-15be-454a-8d3f-cdf42af8474b
192024-10-09T22:15:44.573ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a5106e57-15be-454a-8d3f-cdf42af8474b
202024-10-09T22:15:44.573ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
212024-10-09T22:15:44.573ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
222024-10-09T22:15:44.573ZINFOcrucible-agent (worker): commit
232024-10-09T22:15:44.576ZINFOcrucible-agent (worker): ok!
242024-10-09T22:15:44.581ZINFOcrucible-agent (worker): SMF ok!
252024-10-09T22:15:45.083ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:58032 req_id = 4b2179ac-e580-4a92-bb0d-7ae71c908e4f response_code = 200 uri = /crucible/0/regions
262024-10-09T22:15:45.573ZINFOcrucible-agent (dropshot): request completed latency_us = 87 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:58032 req_id = b746621e-e408-4710-8f2e-9c79c41441db response_code = 200 uri = /crucible/0/regions/a5106e57-15be-454a-8d3f-cdf42af8474b
272024-10-09T22:15:45.575ZINFOcrucible-agent (datafile): path is "/data/regions/a5106e57-15be-454a-8d3f-cdf42af8474b"
282024-10-09T22:15:45.587ZINFOcrucible-agent (datafile): dataset is oxp_cca70482-eb31-4867-9ee1-c52adad83c57/crucible/regions/a5106e57-15be-454a-8d3f-cdf42af8474b
292024-10-09T22:15:45.597ZINFOcrucible-agent (datafile): region a5106e57-15be-454a-8d3f-cdf42af8474b state: Created -> Tombstoned
302024-10-09T22:15:45.597ZINFOcrucible-agent (worker): applying SMF actions before removal...
312024-10-09T22:15:45.597ZINFOcrucible-agent (dropshot): request completed latency_us = 21962 local_addr = [fd00:1122:3344:101::19]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:58032 req_id = 38615756-584d-46a8-894f-380736b00639 response_code = 204 uri = /crucible/0/regions/a5106e57-15be-454a-8d3f-cdf42af8474b
322024-10-09T22:15:45.598ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-a5106e57-15be-454a-8d3f-cdf42af8474b (instance states: (Some(Online), None))
332024-10-09T22:15:45.600ZINFOcrucible-agent (dropshot): request completed latency_us = 34 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:58032 req_id = 52d8dccd-eba7-4765-af95-6dd0d3717b7a response_code = 200 uri = /crucible/0/regions/a5106e57-15be-454a-8d3f-cdf42af8474b
342024-10-09T22:15:45.602ZINFOcrucible-agent (worker): SMF ok!
352024-10-09T22:15:45.613ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_cca70482-eb31-4867-9ee1-c52adad83c57/crucible/regions/a5106e57-15be-454a-8d3f-cdf42af8474b" region = a5106e57-15be-454a-8d3f-cdf42af8474b
362024-10-09T22:15:45.672ZINFOcrucible-agent (dropshot): request completed latency_us = 88 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:58032 req_id = 94e2b0f6-1f19-4034-a907-a826b7be5619 response_code = 200 uri = /crucible/0/regions/a5106e57-15be-454a-8d3f-cdf42af8474b
372024-10-09T22:15:45.676ZINFOcrucible-agent (datafile): region a5106e57-15be-454a-8d3f-cdf42af8474b state: Tombstoned -> Destroyed
382024-10-09T22:15:45.764ZINFOcrucible-agent (dropshot): request completed latency_us = 29 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:58032 req_id = 185e7478-ae22-4595-9b93-36fc9ed257b8 response_code = 200 uri = /crucible/0/regions/a5106e57-15be-454a-8d3f-cdf42af8474b
392024-10-09T22:16:18.892ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:48363
402024-10-09T22:16:18.892ZINFOcrucible-agent (datafile): region 75da1c77-d142-486f-965b-14fa93706591 state: Requested
412024-10-09T22:16:18.892ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
422024-10-09T22:16:18.892ZINFOcrucible-agent (dropshot): request completed latency_us = 215 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48363 req_id = 4a42d62d-9318-44c8-8c1b-284a690f5861 response_code = 200 uri = /crucible/0/regions
432024-10-09T22:16:18.901ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_cca70482-eb31-4867-9ee1-c52adad83c57/crucible/regions/75da1c77-d142-486f-965b-14fa93706591
442024-10-09T22:16:18.901ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_cca70482-eb31-4867-9ee1-c52adad83c57/crucible/regions/75da1c77-d142-486f-965b-14fa93706591
452024-10-09T22:16:18.952ZINFOcrucible-agent (worker): creating region Region { id: RegionId("75da1c77-d142-486f-965b-14fa93706591"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/75da1c77-d142-486f-965b-14fa93706591" region = 75da1c77-d142-486f-965b-14fa93706591
462024-10-09T22:16:18.954ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "75da1c77-d142-486f-965b-14fa93706591" "--data" "/data/regions/75da1c77-d142-486f-965b-14fa93706591" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 75da1c77-d142-486f-965b-14fa93706591
472024-10-09T22:16:19.017ZINFOcrucible-agent (worker): region files created ok region = 75da1c77-d142-486f-965b-14fa93706591
482024-10-09T22:16:19.017ZINFOcrucible-agent (datafile): region 75da1c77-d142-486f-965b-14fa93706591 state: Requested -> Created
492024-10-09T22:16:19.017ZINFOcrucible-agent (worker): applying SMF actions post create...
502024-10-09T22:16:19.019ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-75da1c77-d142-486f-965b-14fa93706591
512024-10-09T22:16:19.022ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-75da1c77-d142-486f-965b-14fa93706591
522024-10-09T22:16:19.029ZINFOcrucible-agent (worker): creating config property group
532024-10-09T22:16:19.032ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-75da1c77-d142-486f-965b-14fa93706591
542024-10-09T22:16:19.033ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/75da1c77-d142-486f-965b-14fa93706591
552024-10-09T22:16:19.033ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
562024-10-09T22:16:19.033ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
572024-10-09T22:16:19.033ZINFOcrucible-agent (worker): commit
582024-10-09T22:16:19.036ZINFOcrucible-agent (worker): ok!
592024-10-09T22:16:19.041ZINFOcrucible-agent (worker): SMF ok!
602024-10-09T22:16:19.213ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48363 req_id = 71972c30-2ffb-4b31-980e-9389f8789e9c response_code = 200 uri = /crucible/0/regions
612024-10-09T22:17:02.795ZINFOcrucible-agent (dropshot): request completed latency_us = 68 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:48363 req_id = 4c8490f5-bdb7-418b-aebe-b25790cb8742 response_code = 200 uri = /crucible/0/regions/75da1c77-d142-486f-965b-14fa93706591
622024-10-09T22:17:02.798ZINFOcrucible-agent (datafile): path is "/data/regions/75da1c77-d142-486f-965b-14fa93706591"
632024-10-09T22:17:02.808ZINFOcrucible-agent (datafile): dataset is oxp_cca70482-eb31-4867-9ee1-c52adad83c57/crucible/regions/75da1c77-d142-486f-965b-14fa93706591
642024-10-09T22:17:02.818ZINFOcrucible-agent (datafile): region 75da1c77-d142-486f-965b-14fa93706591 state: Created -> Tombstoned
652024-10-09T22:17:02.818ZINFOcrucible-agent (worker): applying SMF actions before removal...
662024-10-09T22:17:02.818ZINFOcrucible-agent (dropshot): request completed latency_us = 19918 local_addr = [fd00:1122:3344:101::19]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:48363 req_id = 357a861f-96cd-4f54-9ff7-8b69a5f94a65 response_code = 204 uri = /crucible/0/regions/75da1c77-d142-486f-965b-14fa93706591
672024-10-09T22:17:02.820ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-75da1c77-d142-486f-965b-14fa93706591 (instance states: (Some(Online), None))
682024-10-09T22:17:02.821ZINFOcrucible-agent (dropshot): request completed latency_us = 43 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:48363 req_id = 3fd811d3-2649-40d7-aa83-55a983b6785e response_code = 200 uri = /crucible/0/regions/75da1c77-d142-486f-965b-14fa93706591
692024-10-09T22:17:02.826ZINFOcrucible-agent (worker): SMF ok!
702024-10-09T22:17:02.836ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_cca70482-eb31-4867-9ee1-c52adad83c57/crucible/regions/75da1c77-d142-486f-965b-14fa93706591" region = 75da1c77-d142-486f-965b-14fa93706591
712024-10-09T22:17:02.897ZINFOcrucible-agent (datafile): region 75da1c77-d142-486f-965b-14fa93706591 state: Tombstoned -> Destroyed
722024-10-09T22:17:02.961ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:48363 req_id = 495ed826-cd3f-4510-93db-0f753543b30c response_code = 200 uri = /crucible/0/regions/75da1c77-d142-486f-965b-14fa93706591