Max level shown:
12024-11-28T02:04:07.049ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:36642
22024-11-28T02:04:07.049ZINFOcrucible-agent (datafile): region 38619633-d10e-4e6c-a2bd-6ba1d4bf52f5 state: Requested
32024-11-28T02:04:07.050ZINFOcrucible-agent (dropshot): request completed latency_us = 595 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:36642 req_id = 5e778cf7-633f-44f6-8bb0-3e18f8a56f91 response_code = 200 uri = /crucible/0/regions
42024-11-28T02:04:07.050ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52024-11-28T02:04:07.061ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_830b5e11-e6e4-4144-bc16-2396db6d872e/crucible/regions/38619633-d10e-4e6c-a2bd-6ba1d4bf52f5
62024-11-28T02:04:07.061ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_830b5e11-e6e4-4144-bc16-2396db6d872e/crucible/regions/38619633-d10e-4e6c-a2bd-6ba1d4bf52f5
72024-11-28T02:04:07.135ZINFOcrucible-agent (worker): creating region Region { id: RegionId("38619633-d10e-4e6c-a2bd-6ba1d4bf52f5"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/38619633-d10e-4e6c-a2bd-6ba1d4bf52f5" region = 38619633-d10e-4e6c-a2bd-6ba1d4bf52f5
82024-11-28T02:04:07.137ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "38619633-d10e-4e6c-a2bd-6ba1d4bf52f5" "--data" "/data/regions/38619633-d10e-4e6c-a2bd-6ba1d4bf52f5" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 38619633-d10e-4e6c-a2bd-6ba1d4bf52f5
92024-11-28T02:04:07.215ZINFOcrucible-agent (worker): region files created ok region = 38619633-d10e-4e6c-a2bd-6ba1d4bf52f5
102024-11-28T02:04:07.215ZINFOcrucible-agent (datafile): region 38619633-d10e-4e6c-a2bd-6ba1d4bf52f5 state: Requested -> Created
112024-11-28T02:04:07.215ZINFOcrucible-agent (worker): applying SMF actions post create...
122024-11-28T02:04:07.216ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-38619633-d10e-4e6c-a2bd-6ba1d4bf52f5
132024-11-28T02:04:07.224ZINFOcrucible-agent (dropshot): request completed latency_us = 80 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:36642 req_id = 2bd061fe-a704-4d2f-9c51-19a4bdf245d5 response_code = 200 uri = /crucible/0/regions
142024-11-28T02:04:07.231ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-38619633-d10e-4e6c-a2bd-6ba1d4bf52f5
152024-11-28T02:04:07.245ZINFOcrucible-agent (worker): creating config property group
162024-11-28T02:04:07.249ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-38619633-d10e-4e6c-a2bd-6ba1d4bf52f5
172024-11-28T02:04:07.249ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/38619633-d10e-4e6c-a2bd-6ba1d4bf52f5
182024-11-28T02:04:07.249ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192024-11-28T02:04:07.249ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
202024-11-28T02:04:07.249ZINFOcrucible-agent (worker): commit
212024-11-28T02:04:07.253ZINFOcrucible-agent (worker): ok!
222024-11-28T02:04:07.258ZINFOcrucible-agent (worker): SMF ok!
232024-11-28T02:04:08.798ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:36642 req_id = 32b504a2-60df-4ea4-a61c-346cc7dfdfc7 response_code = 200 uri = /crucible/0/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08
242024-11-28T02:04:08.801ZINFOcrucible-agent (datafile): path is "/data/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08"
252024-11-28T02:04:08.810ZINFOcrucible-agent (datafile): dataset is oxp_830b5e11-e6e4-4144-bc16-2396db6d872e/crucible/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08
262024-11-28T02:04:08.819ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_830b5e11-e6e4-4144-bc16-2396db6d872e/crucible/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08@9eabf251-a7d1-4c56-b121-eada8b4b2c42
272024-11-28T02:04:08.819ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_830b5e11-e6e4-4144-bc16-2396db6d872e/crucible/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08", "9eabf251-a7d1-4c56-b121-eada8b4b2c42"]
282024-11-28T02:04:08.831ZINFOcrucible-agent (dropshot): request completed latency_us = 30062 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:36642 req_id = 30bfbdec-aae0-4c91-b99a-85d072a2441f response_code = 200 uri = /crucible/0/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08/snapshots/9eabf251-a7d1-4c56-b121-eada8b4b2c42
292024-11-28T02:04:08.834ZINFOcrucible-agent (datafile): path is "/data/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08"
302024-11-28T02:04:08.843ZINFOcrucible-agent (datafile): dataset is oxp_830b5e11-e6e4-4144-bc16-2396db6d872e/crucible/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08
312024-11-28T02:04:08.850ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_830b5e11-e6e4-4144-bc16-2396db6d872e/crucible/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08@9eabf251-a7d1-4c56-b121-eada8b4b2c42
322024-11-28T02:04:08.850ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_830b5e11-e6e4-4144-bc16-2396db6d872e/crucible/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08", "9eabf251-a7d1-4c56-b121-eada8b4b2c42"]
332024-11-28T02:04:08.860ZINFOcrucible-agent (datafile): requesting running snapshot 66fc8e24-46ea-4c26-8491-6db6803c9b08-9eabf251-a7d1-4c56-b121-eada8b4b2c42 state: Requested
342024-11-28T02:04:08.860ZINFOcrucible-agent (worker): applying SMF actions for region 66fc8e24-46ea-4c26-8491-6db6803c9b08 running snapshot 9eabf251-a7d1-4c56-b121-eada8b4b2c42 (state Requested)...
352024-11-28T02:04:08.860ZINFOcrucible-agent (dropshot): request completed latency_us = 25782 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:36642 req_id = 4ed7778b-f1a6-4c05-9b5d-0f763444455e response_code = 200 uri = /crucible/0/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08/snapshots/9eabf251-a7d1-4c56-b121-eada8b4b2c42/run
362024-11-28T02:04:08.863ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-66fc8e24-46ea-4c26-8491-6db6803c9b08-9eabf251-a7d1-4c56-b121-eada8b4b2c42
372024-11-28T02:04:08.866ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-66fc8e24-46ea-4c26-8491-6db6803c9b08-9eabf251-a7d1-4c56-b121-eada8b4b2c42
382024-11-28T02:04:08.877ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-66fc8e24-46ea-4c26-8491-6db6803c9b08-9eabf251-a7d1-4c56-b121-eada8b4b2c42
392024-11-28T02:04:08.877ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08/.zfs/snapshot/9eabf251-a7d1-4c56-b121-eada8b4b2c42
402024-11-28T02:04:08.877ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412024-11-28T02:04:08.877ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422024-11-28T02:04:08.877ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
432024-11-28T02:04:08.877ZINFOcrucible-agent (worker): commit
442024-11-28T02:04:08.882ZINFOcrucible-agent (worker): ok!
452024-11-28T02:04:08.887ZINFOcrucible-agent (worker): SMF ok!
462024-11-28T02:04:08.887ZINFOcrucible-agent (datafile): region 66fc8e24-46ea-4c26-8491-6db6803c9b08 running snapshot 9eabf251-a7d1-4c56-b121-eada8b4b2c42 state: Requested -> Created
472024-11-28T02:04:09.012ZINFOcrucible-agent (datafile): path is "/data/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08"
482024-11-28T02:04:09.020ZINFOcrucible-agent (datafile): dataset is oxp_830b5e11-e6e4-4144-bc16-2396db6d872e/crucible/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08
492024-11-28T02:04:09.028ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_830b5e11-e6e4-4144-bc16-2396db6d872e/crucible/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08@9eabf251-a7d1-4c56-b121-eada8b4b2c42
502024-11-28T02:04:09.028ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_830b5e11-e6e4-4144-bc16-2396db6d872e/crucible/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08", "9eabf251-a7d1-4c56-b121-eada8b4b2c42"]
512024-11-28T02:04:09.036ZINFOcrucible-agent (dropshot): request completed latency_us = 24022 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:36642 req_id = 56fae17d-3468-4f7e-9c37-324e208ba549 response_code = 200 uri = /crucible/0/regions/66fc8e24-46ea-4c26-8491-6db6803c9b08/snapshots/9eabf251-a7d1-4c56-b121-eada8b4b2c42/run