Max level shown:
12025-05-30T21:37:12.593ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::c]:35767
22025-05-30T21:37:12.594ZINFOcrucible-agent (datafile): region 35335021-983a-4aff-aba6-01748ac0b5e9 state: Requested
32025-05-30T21:37:12.594ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-05-30T21:37:12.594ZINFOcrucible-agent (dropshot): request completed latency_us = 885 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:35767 req_id = db1f8e45-3cec-434f-b043-57c4d2e28bb3 response_code = 200 uri = /crucible/0/regions
52025-05-30T21:37:12.606ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_fe2cc2ed-07a4-4bd2-b950-9953f4efb6fe/crucible/regions/35335021-983a-4aff-aba6-01748ac0b5e9
62025-05-30T21:37:12.606ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_fe2cc2ed-07a4-4bd2-b950-9953f4efb6fe/crucible/regions/35335021-983a-4aff-aba6-01748ac0b5e9
72025-05-30T21:37:12.676ZINFOcrucible-agent (worker): creating region Region { id: RegionId("35335021-983a-4aff-aba6-01748ac0b5e9"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/35335021-983a-4aff-aba6-01748ac0b5e9" region = 35335021-983a-4aff-aba6-01748ac0b5e9
82025-05-30T21:37:12.676ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "35335021-983a-4aff-aba6-01748ac0b5e9" "--data" "/data/regions/35335021-983a-4aff-aba6-01748ac0b5e9" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 35335021-983a-4aff-aba6-01748ac0b5e9
92025-05-30T21:37:12.728ZINFOcrucible-agent (dropshot): request completed latency_us = 68 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:35767 req_id = 6074cce5-567e-4b30-a734-d34bb0a2b832 response_code = 200 uri = /crucible/0/regions
102025-05-30T21:37:12.813ZINFOcrucible-agent (worker): region files created ok region = 35335021-983a-4aff-aba6-01748ac0b5e9
112025-05-30T21:37:12.813ZINFOcrucible-agent (datafile): region 35335021-983a-4aff-aba6-01748ac0b5e9 state: Requested -> Created
122025-05-30T21:37:12.813ZINFOcrucible-agent (worker): applying SMF actions post create...
132025-05-30T21:37:12.813ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-35335021-983a-4aff-aba6-01748ac0b5e9
142025-05-30T21:37:12.826ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-35335021-983a-4aff-aba6-01748ac0b5e9
152025-05-30T21:37:12.853ZINFOcrucible-agent (worker): creating config property group
162025-05-30T21:37:12.862ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-35335021-983a-4aff-aba6-01748ac0b5e9
172025-05-30T21:37:12.862ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/35335021-983a-4aff-aba6-01748ac0b5e9
182025-05-30T21:37:12.862ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-05-30T21:37:12.863ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
202025-05-30T21:37:12.863ZINFOcrucible-agent (worker): commit
212025-05-30T21:37:12.871ZINFOcrucible-agent (worker): ok!
222025-05-30T21:37:12.883ZINFOcrucible-agent (worker): SMF ok!
232025-05-30T21:37:12.999ZINFOcrucible-agent (dropshot): request completed latency_us = 46 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:35767 req_id = d7e29d34-67ab-4fae-9caf-4760c0db83aa response_code = 200 uri = /crucible/0/regions
242025-05-30T21:37:13.715ZINFOcrucible-agent (dropshot): request completed latency_us = 2959 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:35767 req_id = d3770704-457d-435f-9480-c34fe4f20147 response_code = 200 uri = /crucible/0/regions/b401000a-01f7-4295-84ac-135932448892
252025-05-30T21:37:13.718ZINFOcrucible-agent (datafile): path is "/data/regions/b401000a-01f7-4295-84ac-135932448892"
262025-05-30T21:37:13.727ZINFOcrucible-agent (datafile): dataset is oxp_fe2cc2ed-07a4-4bd2-b950-9953f4efb6fe/crucible/regions/b401000a-01f7-4295-84ac-135932448892
272025-05-30T21:37:13.736ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_fe2cc2ed-07a4-4bd2-b950-9953f4efb6fe/crucible/regions/b401000a-01f7-4295-84ac-135932448892@df61edab-3a2e-438f-8a5e-636174851aa4
282025-05-30T21:37:13.736ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_fe2cc2ed-07a4-4bd2-b950-9953f4efb6fe/crucible/regions/b401000a-01f7-4295-84ac-135932448892", "df61edab-3a2e-438f-8a5e-636174851aa4"]
292025-05-30T21:37:13.748ZINFOcrucible-agent (dropshot): request completed latency_us = 30112 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:35767 req_id = f3f88740-fad8-4970-a0f4-9c0fde2b5adf response_code = 200 uri = /crucible/0/regions/b401000a-01f7-4295-84ac-135932448892/snapshots/df61edab-3a2e-438f-8a5e-636174851aa4
302025-05-30T21:37:13.753ZINFOcrucible-agent (datafile): path is "/data/regions/b401000a-01f7-4295-84ac-135932448892"
312025-05-30T21:37:13.761ZINFOcrucible-agent (datafile): dataset is oxp_fe2cc2ed-07a4-4bd2-b950-9953f4efb6fe/crucible/regions/b401000a-01f7-4295-84ac-135932448892
322025-05-30T21:37:13.768ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_fe2cc2ed-07a4-4bd2-b950-9953f4efb6fe/crucible/regions/b401000a-01f7-4295-84ac-135932448892@df61edab-3a2e-438f-8a5e-636174851aa4
332025-05-30T21:37:13.768ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_fe2cc2ed-07a4-4bd2-b950-9953f4efb6fe/crucible/regions/b401000a-01f7-4295-84ac-135932448892", "df61edab-3a2e-438f-8a5e-636174851aa4"]
342025-05-30T21:37:13.778ZINFOcrucible-agent (datafile): requesting running snapshot b401000a-01f7-4295-84ac-135932448892-df61edab-3a2e-438f-8a5e-636174851aa4 state: Requested
352025-05-30T21:37:13.778ZINFOcrucible-agent (worker): applying SMF actions for region b401000a-01f7-4295-84ac-135932448892 running snapshot df61edab-3a2e-438f-8a5e-636174851aa4 (state Requested)...
362025-05-30T21:37:13.779ZINFOcrucible-agent (dropshot): request completed latency_us = 25186 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:35767 req_id = 850f9486-7e68-4bb1-91b1-da9657a44260 response_code = 200 uri = /crucible/0/regions/b401000a-01f7-4295-84ac-135932448892/snapshots/df61edab-3a2e-438f-8a5e-636174851aa4/run
372025-05-30T21:37:13.781ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-b401000a-01f7-4295-84ac-135932448892-df61edab-3a2e-438f-8a5e-636174851aa4
382025-05-30T21:37:13.795ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-b401000a-01f7-4295-84ac-135932448892-df61edab-3a2e-438f-8a5e-636174851aa4
392025-05-30T21:37:13.815ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-b401000a-01f7-4295-84ac-135932448892-df61edab-3a2e-438f-8a5e-636174851aa4
402025-05-30T21:37:13.816ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b401000a-01f7-4295-84ac-135932448892/.zfs/snapshot/df61edab-3a2e-438f-8a5e-636174851aa4
412025-05-30T21:37:13.816ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
422025-05-30T21:37:13.816ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
432025-05-30T21:37:13.816ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
442025-05-30T21:37:13.816ZINFOcrucible-agent (worker): commit
452025-05-30T21:37:13.823ZINFOcrucible-agent (worker): ok!
462025-05-30T21:37:13.832ZINFOcrucible-agent (worker): SMF ok!
472025-05-30T21:37:13.832ZINFOcrucible-agent (datafile): region b401000a-01f7-4295-84ac-135932448892 running snapshot df61edab-3a2e-438f-8a5e-636174851aa4 state: Requested -> Created
482025-05-30T21:37:14.106ZINFOcrucible-agent (datafile): path is "/data/regions/b401000a-01f7-4295-84ac-135932448892"
492025-05-30T21:37:14.115ZINFOcrucible-agent (datafile): dataset is oxp_fe2cc2ed-07a4-4bd2-b950-9953f4efb6fe/crucible/regions/b401000a-01f7-4295-84ac-135932448892
502025-05-30T21:37:14.123ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_fe2cc2ed-07a4-4bd2-b950-9953f4efb6fe/crucible/regions/b401000a-01f7-4295-84ac-135932448892@df61edab-3a2e-438f-8a5e-636174851aa4
512025-05-30T21:37:14.123ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_fe2cc2ed-07a4-4bd2-b950-9953f4efb6fe/crucible/regions/b401000a-01f7-4295-84ac-135932448892", "df61edab-3a2e-438f-8a5e-636174851aa4"]
522025-05-30T21:37:14.131ZINFOcrucible-agent (dropshot): request completed latency_us = 24925 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:35767 req_id = 3b7fb99a-af86-4548-9df3-5f849d4acdf3 response_code = 200 uri = /crucible/0/regions/b401000a-01f7-4295-84ac-135932448892/snapshots/df61edab-3a2e-438f-8a5e-636174851aa4/run