Max level shown:
12024-12-02T16:55:20.700ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:45111
22024-12-02T16:55:20.700ZINFOcrucible-agent (datafile): region fc5123b9-1db9-48b4-8d89-70cd56d01d94 state: Requested
32024-12-02T16:55:20.701ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42024-12-02T16:55:20.701ZINFOcrucible-agent (dropshot): request completed latency_us = 590 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:45111 req_id = 5f5c1b6c-4854-40b4-aa0a-8d3cdce2950a response_code = 200 uri = /crucible/0/regions
52024-12-02T16:55:20.715ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_f2b67eee-c247-4902-8376-b0deafea2c96/crucible/regions/fc5123b9-1db9-48b4-8d89-70cd56d01d94
62024-12-02T16:55:20.715ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_f2b67eee-c247-4902-8376-b0deafea2c96/crucible/regions/fc5123b9-1db9-48b4-8d89-70cd56d01d94
72024-12-02T16:55:20.773ZINFOcrucible-agent (worker): creating region Region { id: RegionId("fc5123b9-1db9-48b4-8d89-70cd56d01d94"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/fc5123b9-1db9-48b4-8d89-70cd56d01d94" region = fc5123b9-1db9-48b4-8d89-70cd56d01d94
82024-12-02T16:55:20.773ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "fc5123b9-1db9-48b4-8d89-70cd56d01d94" "--data" "/data/regions/fc5123b9-1db9-48b4-8d89-70cd56d01d94" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = fc5123b9-1db9-48b4-8d89-70cd56d01d94
92024-12-02T16:55:20.883ZINFOcrucible-agent (worker): region files created ok region = fc5123b9-1db9-48b4-8d89-70cd56d01d94
102024-12-02T16:55:20.883ZINFOcrucible-agent (datafile): region fc5123b9-1db9-48b4-8d89-70cd56d01d94 state: Requested -> Created
112024-12-02T16:55:20.883ZINFOcrucible-agent (worker): applying SMF actions post create...
122024-12-02T16:55:20.896ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-fc5123b9-1db9-48b4-8d89-70cd56d01d94
132024-12-02T16:55:20.901ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-fc5123b9-1db9-48b4-8d89-70cd56d01d94
142024-12-02T16:55:20.914ZINFOcrucible-agent (worker): creating config property group
152024-12-02T16:55:20.918ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-fc5123b9-1db9-48b4-8d89-70cd56d01d94
162024-12-02T16:55:20.919ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fc5123b9-1db9-48b4-8d89-70cd56d01d94
172024-12-02T16:55:20.919ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182024-12-02T16:55:20.919ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
192024-12-02T16:55:20.919ZINFOcrucible-agent (worker): commit
202024-12-02T16:55:20.924ZINFOcrucible-agent (worker): ok!
212024-12-02T16:55:20.930ZINFOcrucible-agent (worker): SMF ok!
222024-12-02T16:55:21.050ZINFOcrucible-agent (dropshot): request completed latency_us = 748 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:45111 req_id = 153859b5-178f-43fb-8907-19b1223134ac response_code = 200 uri = /crucible/0/regions
232024-12-02T16:55:22.175ZINFOcrucible-agent (dropshot): request completed latency_us = 2923 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:45111 req_id = 55a6eb54-6baf-4111-af7f-d0857248ec49 response_code = 200 uri = /crucible/0/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667
242024-12-02T16:55:22.178ZINFOcrucible-agent (datafile): path is "/data/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667"
252024-12-02T16:55:22.186ZINFOcrucible-agent (datafile): dataset is oxp_f2b67eee-c247-4902-8376-b0deafea2c96/crucible/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667
262024-12-02T16:55:22.195ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f2b67eee-c247-4902-8376-b0deafea2c96/crucible/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667@d9d176ee-a185-4bb3-bdac-98f070c5818d
272024-12-02T16:55:22.195ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f2b67eee-c247-4902-8376-b0deafea2c96/crucible/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667", "d9d176ee-a185-4bb3-bdac-98f070c5818d"]
282024-12-02T16:55:22.205ZINFOcrucible-agent (dropshot): request completed latency_us = 27761 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:45111 req_id = b8d9e94c-7b21-4316-804b-87b35581da89 response_code = 200 uri = /crucible/0/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667/snapshots/d9d176ee-a185-4bb3-bdac-98f070c5818d
292024-12-02T16:55:22.209ZINFOcrucible-agent (datafile): path is "/data/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667"
302024-12-02T16:55:22.217ZINFOcrucible-agent (datafile): dataset is oxp_f2b67eee-c247-4902-8376-b0deafea2c96/crucible/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667
312024-12-02T16:55:22.224ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f2b67eee-c247-4902-8376-b0deafea2c96/crucible/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667@d9d176ee-a185-4bb3-bdac-98f070c5818d
322024-12-02T16:55:22.224ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f2b67eee-c247-4902-8376-b0deafea2c96/crucible/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667", "d9d176ee-a185-4bb3-bdac-98f070c5818d"]
332024-12-02T16:55:22.233ZINFOcrucible-agent (datafile): requesting running snapshot b9774cea-2b61-4c7a-8670-f2a33dd35667-d9d176ee-a185-4bb3-bdac-98f070c5818d state: Requested
342024-12-02T16:55:22.233ZINFOcrucible-agent (worker): applying SMF actions for region b9774cea-2b61-4c7a-8670-f2a33dd35667 running snapshot d9d176ee-a185-4bb3-bdac-98f070c5818d (state Requested)...
352024-12-02T16:55:22.233ZINFOcrucible-agent (dropshot): request completed latency_us = 24557 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:45111 req_id = 7a647932-0084-43c0-9ada-0e074221e791 response_code = 200 uri = /crucible/0/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667/snapshots/d9d176ee-a185-4bb3-bdac-98f070c5818d/run
362024-12-02T16:55:22.235ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-b9774cea-2b61-4c7a-8670-f2a33dd35667-d9d176ee-a185-4bb3-bdac-98f070c5818d
372024-12-02T16:55:22.244ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-b9774cea-2b61-4c7a-8670-f2a33dd35667-d9d176ee-a185-4bb3-bdac-98f070c5818d
382024-12-02T16:55:22.257ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-b9774cea-2b61-4c7a-8670-f2a33dd35667-d9d176ee-a185-4bb3-bdac-98f070c5818d
392024-12-02T16:55:22.257ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667/.zfs/snapshot/d9d176ee-a185-4bb3-bdac-98f070c5818d
402024-12-02T16:55:22.257ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412024-12-02T16:55:22.257ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422024-12-02T16:55:22.257ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
432024-12-02T16:55:22.258ZINFOcrucible-agent (worker): commit
442024-12-02T16:55:22.262ZINFOcrucible-agent (worker): ok!
452024-12-02T16:55:22.267ZINFOcrucible-agent (worker): SMF ok!
462024-12-02T16:55:22.267ZINFOcrucible-agent (datafile): region b9774cea-2b61-4c7a-8670-f2a33dd35667 running snapshot d9d176ee-a185-4bb3-bdac-98f070c5818d state: Requested -> Created
472024-12-02T16:55:22.559ZINFOcrucible-agent (datafile): path is "/data/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667"
482024-12-02T16:55:22.568ZINFOcrucible-agent (datafile): dataset is oxp_f2b67eee-c247-4902-8376-b0deafea2c96/crucible/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667
492024-12-02T16:55:22.578ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f2b67eee-c247-4902-8376-b0deafea2c96/crucible/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667@d9d176ee-a185-4bb3-bdac-98f070c5818d
502024-12-02T16:55:22.578ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f2b67eee-c247-4902-8376-b0deafea2c96/crucible/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667", "d9d176ee-a185-4bb3-bdac-98f070c5818d"]
512024-12-02T16:55:22.586ZINFOcrucible-agent (dropshot): request completed latency_us = 26620 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:45111 req_id = 146e8674-1431-4560-be95-3df3ba20c60f response_code = 200 uri = /crucible/0/regions/b9774cea-2b61-4c7a-8670-f2a33dd35667/snapshots/d9d176ee-a185-4bb3-bdac-98f070c5818d/run