Max level shown:
1 [ Nov 26 23:31:56 Enabled. ]
2 [ Nov 26 23:31:56 Rereading configuration. ]
3 [ Nov 26 23:31:58 Rereading configuration. ]
4 [ Nov 26 23:32:01 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible -l [fd00:1122:3344:101::13]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-11-26T23:32:01.369ZINFOcrucible-agent: dataset: "oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible"
72024-11-26T23:32:01.373ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::13]:32345
82024-11-26T23:32:01.373ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-11-26T23:32:01.390ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-11-26T23:32:01.452ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::13]:32345
112024-11-26T23:33:11.765ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::a]:43413
122024-11-26T23:33:11.797ZINFOcrucible-agent (datafile): region aa520c41-96fb-4108-a699-b9de672e29f7 state: Requested
132024-11-26T23:33:11.802ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
142024-11-26T23:33:11.807ZINFOcrucible-agent (dropshot): request completed latency_us = 15553 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43413 req_id = d7adb0ce-90cb-49e9-a955-0bea4dc109e3 response_code = 200 uri = /crucible/0/regions
152024-11-26T23:33:11.814ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible/regions/aa520c41-96fb-4108-a699-b9de672e29f7
162024-11-26T23:33:11.814ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible/regions/aa520c41-96fb-4108-a699-b9de672e29f7
172024-11-26T23:33:11.884ZINFOcrucible-agent (worker): creating region Region { id: RegionId("aa520c41-96fb-4108-a699-b9de672e29f7"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/aa520c41-96fb-4108-a699-b9de672e29f7" region = aa520c41-96fb-4108-a699-b9de672e29f7
182024-11-26T23:33:11.886ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "aa520c41-96fb-4108-a699-b9de672e29f7" "--data" "/data/regions/aa520c41-96fb-4108-a699-b9de672e29f7" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = aa520c41-96fb-4108-a699-b9de672e29f7
192024-11-26T23:33:12.133ZINFOcrucible-agent (worker): region files created ok region = aa520c41-96fb-4108-a699-b9de672e29f7
202024-11-26T23:33:12.133ZINFOcrucible-agent (datafile): region aa520c41-96fb-4108-a699-b9de672e29f7 state: Requested -> Created
212024-11-26T23:33:12.134ZINFOcrucible-agent (worker): applying SMF actions post create...
222024-11-26T23:33:12.134ZINFOcrucible-agent (dropshot): request completed latency_us = 100 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43413 req_id = 3e14918a-3984-4b63-97a5-f6019b599b3d response_code = 200 uri = /crucible/0/regions
232024-11-26T23:33:12.134ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-aa520c41-96fb-4108-a699-b9de672e29f7
242024-11-26T23:33:12.141ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-aa520c41-96fb-4108-a699-b9de672e29f7
252024-11-26T23:33:12.154ZINFOcrucible-agent (worker): creating config property group
262024-11-26T23:33:12.157ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-aa520c41-96fb-4108-a699-b9de672e29f7
272024-11-26T23:33:12.158ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/aa520c41-96fb-4108-a699-b9de672e29f7
282024-11-26T23:33:12.158ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-11-26T23:33:12.158ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
302024-11-26T23:33:12.158ZINFOcrucible-agent (worker): commit
312024-11-26T23:33:12.162ZINFOcrucible-agent (worker): ok!
322024-11-26T23:33:12.167ZINFOcrucible-agent (worker): SMF ok!
332024-11-26T23:43:34.217ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::a]:43751
342024-11-26T23:43:34.220ZINFOcrucible-agent (dropshot): request completed latency_us = 3237 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:43751 req_id = ead3a064-f464-46dc-b53a-b84a745c7da1 response_code = 200 uri = /crucible/0/regions/aa520c41-96fb-4108-a699-b9de672e29f7
352024-11-26T23:43:34.224ZINFOcrucible-agent (datafile): path is "/data/regions/aa520c41-96fb-4108-a699-b9de672e29f7"
362024-11-26T23:43:34.235ZINFOcrucible-agent (datafile): dataset is oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible/regions/aa520c41-96fb-4108-a699-b9de672e29f7
372024-11-26T23:43:34.246ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible/regions/aa520c41-96fb-4108-a699-b9de672e29f7@70613474-4464-4e96-8c54-9d160de7b02c
382024-11-26T23:43:34.246ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible/regions/aa520c41-96fb-4108-a699-b9de672e29f7", "70613474-4464-4e96-8c54-9d160de7b02c"]
392024-11-26T23:43:34.257ZINFOcrucible-agent (dropshot): request completed latency_us = 33575 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:43751 req_id = 07e7930d-1135-4882-bdf0-f2dd3c8489b8 response_code = 200 uri = /crucible/0/regions/aa520c41-96fb-4108-a699-b9de672e29f7/snapshots/70613474-4464-4e96-8c54-9d160de7b02c
402024-11-26T23:43:34.262ZINFOcrucible-agent (datafile): path is "/data/regions/aa520c41-96fb-4108-a699-b9de672e29f7"
412024-11-26T23:43:34.271ZINFOcrucible-agent (datafile): dataset is oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible/regions/aa520c41-96fb-4108-a699-b9de672e29f7
422024-11-26T23:43:34.279ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible/regions/aa520c41-96fb-4108-a699-b9de672e29f7@70613474-4464-4e96-8c54-9d160de7b02c
432024-11-26T23:43:34.279ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible/regions/aa520c41-96fb-4108-a699-b9de672e29f7", "70613474-4464-4e96-8c54-9d160de7b02c"]
442024-11-26T23:43:34.288ZINFOcrucible-agent (datafile): requesting running snapshot aa520c41-96fb-4108-a699-b9de672e29f7-70613474-4464-4e96-8c54-9d160de7b02c state: Requested
452024-11-26T23:43:34.289ZINFOcrucible-agent (worker): applying SMF actions for region aa520c41-96fb-4108-a699-b9de672e29f7 running snapshot 70613474-4464-4e96-8c54-9d160de7b02c (state Requested)...
462024-11-26T23:43:34.289ZINFOcrucible-agent (dropshot): request completed latency_us = 27085 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43751 req_id = e22e3b92-c959-4286-aab8-b004db24e977 response_code = 200 uri = /crucible/0/regions/aa520c41-96fb-4108-a699-b9de672e29f7/snapshots/70613474-4464-4e96-8c54-9d160de7b02c/run
472024-11-26T23:43:34.299ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-aa520c41-96fb-4108-a699-b9de672e29f7-70613474-4464-4e96-8c54-9d160de7b02c
482024-11-26T23:43:34.303ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-aa520c41-96fb-4108-a699-b9de672e29f7-70613474-4464-4e96-8c54-9d160de7b02c
492024-11-26T23:43:34.317ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-aa520c41-96fb-4108-a699-b9de672e29f7-70613474-4464-4e96-8c54-9d160de7b02c
502024-11-26T23:43:34.317ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/aa520c41-96fb-4108-a699-b9de672e29f7/.zfs/snapshot/70613474-4464-4e96-8c54-9d160de7b02c
512024-11-26T23:43:34.317ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
522024-11-26T23:43:34.317ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
532024-11-26T23:43:34.318ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
542024-11-26T23:43:34.318ZINFOcrucible-agent (worker): commit
552024-11-26T23:43:34.322ZINFOcrucible-agent (worker): ok!
562024-11-26T23:43:34.327ZINFOcrucible-agent (worker): SMF ok!
572024-11-26T23:43:34.327ZINFOcrucible-agent (datafile): region aa520c41-96fb-4108-a699-b9de672e29f7 running snapshot 70613474-4464-4e96-8c54-9d160de7b02c state: Requested -> Created
582024-11-26T23:43:34.512ZINFOcrucible-agent (datafile): path is "/data/regions/aa520c41-96fb-4108-a699-b9de672e29f7"
592024-11-26T23:43:34.520ZINFOcrucible-agent (datafile): dataset is oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible/regions/aa520c41-96fb-4108-a699-b9de672e29f7
602024-11-26T23:43:34.529ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible/regions/aa520c41-96fb-4108-a699-b9de672e29f7@70613474-4464-4e96-8c54-9d160de7b02c
612024-11-26T23:43:34.529ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_c4c104fb-1c27-49e0-888c-318f50cd7fdc/crucible/regions/aa520c41-96fb-4108-a699-b9de672e29f7", "70613474-4464-4e96-8c54-9d160de7b02c"]
622024-11-26T23:43:34.537ZINFOcrucible-agent (dropshot): request completed latency_us = 26097 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43751 req_id = c1ca4728-dd3f-4ad2-bd97-86f0b98416b9 response_code = 200 uri = /crucible/0/regions/aa520c41-96fb-4108-a699-b9de672e29f7/snapshots/70613474-4464-4e96-8c54-9d160de7b02c/run