Max level shown:
1 [ Nov 21 19:33:05 Enabled. ]
2 [ Nov 21 19:33:05 Rereading configuration. ]
3 [ Nov 21 19:33:07 Rereading configuration. ]
4 [ Nov 21 19:33:11 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible -l [fd00:1122:3344:101::18]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-11-21T19:33:11.088ZINFOcrucible-agent: dataset: "oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible"
72024-11-21T19:33:11.090ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::18]:32345
82024-11-21T19:33:11.090ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-11-21T19:33:11.109ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-11-21T19:33:11.185ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::18]:32345
112024-11-21T19:34:19.926ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:50677
122024-11-21T19:34:19.964ZINFOcrucible-agent (datafile): region 5262a799-d275-4aa5-adae-f686dec6a765 state: Requested
132024-11-21T19:34:19.968ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
142024-11-21T19:34:19.973ZINFOcrucible-agent (dropshot): request completed latency_us = 15433 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50677 req_id = 97a1b3b1-b979-4d5c-8354-aad2db562946 response_code = 200 uri = /crucible/0/regions
152024-11-21T19:34:19.980ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible/regions/5262a799-d275-4aa5-adae-f686dec6a765
162024-11-21T19:34:19.980ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible/regions/5262a799-d275-4aa5-adae-f686dec6a765
172024-11-21T19:34:20.036ZINFOcrucible-agent (worker): creating region Region { id: RegionId("5262a799-d275-4aa5-adae-f686dec6a765"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/5262a799-d275-4aa5-adae-f686dec6a765" region = 5262a799-d275-4aa5-adae-f686dec6a765
182024-11-21T19:34:20.038ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "5262a799-d275-4aa5-adae-f686dec6a765" "--data" "/data/regions/5262a799-d275-4aa5-adae-f686dec6a765" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 5262a799-d275-4aa5-adae-f686dec6a765
192024-11-21T19:34:20.352ZINFOcrucible-agent (dropshot): request completed latency_us = 75 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50677 req_id = df5bdcf1-b82c-4e58-af4b-aef555f4292b response_code = 200 uri = /crucible/0/regions
202024-11-21T19:34:20.372ZINFOcrucible-agent (worker): region files created ok region = 5262a799-d275-4aa5-adae-f686dec6a765
212024-11-21T19:34:20.372ZINFOcrucible-agent (datafile): region 5262a799-d275-4aa5-adae-f686dec6a765 state: Requested -> Created
222024-11-21T19:34:20.373ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-11-21T19:34:20.373ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-5262a799-d275-4aa5-adae-f686dec6a765
242024-11-21T19:34:20.383ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-5262a799-d275-4aa5-adae-f686dec6a765
252024-11-21T19:34:20.410ZINFOcrucible-agent (worker): creating config property group
262024-11-21T19:34:20.419ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-5262a799-d275-4aa5-adae-f686dec6a765
272024-11-21T19:34:20.419ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/5262a799-d275-4aa5-adae-f686dec6a765
282024-11-21T19:34:20.419ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-11-21T19:34:20.420ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
302024-11-21T19:34:20.420ZINFOcrucible-agent (worker): commit
312024-11-21T19:34:20.426ZINFOcrucible-agent (worker): ok!
322024-11-21T19:34:20.435ZINFOcrucible-agent (worker): SMF ok!
332024-11-21T19:34:20.958ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50677 req_id = 43f0f620-c350-4412-8989-af257210b99a response_code = 200 uri = /crucible/0/regions
342024-11-21T19:44:51.997ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:53408
352024-11-21T19:44:52.000ZINFOcrucible-agent (dropshot): request completed latency_us = 3488 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:53408 req_id = 40c219ed-6025-45b6-a4ae-31e1f70ed0fe response_code = 200 uri = /crucible/0/regions/5262a799-d275-4aa5-adae-f686dec6a765
362024-11-21T19:44:52.004ZINFOcrucible-agent (datafile): path is "/data/regions/5262a799-d275-4aa5-adae-f686dec6a765"
372024-11-21T19:44:52.014ZINFOcrucible-agent (datafile): dataset is oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible/regions/5262a799-d275-4aa5-adae-f686dec6a765
382024-11-21T19:44:52.022ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible/regions/5262a799-d275-4aa5-adae-f686dec6a765@728cd9b1-e3ec-4e7d-b786-e0e10f0569d2
392024-11-21T19:44:52.023ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible/regions/5262a799-d275-4aa5-adae-f686dec6a765", "728cd9b1-e3ec-4e7d-b786-e0e10f0569d2"]
402024-11-21T19:44:52.034ZINFOcrucible-agent (dropshot): request completed latency_us = 30290 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:53408 req_id = 531009d1-5657-44de-b1ba-db637658b315 response_code = 200 uri = /crucible/0/regions/5262a799-d275-4aa5-adae-f686dec6a765/snapshots/728cd9b1-e3ec-4e7d-b786-e0e10f0569d2
412024-11-21T19:44:52.038ZINFOcrucible-agent (datafile): path is "/data/regions/5262a799-d275-4aa5-adae-f686dec6a765"
422024-11-21T19:44:52.047ZINFOcrucible-agent (datafile): dataset is oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible/regions/5262a799-d275-4aa5-adae-f686dec6a765
432024-11-21T19:44:52.055ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible/regions/5262a799-d275-4aa5-adae-f686dec6a765@728cd9b1-e3ec-4e7d-b786-e0e10f0569d2
442024-11-21T19:44:52.055ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible/regions/5262a799-d275-4aa5-adae-f686dec6a765", "728cd9b1-e3ec-4e7d-b786-e0e10f0569d2"]
452024-11-21T19:44:52.064ZINFOcrucible-agent (datafile): requesting running snapshot 5262a799-d275-4aa5-adae-f686dec6a765-728cd9b1-e3ec-4e7d-b786-e0e10f0569d2 state: Requested
462024-11-21T19:44:52.064ZINFOcrucible-agent (worker): applying SMF actions for region 5262a799-d275-4aa5-adae-f686dec6a765 running snapshot 728cd9b1-e3ec-4e7d-b786-e0e10f0569d2 (state Requested)...
472024-11-21T19:44:52.064ZINFOcrucible-agent (dropshot): request completed latency_us = 26123 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:53408 req_id = cbdaf5fb-c93d-434f-ad69-b83a98a7b9b1 response_code = 200 uri = /crucible/0/regions/5262a799-d275-4aa5-adae-f686dec6a765/snapshots/728cd9b1-e3ec-4e7d-b786-e0e10f0569d2/run
482024-11-21T19:44:52.074ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-5262a799-d275-4aa5-adae-f686dec6a765-728cd9b1-e3ec-4e7d-b786-e0e10f0569d2
492024-11-21T19:44:52.081ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-5262a799-d275-4aa5-adae-f686dec6a765-728cd9b1-e3ec-4e7d-b786-e0e10f0569d2
502024-11-21T19:44:52.099ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-5262a799-d275-4aa5-adae-f686dec6a765-728cd9b1-e3ec-4e7d-b786-e0e10f0569d2
512024-11-21T19:44:52.099ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/5262a799-d275-4aa5-adae-f686dec6a765/.zfs/snapshot/728cd9b1-e3ec-4e7d-b786-e0e10f0569d2
522024-11-21T19:44:52.099ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
532024-11-21T19:44:52.099ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
542024-11-21T19:44:52.099ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
552024-11-21T19:44:52.099ZINFOcrucible-agent (worker): commit
562024-11-21T19:44:52.107ZINFOcrucible-agent (worker): ok!
572024-11-21T19:44:52.114ZINFOcrucible-agent (worker): SMF ok!
582024-11-21T19:44:52.114ZINFOcrucible-agent (datafile): region 5262a799-d275-4aa5-adae-f686dec6a765 running snapshot 728cd9b1-e3ec-4e7d-b786-e0e10f0569d2 state: Requested -> Created
592024-11-21T19:44:52.402ZINFOcrucible-agent (datafile): path is "/data/regions/5262a799-d275-4aa5-adae-f686dec6a765"
602024-11-21T19:44:52.411ZINFOcrucible-agent (datafile): dataset is oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible/regions/5262a799-d275-4aa5-adae-f686dec6a765
612024-11-21T19:44:52.420ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible/regions/5262a799-d275-4aa5-adae-f686dec6a765@728cd9b1-e3ec-4e7d-b786-e0e10f0569d2
622024-11-21T19:44:52.420ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_8a7d01cd-9218-405e-81a7-0632ac6eff3f/crucible/regions/5262a799-d275-4aa5-adae-f686dec6a765", "728cd9b1-e3ec-4e7d-b786-e0e10f0569d2"]
632024-11-21T19:44:52.428ZINFOcrucible-agent (dropshot): request completed latency_us = 25489 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:53408 req_id = adc35333-4b41-4355-b6f5-29ac6f242f08 response_code = 200 uri = /crucible/0/regions/5262a799-d275-4aa5-adae-f686dec6a765/snapshots/728cd9b1-e3ec-4e7d-b786-e0e10f0569d2/run