Max level shown:
1 [ Oct 18 17:56:19 Enabled. ]
2 [ Oct 18 17:56:19 Rereading configuration. ]
3 [ Oct 18 17:56:21 Rereading configuration. ]
4 [ Oct 18 17:56:22 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_883d1cbf-e1f8-4c86-b18f-d626feb923fd/crucible -l [fd00:1122:3344:101::1a]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-10-18T17:56:23.060ZINFOcrucible-agent: dataset: "oxp_883d1cbf-e1f8-4c86-b18f-d626feb923fd/crucible"
72024-10-18T17:56:23.065ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::1a]:32345
82024-10-18T17:56:23.065ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-10-18T17:56:23.080ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-10-18T17:56:23.222ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::1a]:32345
112024-10-18T17:56:59.219ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::a]:62705
122024-10-18T17:56:59.259ZINFOcrucible-agent (datafile): region d52c189e-5059-497c-8d7f-cf8ac353b2cd state: Requested
132024-10-18T17:56:59.268ZINFOcrucible-agent (dropshot): request completed latency_us = 18199 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62705 req_id = 3b7b72a1-506e-490b-bad5-b97b255087ec response_code = 200 uri = /crucible/0/regions
142024-10-18T17:56:59.268ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152024-10-18T17:56:59.278ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_883d1cbf-e1f8-4c86-b18f-d626feb923fd/crucible/regions/d52c189e-5059-497c-8d7f-cf8ac353b2cd
162024-10-18T17:56:59.278ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_883d1cbf-e1f8-4c86-b18f-d626feb923fd/crucible/regions/d52c189e-5059-497c-8d7f-cf8ac353b2cd
172024-10-18T17:56:59.334ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d52c189e-5059-497c-8d7f-cf8ac353b2cd"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d52c189e-5059-497c-8d7f-cf8ac353b2cd" region = d52c189e-5059-497c-8d7f-cf8ac353b2cd
182024-10-18T17:56:59.339ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d52c189e-5059-497c-8d7f-cf8ac353b2cd" "--data" "/data/regions/d52c189e-5059-497c-8d7f-cf8ac353b2cd" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = d52c189e-5059-497c-8d7f-cf8ac353b2cd
192024-10-18T17:56:59.627ZINFOcrucible-agent (dropshot): request completed latency_us = 83 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62705 req_id = 364d0c12-82fb-4227-9a41-fa518a36c007 response_code = 200 uri = /crucible/0/regions
202024-10-18T17:56:59.645ZINFOcrucible-agent (worker): region files created ok region = d52c189e-5059-497c-8d7f-cf8ac353b2cd
212024-10-18T17:56:59.645ZINFOcrucible-agent (datafile): region d52c189e-5059-497c-8d7f-cf8ac353b2cd state: Requested -> Created
222024-10-18T17:56:59.645ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-10-18T17:56:59.648ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d52c189e-5059-497c-8d7f-cf8ac353b2cd
242024-10-18T17:56:59.654ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d52c189e-5059-497c-8d7f-cf8ac353b2cd
252024-10-18T17:56:59.668ZINFOcrucible-agent (worker): creating config property group
262024-10-18T17:56:59.679ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d52c189e-5059-497c-8d7f-cf8ac353b2cd
272024-10-18T17:56:59.680ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d52c189e-5059-497c-8d7f-cf8ac353b2cd
282024-10-18T17:56:59.680ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-10-18T17:56:59.680ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
302024-10-18T17:56:59.680ZINFOcrucible-agent (worker): commit
312024-10-18T17:56:59.684ZINFOcrucible-agent (worker): ok!
322024-10-18T17:56:59.690ZINFOcrucible-agent (worker): SMF ok!
332024-10-18T17:56:59.963ZINFOcrucible-agent (dropshot): request completed latency_us = 69 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62705 req_id = 7aa2250f-89ff-420c-bea4-fae94e9c1b7f response_code = 200 uri = /crucible/0/regions
342024-10-18T17:57:00.368ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:62705 req_id = e4da0ead-dd78-4d0f-b310-335c57e80cec response_code = 200 uri = /crucible/0/regions/d52c189e-5059-497c-8d7f-cf8ac353b2cd
352024-10-18T17:57:00.371ZINFOcrucible-agent (datafile): path is "/data/regions/d52c189e-5059-497c-8d7f-cf8ac353b2cd"
362024-10-18T17:57:00.381ZINFOcrucible-agent (datafile): dataset is oxp_883d1cbf-e1f8-4c86-b18f-d626feb923fd/crucible/regions/d52c189e-5059-497c-8d7f-cf8ac353b2cd
372024-10-18T17:57:00.392ZINFOcrucible-agent (datafile): region d52c189e-5059-497c-8d7f-cf8ac353b2cd state: Created -> Tombstoned
382024-10-18T17:57:00.392ZINFOcrucible-agent (worker): applying SMF actions before removal...
392024-10-18T17:57:00.392ZINFOcrucible-agent (dropshot): request completed latency_us = 21233 local_addr = [fd00:1122:3344:101::1a]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:62705 req_id = 53d57d2e-73cc-4ea8-871b-8d8e8d1c7c19 response_code = 204 uri = /crucible/0/regions/d52c189e-5059-497c-8d7f-cf8ac353b2cd
402024-10-18T17:57:00.393ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-d52c189e-5059-497c-8d7f-cf8ac353b2cd (instance states: (Some(Online), None))
412024-10-18T17:57:00.395ZINFOcrucible-agent (dropshot): request completed latency_us = 32 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:62705 req_id = e0fc1ff0-27e8-420e-ba5e-284cb318b65d response_code = 200 uri = /crucible/0/regions/d52c189e-5059-497c-8d7f-cf8ac353b2cd
422024-10-18T17:57:00.400ZINFOcrucible-agent (worker): SMF ok!
432024-10-18T17:57:00.412ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_883d1cbf-e1f8-4c86-b18f-d626feb923fd/crucible/regions/d52c189e-5059-497c-8d7f-cf8ac353b2cd" region = d52c189e-5059-497c-8d7f-cf8ac353b2cd
442024-10-18T17:57:00.485ZINFOcrucible-agent (datafile): region d52c189e-5059-497c-8d7f-cf8ac353b2cd state: Tombstoned -> Destroyed
452024-10-18T17:57:00.505ZINFOcrucible-agent (dropshot): request completed latency_us = 68 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:62705 req_id = a9a79ed6-c015-4189-ae75-5ed482f47949 response_code = 200 uri = /crucible/0/regions/d52c189e-5059-497c-8d7f-cf8ac353b2cd
462024-10-18T17:57:02.330ZINFOcrucible-agent (datafile): region d90bf3df-f84b-4e1a-b974-984c22788141 state: Requested
472024-10-18T17:57:02.330ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
482024-10-18T17:57:02.330ZINFOcrucible-agent (dropshot): request completed latency_us = 242 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62705 req_id = 3fa59c66-2731-46d3-b34b-c20d4f0eb847 response_code = 200 uri = /crucible/0/regions
492024-10-18T17:57:02.338ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_883d1cbf-e1f8-4c86-b18f-d626feb923fd/crucible/regions/d90bf3df-f84b-4e1a-b974-984c22788141
502024-10-18T17:57:02.338ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_883d1cbf-e1f8-4c86-b18f-d626feb923fd/crucible/regions/d90bf3df-f84b-4e1a-b974-984c22788141
512024-10-18T17:57:02.414ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d90bf3df-f84b-4e1a-b974-984c22788141"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d90bf3df-f84b-4e1a-b974-984c22788141" region = d90bf3df-f84b-4e1a-b974-984c22788141
522024-10-18T17:57:02.415ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d90bf3df-f84b-4e1a-b974-984c22788141" "--data" "/data/regions/d90bf3df-f84b-4e1a-b974-984c22788141" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = d90bf3df-f84b-4e1a-b974-984c22788141
532024-10-18T17:57:02.485ZINFOcrucible-agent (worker): region files created ok region = d90bf3df-f84b-4e1a-b974-984c22788141
542024-10-18T17:57:02.485ZINFOcrucible-agent (datafile): region d90bf3df-f84b-4e1a-b974-984c22788141 state: Requested -> Created
552024-10-18T17:57:02.485ZINFOcrucible-agent (worker): applying SMF actions post create...
562024-10-18T17:57:02.487ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d90bf3df-f84b-4e1a-b974-984c22788141
572024-10-18T17:57:02.491ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d90bf3df-f84b-4e1a-b974-984c22788141
582024-10-18T17:57:02.499ZINFOcrucible-agent (worker): creating config property group
592024-10-18T17:57:02.505ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d90bf3df-f84b-4e1a-b974-984c22788141
602024-10-18T17:57:02.505ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d90bf3df-f84b-4e1a-b974-984c22788141
612024-10-18T17:57:02.506ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
622024-10-18T17:57:02.506ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
632024-10-18T17:57:02.506ZINFOcrucible-agent (worker): commit
642024-10-18T17:57:02.510ZINFOcrucible-agent (worker): ok!
652024-10-18T17:57:02.515ZINFOcrucible-agent (worker): SMF ok!
662024-10-18T17:57:02.644ZINFOcrucible-agent (dropshot): request completed latency_us = 77 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62705 req_id = ec61af99-ab1a-4f48-bf2f-61c0747cabdd response_code = 200 uri = /crucible/0/regions