Max level shown:
1 [ Oct 22 17:00:01 Enabled. ]
2 [ Oct 22 17:00:01 Rereading configuration. ]
3 [ Oct 22 17:00:03 Rereading configuration. ]
4 [ Oct 22 17:00:04 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_49cf1fae-a556-4427-9025-e342dceede7e/crucible -l [fd00:1122:3344:101::14]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-10-22T17:00:04.499ZINFOcrucible-agent: dataset: "oxp_49cf1fae-a556-4427-9025-e342dceede7e/crucible"
72024-10-22T17:00:04.502ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::14]:32345
82024-10-22T17:00:04.502ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-10-22T17:00:04.549ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-10-22T17:00:04.790ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::14]:32345
112024-10-22T17:01:08.785ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::b]:65355
122024-10-22T17:01:08.817ZINFOcrucible-agent (datafile): region d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96 state: Requested
132024-10-22T17:01:08.822ZINFOcrucible-agent (dropshot): request completed latency_us = 12904 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:65355 req_id = bd6aeedb-6e30-4a5f-bba2-9a745ed181e5 response_code = 200 uri = /crucible/0/regions
142024-10-22T17:01:08.823ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152024-10-22T17:01:08.832ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_49cf1fae-a556-4427-9025-e342dceede7e/crucible/regions/d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
162024-10-22T17:01:08.832ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_49cf1fae-a556-4427-9025-e342dceede7e/crucible/regions/d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
172024-10-22T17:01:08.897ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96" region = d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
182024-10-22T17:01:08.901ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96" "--data" "/data/regions/d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
192024-10-22T17:01:09.017ZINFOcrucible-agent (worker): region files created ok region = d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
202024-10-22T17:01:09.017ZINFOcrucible-agent (datafile): region d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96 state: Requested -> Created
212024-10-22T17:01:09.017ZINFOcrucible-agent (worker): applying SMF actions post create...
222024-10-22T17:01:09.020ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
232024-10-22T17:01:09.025ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
242024-10-22T17:01:09.041ZINFOcrucible-agent (worker): creating config property group
252024-10-22T17:01:09.044ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
262024-10-22T17:01:09.051ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
272024-10-22T17:01:09.051ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
282024-10-22T17:01:09.051ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
292024-10-22T17:01:09.051ZINFOcrucible-agent (worker): commit
302024-10-22T17:01:09.055ZINFOcrucible-agent (worker): ok!
312024-10-22T17:01:09.059ZINFOcrucible-agent (worker): SMF ok!
322024-10-22T17:01:09.173ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:65355 req_id = ea53f765-4519-4dd8-93bf-e06c9c80ebe8 response_code = 200 uri = /crucible/0/regions
332024-10-22T17:01:10.140ZINFOcrucible-agent (dropshot): request completed latency_us = 77 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:65355 req_id = 35b75bb9-ce1b-47bc-ac14-6777c37f962c response_code = 200 uri = /crucible/0/regions/d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
342024-10-22T17:01:10.142ZINFOcrucible-agent (datafile): path is "/data/regions/d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96"
352024-10-22T17:01:10.151ZINFOcrucible-agent (datafile): dataset is oxp_49cf1fae-a556-4427-9025-e342dceede7e/crucible/regions/d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
362024-10-22T17:01:10.160ZINFOcrucible-agent (datafile): region d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96 state: Created -> Tombstoned
372024-10-22T17:01:10.160ZINFOcrucible-agent (worker): applying SMF actions before removal...
382024-10-22T17:01:10.160ZINFOcrucible-agent (dropshot): request completed latency_us = 18184 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:65355 req_id = 3590e424-d394-4575-8c99-a3f3c4f50b9a response_code = 204 uri = /crucible/0/regions/d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
392024-10-22T17:01:10.161ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96 (instance states: (Some(Online), None))
402024-10-22T17:01:10.165ZINFOcrucible-agent (dropshot): request completed latency_us = 30 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:65355 req_id = 96f342df-5a7b-4ec7-bb22-bb90e928a21f response_code = 200 uri = /crucible/0/regions/d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
412024-10-22T17:01:10.166ZINFOcrucible-agent (worker): SMF ok!
422024-10-22T17:01:10.175ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_49cf1fae-a556-4427-9025-e342dceede7e/crucible/regions/d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96" region = d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
432024-10-22T17:01:10.227ZINFOcrucible-agent (datafile): region d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96 state: Tombstoned -> Destroyed
442024-10-22T17:01:10.241ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:65355 req_id = 474edb9f-27c5-40d5-8ac3-3d600e20e66c response_code = 200 uri = /crucible/0/regions/d84f5f54-3fa8-47a4-9fb0-38a12d3f8e96
452024-10-22T17:04:43.800ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:65471
462024-10-22T17:04:43.801ZINFOcrucible-agent (datafile): region c179ded2-4e80-4b21-83b7-5d925cf11aba state: Requested
472024-10-22T17:04:43.805ZINFOcrucible-agent (dropshot): request completed latency_us = 5020 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:65471 req_id = 9dee6d51-cb49-4a54-b39a-8db8d261931a response_code = 200 uri = /crucible/0/regions
482024-10-22T17:04:43.806ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-10-22T17:04:43.816ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_49cf1fae-a556-4427-9025-e342dceede7e/crucible/regions/c179ded2-4e80-4b21-83b7-5d925cf11aba
502024-10-22T17:04:43.816ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_49cf1fae-a556-4427-9025-e342dceede7e/crucible/regions/c179ded2-4e80-4b21-83b7-5d925cf11aba
512024-10-22T17:04:44.044ZINFOcrucible-agent (dropshot): request completed latency_us = 795 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:65471 req_id = b4338c0d-4cf4-4fad-8d20-c70e4f811da1 response_code = 200 uri = /crucible/0/regions
522024-10-22T17:04:44.075ZINFOcrucible-agent (worker): creating region Region { id: RegionId("c179ded2-4e80-4b21-83b7-5d925cf11aba"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/c179ded2-4e80-4b21-83b7-5d925cf11aba" region = c179ded2-4e80-4b21-83b7-5d925cf11aba
532024-10-22T17:04:44.077ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "c179ded2-4e80-4b21-83b7-5d925cf11aba" "--data" "/data/regions/c179ded2-4e80-4b21-83b7-5d925cf11aba" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = c179ded2-4e80-4b21-83b7-5d925cf11aba
542024-10-22T17:04:44.537ZINFOcrucible-agent (worker): region files created ok region = c179ded2-4e80-4b21-83b7-5d925cf11aba
552024-10-22T17:04:44.537ZINFOcrucible-agent (datafile): region c179ded2-4e80-4b21-83b7-5d925cf11aba state: Requested -> Created
562024-10-22T17:04:44.537ZINFOcrucible-agent (worker): applying SMF actions post create...
572024-10-22T17:04:44.539ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-c179ded2-4e80-4b21-83b7-5d925cf11aba
582024-10-22T17:04:44.569ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-c179ded2-4e80-4b21-83b7-5d925cf11aba
592024-10-22T17:04:44.616ZINFOcrucible-agent (dropshot): request completed latency_us = 78 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:65471 req_id = 2761b2a3-9e2c-4e18-8cc5-e825b159ddb8 response_code = 200 uri = /crucible/0/regions
602024-10-22T17:04:44.640ZINFOcrucible-agent (worker): creating config property group
612024-10-22T17:04:44.695ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-c179ded2-4e80-4b21-83b7-5d925cf11aba
622024-10-22T17:04:44.695ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/c179ded2-4e80-4b21-83b7-5d925cf11aba
632024-10-22T17:04:44.695ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
642024-10-22T17:04:44.695ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
652024-10-22T17:04:44.695ZINFOcrucible-agent (worker): commit
662024-10-22T17:04:44.768ZINFOcrucible-agent (worker): ok!
672024-10-22T17:04:44.801ZINFOcrucible-agent (worker): SMF ok!