Max level shown:
12024-09-05T00:48:50.975ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:32777
22024-09-05T00:48:51.002ZINFOcrucible-agent (datafile): region 030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6 state: Requested
32024-09-05T00:48:51.010ZINFOcrucible-agent (dropshot): request completed latency_us = 17359 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:32777 req_id = ba127b41-8789-4391-9e90-29b51884e4aa response_code = 200 uri = /crucible/0/regions
42024-09-05T00:48:51.010ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52024-09-05T00:48:51.029ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_5e93ddc1-113d-4d72-b455-60d0ee285e5b/crucible/regions/030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
62024-09-05T00:48:51.029ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_5e93ddc1-113d-4d72-b455-60d0ee285e5b/crucible/regions/030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
72024-09-05T00:48:51.096ZINFOcrucible-agent (worker): creating region Region { id: RegionId("030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6" region = 030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
82024-09-05T00:48:51.099ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6" "--data" "/data/regions/030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
92024-09-05T00:48:51.255ZINFOcrucible-agent (dropshot): request completed latency_us = 84 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:32777 req_id = 12a1680d-c31d-4651-accc-aa8f3578a3c6 response_code = 200 uri = /crucible/0/regions
102024-09-05T00:48:51.448ZINFOcrucible-agent (worker): region files created ok region = 030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
112024-09-05T00:48:51.448ZINFOcrucible-agent (datafile): region 030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6 state: Requested -> Created
122024-09-05T00:48:51.448ZINFOcrucible-agent (worker): applying SMF actions post create...
132024-09-05T00:48:51.448ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
142024-09-05T00:48:51.455ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
152024-09-05T00:48:51.478ZINFOcrucible-agent (worker): creating config property group
162024-09-05T00:48:51.484ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
172024-09-05T00:48:51.494ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
182024-09-05T00:48:51.495ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192024-09-05T00:48:51.496ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
202024-09-05T00:48:51.496ZINFOcrucible-agent (worker): commit
212024-09-05T00:48:51.502ZINFOcrucible-agent (worker): ok!
222024-09-05T00:48:51.508ZINFOcrucible-agent (worker): SMF ok!
232024-09-05T00:48:51.736ZINFOcrucible-agent (dropshot): request completed latency_us = 86 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:32777 req_id = c72baa9c-544b-4729-a02e-be5c56f2a06a response_code = 200 uri = /crucible/0/regions
242024-09-05T00:48:56.285ZINFOcrucible-agent (datafile): region 7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d state: Requested
252024-09-05T00:48:56.285ZINFOcrucible-agent (dropshot): request completed latency_us = 243 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:32777 req_id = 437f940d-7244-4c4b-ab25-44a9940be88c response_code = 200 uri = /crucible/0/regions
262024-09-05T00:48:56.285ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
272024-09-05T00:48:56.296ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_5e93ddc1-113d-4d72-b455-60d0ee285e5b/crucible/regions/7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
282024-09-05T00:48:56.296ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_5e93ddc1-113d-4d72-b455-60d0ee285e5b/crucible/regions/7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
292024-09-05T00:48:56.377ZINFOcrucible-agent (worker): creating region Region { id: RegionId("7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d" region = 7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
302024-09-05T00:48:56.377ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d" "--data" "/data/regions/7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
312024-09-05T00:48:56.482ZINFOcrucible-agent (worker): region files created ok region = 7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
322024-09-05T00:48:56.482ZINFOcrucible-agent (datafile): region 7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d state: Requested -> Created
332024-09-05T00:48:56.482ZINFOcrucible-agent (worker): applying SMF actions post create...
342024-09-05T00:48:56.483ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
352024-09-05T00:48:56.484ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
362024-09-05T00:48:56.484ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::16
372024-09-05T00:48:56.484ZINFOcrucible-agent (worker): reconfig required
382024-09-05T00:48:56.484ZINFOcrucible-agent (worker): using existing config property group
392024-09-05T00:48:56.484ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
402024-09-05T00:48:56.484ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
412024-09-05T00:48:56.484ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
422024-09-05T00:48:56.484ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
432024-09-05T00:48:56.484ZINFOcrucible-agent (worker): commit
442024-09-05T00:48:56.491ZINFOcrucible-agent (worker): ok!
452024-09-05T00:48:56.492ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
462024-09-05T00:48:56.497ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
472024-09-05T00:48:56.502ZINFOcrucible-agent (dropshot): request completed latency_us = 78 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:32777 req_id = b0aebe8b-312c-417b-a5b1-1ee30c015831 response_code = 200 uri = /crucible/0/regions
482024-09-05T00:48:56.512ZINFOcrucible-agent (worker): creating config property group
492024-09-05T00:48:56.521ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
502024-09-05T00:48:56.521ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
512024-09-05T00:48:56.521ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
522024-09-05T00:48:56.521ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
532024-09-05T00:48:56.521ZINFOcrucible-agent (worker): commit
542024-09-05T00:48:56.528ZINFOcrucible-agent (worker): ok!
552024-09-05T00:48:56.537ZINFOcrucible-agent (worker): SMF ok!
562024-09-05T00:49:22.724ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:32777 req_id = fa6577a9-1119-4873-803d-04bca3327a07 response_code = 200 uri = /crucible/0/regions/7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
572024-09-05T00:49:22.727ZINFOcrucible-agent (datafile): path is "/data/regions/7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d"
582024-09-05T00:49:22.736ZINFOcrucible-agent (datafile): dataset is oxp_5e93ddc1-113d-4d72-b455-60d0ee285e5b/crucible/regions/7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
592024-09-05T00:49:22.744ZINFOcrucible-agent (datafile): region 7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d state: Created -> Tombstoned
602024-09-05T00:49:22.745ZINFOcrucible-agent (worker): applying SMF actions before removal...
612024-09-05T00:49:22.745ZINFOcrucible-agent (dropshot): request completed latency_us = 18074 local_addr = [fd00:1122:3344:101::16]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:32777 req_id = 6a9cee52-53d2-4d5f-aa11-15e6faba2d17 response_code = 204 uri = /crucible/0/regions/7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
622024-09-05T00:49:22.746ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d (instance states: (Some(Online), None))
632024-09-05T00:49:22.748ZINFOcrucible-agent (dropshot): request completed latency_us = 21 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:32777 req_id = ad5ad4e8-d157-4499-bb45-ef19ddccfca9 response_code = 200 uri = /crucible/0/regions/7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
642024-09-05T00:49:22.751ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
652024-09-05T00:49:22.751ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
662024-09-05T00:49:22.751ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::16
672024-09-05T00:49:22.751ZINFOcrucible-agent (worker): reconfig required
682024-09-05T00:49:22.751ZINFOcrucible-agent (worker): using existing config property group
692024-09-05T00:49:22.751ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
702024-09-05T00:49:22.751ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/030b39ce-e1a9-4b6a-a36e-a9e5bbb8e7a6
712024-09-05T00:49:22.751ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
722024-09-05T00:49:22.751ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
732024-09-05T00:49:22.751ZINFOcrucible-agent (worker): commit
742024-09-05T00:49:22.756ZINFOcrucible-agent (worker): ok!
752024-09-05T00:49:22.756ZINFOcrucible-agent (worker): SMF ok!
762024-09-05T00:49:22.766ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_5e93ddc1-113d-4d72-b455-60d0ee285e5b/crucible/regions/7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d" region = 7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d
772024-09-05T00:49:22.816ZINFOcrucible-agent (datafile): region 7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d state: Tombstoned -> Destroyed
782024-09-05T00:49:22.835ZINFOcrucible-agent (dropshot): request completed latency_us = 58 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:32777 req_id = 9453b069-2c14-4269-9c0c-4a8957fbf570 response_code = 200 uri = /crucible/0/regions/7e130e78-2c6a-4976-9fa2-1ce3ec05fc3d