Max level shown:
12024-08-28T02:17:34.515ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::a]:52341
22024-08-28T02:17:34.542ZINFOcrucible-agent (datafile): region 971d742a-9853-4091-b634-ef2d86183201 state: Requested
32024-08-28T02:17:34.549ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42024-08-28T02:17:34.553ZINFOcrucible-agent (dropshot): request completed latency_us = 18120 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52341 req_id = ed53e7f4-204d-4d4e-a705-7748aaf28d60 response_code = 200 uri = /crucible/0/regions
52024-08-28T02:17:34.562ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_79e9b421-598b-4117-bcfb-c4abc62898c1/crucible/regions/971d742a-9853-4091-b634-ef2d86183201
62024-08-28T02:17:34.562ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_79e9b421-598b-4117-bcfb-c4abc62898c1/crucible/regions/971d742a-9853-4091-b634-ef2d86183201
72024-08-28T02:17:34.628ZINFOcrucible-agent (worker): creating region Region { id: RegionId("971d742a-9853-4091-b634-ef2d86183201"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/971d742a-9853-4091-b634-ef2d86183201" region = 971d742a-9853-4091-b634-ef2d86183201
82024-08-28T02:17:34.630ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "971d742a-9853-4091-b634-ef2d86183201" "--data" "/data/regions/971d742a-9853-4091-b634-ef2d86183201" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 971d742a-9853-4091-b634-ef2d86183201
92024-08-28T02:17:34.896ZINFOcrucible-agent (dropshot): request completed latency_us = 80 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52341 req_id = 175485b9-06fc-455d-8f1f-55f2680e448e response_code = 200 uri = /crucible/0/regions
102024-08-28T02:17:34.923ZINFOcrucible-agent (worker): region files created ok region = 971d742a-9853-4091-b634-ef2d86183201
112024-08-28T02:17:34.923ZINFOcrucible-agent (datafile): region 971d742a-9853-4091-b634-ef2d86183201 state: Requested -> Created
122024-08-28T02:17:34.924ZINFOcrucible-agent (worker): applying SMF actions post create...
132024-08-28T02:17:34.924ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-971d742a-9853-4091-b634-ef2d86183201
142024-08-28T02:17:34.932ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-971d742a-9853-4091-b634-ef2d86183201
152024-08-28T02:17:34.951ZINFOcrucible-agent (worker): creating config property group
162024-08-28T02:17:34.953ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-971d742a-9853-4091-b634-ef2d86183201
172024-08-28T02:17:34.962ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/971d742a-9853-4091-b634-ef2d86183201
182024-08-28T02:17:34.964ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192024-08-28T02:17:34.964ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
202024-08-28T02:17:34.965ZINFOcrucible-agent (worker): commit
212024-08-28T02:17:34.969ZINFOcrucible-agent (worker): ok!
222024-08-28T02:17:34.974ZINFOcrucible-agent (worker): SMF ok!
232024-08-28T02:17:35.392ZINFOcrucible-agent (dropshot): request completed latency_us = 86 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52341 req_id = 01b65a2b-82a4-4343-88fc-b84ce6c5ae32 response_code = 200 uri = /crucible/0/regions
242024-08-28T02:17:40.157ZINFOcrucible-agent (datafile): region 4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f state: Requested
252024-08-28T02:17:40.158ZINFOcrucible-agent (dropshot): request completed latency_us = 939 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52341 req_id = d36717ed-c1e4-4146-8a39-36ac0c34c8b0 response_code = 200 uri = /crucible/0/regions
262024-08-28T02:17:40.158ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
272024-08-28T02:17:40.167ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_79e9b421-598b-4117-bcfb-c4abc62898c1/crucible/regions/4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
282024-08-28T02:17:40.167ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_79e9b421-598b-4117-bcfb-c4abc62898c1/crucible/regions/4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
292024-08-28T02:17:40.219ZINFOcrucible-agent (worker): creating region Region { id: RegionId("4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f" region = 4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
302024-08-28T02:17:40.219ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f" "--data" "/data/regions/4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
312024-08-28T02:17:40.300ZINFOcrucible-agent (worker): region files created ok region = 4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
322024-08-28T02:17:40.300ZINFOcrucible-agent (datafile): region 4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f state: Requested -> Created
332024-08-28T02:17:40.300ZINFOcrucible-agent (worker): applying SMF actions post create...
342024-08-28T02:17:40.300ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
352024-08-28T02:17:40.303ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
362024-08-28T02:17:40.310ZINFOcrucible-agent (worker): creating config property group
372024-08-28T02:17:40.316ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
382024-08-28T02:17:40.317ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
392024-08-28T02:17:40.317ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
402024-08-28T02:17:40.317ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
412024-08-28T02:17:40.317ZINFOcrucible-agent (worker): commit
422024-08-28T02:17:40.320ZINFOcrucible-agent (worker): ok!
432024-08-28T02:17:40.326ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/971d742a-9853-4091-b634-ef2d86183201
442024-08-28T02:17:40.326ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
452024-08-28T02:17:40.326ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::13
462024-08-28T02:17:40.326ZINFOcrucible-agent (worker): reconfig required
472024-08-28T02:17:40.326ZINFOcrucible-agent (worker): using existing config property group
482024-08-28T02:17:40.326ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-971d742a-9853-4091-b634-ef2d86183201
492024-08-28T02:17:40.326ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/971d742a-9853-4091-b634-ef2d86183201
502024-08-28T02:17:40.326ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
512024-08-28T02:17:40.326ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
522024-08-28T02:17:40.326ZINFOcrucible-agent (worker): commit
532024-08-28T02:17:40.334ZINFOcrucible-agent (worker): ok!
542024-08-28T02:17:40.335ZINFOcrucible-agent (worker): SMF ok!
552024-08-28T02:17:40.507ZINFOcrucible-agent (dropshot): request completed latency_us = 89 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52341 req_id = 7a7fef55-2a02-4da9-bfee-38bcb6a71649 response_code = 200 uri = /crucible/0/regions
562024-08-28T02:18:06.004ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:52341 req_id = 7060997e-5111-4739-8767-c058773631fd response_code = 200 uri = /crucible/0/regions/4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
572024-08-28T02:18:06.007ZINFOcrucible-agent (datafile): path is "/data/regions/4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f"
582024-08-28T02:18:06.017ZINFOcrucible-agent (datafile): dataset is oxp_79e9b421-598b-4117-bcfb-c4abc62898c1/crucible/regions/4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
592024-08-28T02:18:06.025ZINFOcrucible-agent (datafile): region 4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f state: Created -> Tombstoned
602024-08-28T02:18:06.026ZINFOcrucible-agent (worker): applying SMF actions before removal...
612024-08-28T02:18:06.026ZINFOcrucible-agent (dropshot): request completed latency_us = 18502 local_addr = [fd00:1122:3344:101::13]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:52341 req_id = c229f3de-7e58-4130-b7cc-587570a9043e response_code = 204 uri = /crucible/0/regions/4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
622024-08-28T02:18:06.027ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f (instance states: (Some(Online), None))
632024-08-28T02:18:06.030ZINFOcrucible-agent (dropshot): request completed latency_us = 572 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:52341 req_id = c8ae61ca-3d97-4d55-9c5c-441b3f5a1df0 response_code = 200 uri = /crucible/0/regions/4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
642024-08-28T02:18:06.031ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/971d742a-9853-4091-b634-ef2d86183201
652024-08-28T02:18:06.031ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
662024-08-28T02:18:06.031ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::13
672024-08-28T02:18:06.031ZINFOcrucible-agent (worker): reconfig required
682024-08-28T02:18:06.031ZINFOcrucible-agent (worker): using existing config property group
692024-08-28T02:18:06.031ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-971d742a-9853-4091-b634-ef2d86183201
702024-08-28T02:18:06.031ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/971d742a-9853-4091-b634-ef2d86183201
712024-08-28T02:18:06.032ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
722024-08-28T02:18:06.032ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
732024-08-28T02:18:06.032ZINFOcrucible-agent (worker): commit
742024-08-28T02:18:06.035ZINFOcrucible-agent (worker): ok!
752024-08-28T02:18:06.036ZINFOcrucible-agent (worker): SMF ok!
762024-08-28T02:18:06.045ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_79e9b421-598b-4117-bcfb-c4abc62898c1/crucible/regions/4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f" region = 4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f
772024-08-28T02:18:06.096ZINFOcrucible-agent (datafile): region 4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f state: Tombstoned -> Destroyed
782024-08-28T02:18:06.180ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:52341 req_id = ea6c35ae-6546-46a1-a59e-b2f4c485bc68 response_code = 200 uri = /crucible/0/regions/4e120bd6-a413-4a71-8ea9-3e3b57dfdb7f