Max level shown:
12024-08-27T23:51:23.916ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:49969
22024-08-27T23:51:23.943ZINFOcrucible-agent (datafile): region df5ef26d-e779-411b-a5c9-6f4e95091305 state: Requested
32024-08-27T23:51:23.950ZINFOcrucible-agent (dropshot): request completed latency_us = 16211 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49969 req_id = 282febba-186b-4b3e-ae12-ea7872fe7fdc response_code = 200 uri = /crucible/0/regions
42024-08-27T23:51:23.953ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52024-08-27T23:51:23.963ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_261199e3-73de-407f-8de5-043cbf4ed10d/crucible/regions/df5ef26d-e779-411b-a5c9-6f4e95091305
62024-08-27T23:51:23.964ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_261199e3-73de-407f-8de5-043cbf4ed10d/crucible/regions/df5ef26d-e779-411b-a5c9-6f4e95091305
72024-08-27T23:51:24.029ZINFOcrucible-agent (worker): creating region Region { id: RegionId("df5ef26d-e779-411b-a5c9-6f4e95091305"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/df5ef26d-e779-411b-a5c9-6f4e95091305" region = df5ef26d-e779-411b-a5c9-6f4e95091305
82024-08-27T23:51:24.032ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "df5ef26d-e779-411b-a5c9-6f4e95091305" "--data" "/data/regions/df5ef26d-e779-411b-a5c9-6f4e95091305" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = df5ef26d-e779-411b-a5c9-6f4e95091305
92024-08-27T23:51:24.149ZINFOcrucible-agent (dropshot): request completed latency_us = 81 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49969 req_id = 716956db-5b3f-4cb0-9f17-8deedb869a2c response_code = 200 uri = /crucible/0/regions
102024-08-27T23:51:24.334ZINFOcrucible-agent (worker): region files created ok region = df5ef26d-e779-411b-a5c9-6f4e95091305
112024-08-27T23:51:24.334ZINFOcrucible-agent (datafile): region df5ef26d-e779-411b-a5c9-6f4e95091305 state: Requested -> Created
122024-08-27T23:51:24.334ZINFOcrucible-agent (worker): applying SMF actions post create...
132024-08-27T23:51:24.335ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-df5ef26d-e779-411b-a5c9-6f4e95091305
142024-08-27T23:51:24.340ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-df5ef26d-e779-411b-a5c9-6f4e95091305
152024-08-27T23:51:24.361ZINFOcrucible-agent (worker): creating config property group
162024-08-27T23:51:24.365ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-df5ef26d-e779-411b-a5c9-6f4e95091305
172024-08-27T23:51:24.373ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/df5ef26d-e779-411b-a5c9-6f4e95091305
182024-08-27T23:51:24.376ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192024-08-27T23:51:24.376ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
202024-08-27T23:51:24.376ZINFOcrucible-agent (worker): commit
212024-08-27T23:51:24.380ZINFOcrucible-agent (worker): ok!
222024-08-27T23:51:24.384ZINFOcrucible-agent (worker): SMF ok!
232024-08-27T23:51:24.584ZINFOcrucible-agent (dropshot): request completed latency_us = 641 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49969 req_id = cdba60ae-81fd-4bb1-8f2d-b6f3b45cefb7 response_code = 200 uri = /crucible/0/regions
242024-08-27T23:51:29.021ZINFOcrucible-agent (datafile): region 3a5d56d7-ace2-45e6-a216-097865dcc196 state: Requested
252024-08-27T23:51:29.022ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
262024-08-27T23:51:29.022ZINFOcrucible-agent (dropshot): request completed latency_us = 209 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49969 req_id = d7880d90-627b-44e2-9b6c-a79b8d875d30 response_code = 200 uri = /crucible/0/regions
272024-08-27T23:51:29.030ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_261199e3-73de-407f-8de5-043cbf4ed10d/crucible/regions/3a5d56d7-ace2-45e6-a216-097865dcc196
282024-08-27T23:51:29.030ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_261199e3-73de-407f-8de5-043cbf4ed10d/crucible/regions/3a5d56d7-ace2-45e6-a216-097865dcc196
292024-08-27T23:51:29.097ZINFOcrucible-agent (worker): creating region Region { id: RegionId("3a5d56d7-ace2-45e6-a216-097865dcc196"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/3a5d56d7-ace2-45e6-a216-097865dcc196" region = 3a5d56d7-ace2-45e6-a216-097865dcc196
302024-08-27T23:51:29.097ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "3a5d56d7-ace2-45e6-a216-097865dcc196" "--data" "/data/regions/3a5d56d7-ace2-45e6-a216-097865dcc196" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 3a5d56d7-ace2-45e6-a216-097865dcc196
312024-08-27T23:51:29.178ZINFOcrucible-agent (worker): region files created ok region = 3a5d56d7-ace2-45e6-a216-097865dcc196
322024-08-27T23:51:29.178ZINFOcrucible-agent (datafile): region 3a5d56d7-ace2-45e6-a216-097865dcc196 state: Requested -> Created
332024-08-27T23:51:29.178ZINFOcrucible-agent (worker): applying SMF actions post create...
342024-08-27T23:51:29.179ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-3a5d56d7-ace2-45e6-a216-097865dcc196
352024-08-27T23:51:29.182ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-3a5d56d7-ace2-45e6-a216-097865dcc196
362024-08-27T23:51:29.189ZINFOcrucible-agent (worker): creating config property group
372024-08-27T23:51:29.194ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-3a5d56d7-ace2-45e6-a216-097865dcc196
382024-08-27T23:51:29.194ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3a5d56d7-ace2-45e6-a216-097865dcc196
392024-08-27T23:51:29.194ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
402024-08-27T23:51:29.194ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
412024-08-27T23:51:29.194ZINFOcrucible-agent (worker): commit
422024-08-27T23:51:29.199ZINFOcrucible-agent (worker): ok!
432024-08-27T23:51:29.204ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/df5ef26d-e779-411b-a5c9-6f4e95091305
442024-08-27T23:51:29.204ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
452024-08-27T23:51:29.204ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::12
462024-08-27T23:51:29.204ZINFOcrucible-agent (worker): reconfig required
472024-08-27T23:51:29.204ZINFOcrucible-agent (worker): using existing config property group
482024-08-27T23:51:29.204ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-df5ef26d-e779-411b-a5c9-6f4e95091305
492024-08-27T23:51:29.204ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/df5ef26d-e779-411b-a5c9-6f4e95091305
502024-08-27T23:51:29.204ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
512024-08-27T23:51:29.204ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
522024-08-27T23:51:29.204ZINFOcrucible-agent (worker): commit
532024-08-27T23:51:29.207ZINFOcrucible-agent (dropshot): request completed latency_us = 85 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49969 req_id = 7f58573c-322e-4704-9a7f-6838eeb43465 response_code = 200 uri = /crucible/0/regions
542024-08-27T23:51:29.214ZINFOcrucible-agent (worker): ok!
552024-08-27T23:51:29.214ZINFOcrucible-agent (worker): SMF ok!
562024-08-27T23:51:56.906ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:49969 req_id = 7abdf04f-43a6-45ff-be76-cb1a00ef5b9f response_code = 200 uri = /crucible/0/regions/3a5d56d7-ace2-45e6-a216-097865dcc196
572024-08-27T23:51:56.909ZINFOcrucible-agent (datafile): path is "/data/regions/3a5d56d7-ace2-45e6-a216-097865dcc196"
582024-08-27T23:51:56.919ZINFOcrucible-agent (datafile): dataset is oxp_261199e3-73de-407f-8de5-043cbf4ed10d/crucible/regions/3a5d56d7-ace2-45e6-a216-097865dcc196
592024-08-27T23:51:56.927ZINFOcrucible-agent (datafile): region 3a5d56d7-ace2-45e6-a216-097865dcc196 state: Created -> Tombstoned
602024-08-27T23:51:56.927ZINFOcrucible-agent (worker): applying SMF actions before removal...
612024-08-27T23:51:56.927ZINFOcrucible-agent (dropshot): request completed latency_us = 18671 local_addr = [fd00:1122:3344:101::12]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:49969 req_id = 14572b87-4798-4ad1-a563-8489328b45b6 response_code = 204 uri = /crucible/0/regions/3a5d56d7-ace2-45e6-a216-097865dcc196
622024-08-27T23:51:56.928ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-3a5d56d7-ace2-45e6-a216-097865dcc196 (instance states: (Some(Online), None))
632024-08-27T23:51:56.931ZINFOcrucible-agent (dropshot): request completed latency_us = 721 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:49969 req_id = 81f2f880-0b62-4a62-b5b1-ad8f5d83f344 response_code = 200 uri = /crucible/0/regions/3a5d56d7-ace2-45e6-a216-097865dcc196
642024-08-27T23:51:56.965ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/df5ef26d-e779-411b-a5c9-6f4e95091305
652024-08-27T23:51:56.965ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
662024-08-27T23:51:56.965ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::12
672024-08-27T23:51:56.965ZINFOcrucible-agent (worker): reconfig required
682024-08-27T23:51:56.965ZINFOcrucible-agent (worker): using existing config property group
692024-08-27T23:51:56.965ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-df5ef26d-e779-411b-a5c9-6f4e95091305
702024-08-27T23:51:56.965ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/df5ef26d-e779-411b-a5c9-6f4e95091305
712024-08-27T23:51:56.965ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
722024-08-27T23:51:56.966ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
732024-08-27T23:51:56.966ZINFOcrucible-agent (worker): commit
742024-08-27T23:51:57.019ZINFOcrucible-agent (dropshot): request completed latency_us = 58 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:49969 req_id = 969bc214-ecd1-4e29-b1e6-4c869f595172 response_code = 200 uri = /crucible/0/regions/3a5d56d7-ace2-45e6-a216-097865dcc196
752024-08-27T23:51:57.034ZINFOcrucible-agent (worker): ok!
762024-08-27T23:51:57.035ZINFOcrucible-agent (worker): SMF ok!
772024-08-27T23:51:57.045ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_261199e3-73de-407f-8de5-043cbf4ed10d/crucible/regions/3a5d56d7-ace2-45e6-a216-097865dcc196" region = 3a5d56d7-ace2-45e6-a216-097865dcc196
782024-08-27T23:51:57.164ZINFOcrucible-agent (datafile): region 3a5d56d7-ace2-45e6-a216-097865dcc196 state: Tombstoned -> Destroyed
792024-08-27T23:51:57.173ZINFOcrucible-agent (dropshot): request completed latency_us = 661 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:49969 req_id = 3a1b4be4-8ba2-4c32-9118-721e740ae3fb response_code = 200 uri = /crucible/0/regions/3a5d56d7-ace2-45e6-a216-097865dcc196