Max level shown:
12024-03-11T14:45:44.191ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:51096
22024-03-11T14:45:44.223ZINFOcrucible-agent (datafile): region a0aa2b4f-d467-4c7d-8580-235e591932c5 state: Requested
32024-03-11T14:45:44.225ZINFOcrucible-agent (dropshot): request completed latency_us = 19034 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51096 req_id = 8c044664-eecb-4f1e-b0dd-f4eaa4bfe91c response_code = 200 uri = /crucible/0/regions
42024-03-11T14:45:44.227ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52024-03-11T14:45:44.239ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible/regions/a0aa2b4f-d467-4c7d-8580-235e591932c5
62024-03-11T14:45:44.239ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible/regions/a0aa2b4f-d467-4c7d-8580-235e591932c5
72024-03-11T14:45:44.293ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a0aa2b4f-d467-4c7d-8580-235e591932c5"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None } at "/data/regions/a0aa2b4f-d467-4c7d-8580-235e591932c5" region = a0aa2b4f-d467-4c7d-8580-235e591932c5
82024-03-11T14:45:44.507ZINFOcrucible-agent (dropshot): request completed latency_us = 604 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51096 req_id = be886f1b-4c9f-4d7b-a067-a4b893b797ff response_code = 200 uri = /crucible/0/regions
92024-03-11T14:45:44.569ZINFOcrucible-agent (worker): region files created ok region = a0aa2b4f-d467-4c7d-8580-235e591932c5
102024-03-11T14:45:44.570ZINFOcrucible-agent (datafile): region a0aa2b4f-d467-4c7d-8580-235e591932c5 state: Requested -> Created
112024-03-11T14:45:44.570ZINFOcrucible-agent (worker): applying SMF actions post create...
122024-03-11T14:45:44.570ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a0aa2b4f-d467-4c7d-8580-235e591932c5
132024-03-11T14:45:44.575ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a0aa2b4f-d467-4c7d-8580-235e591932c5
142024-03-11T14:45:44.590ZINFOcrucible-agent (worker): creating config property group
152024-03-11T14:45:44.593ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a0aa2b4f-d467-4c7d-8580-235e591932c5
162024-03-11T14:45:44.594ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a0aa2b4f-d467-4c7d-8580-235e591932c5
172024-03-11T14:45:44.596ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
182024-03-11T14:45:44.596ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
192024-03-11T14:45:44.596ZINFOcrucible-agent (worker): commit
202024-03-11T14:45:44.608ZINFOcrucible-agent (worker): ok!
212024-03-11T14:45:44.617ZINFOcrucible-agent (worker): SMF ok!
222024-03-11T14:45:45.227ZINFOcrucible-agent (dropshot): request completed latency_us = 89 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51096 req_id = 73e5035e-709a-449a-88a1-59a54533462c response_code = 200 uri = /crucible/0/regions
232024-03-11T14:45:48.692ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:33550
242024-03-11T14:45:48.692ZINFOcrucible-agent (datafile): region 42de9aaa-82ce-4818-9e2a-f2be5e023bc9 state: Requested
252024-03-11T14:45:48.692ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
262024-03-11T14:45:48.692ZINFOcrucible-agent (dropshot): request completed latency_us = 281 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33550 req_id = ca5cc70e-263a-4af7-b47c-2c65e3868836 response_code = 200 uri = /crucible/0/regions
272024-03-11T14:45:48.702ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible/regions/42de9aaa-82ce-4818-9e2a-f2be5e023bc9
282024-03-11T14:45:48.702ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible/regions/42de9aaa-82ce-4818-9e2a-f2be5e023bc9
292024-03-11T14:45:48.775ZINFOcrucible-agent (worker): creating region Region { id: RegionId("42de9aaa-82ce-4818-9e2a-f2be5e023bc9"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None } at "/data/regions/42de9aaa-82ce-4818-9e2a-f2be5e023bc9" region = 42de9aaa-82ce-4818-9e2a-f2be5e023bc9
302024-03-11T14:45:48.843ZINFOcrucible-agent (worker): region files created ok region = 42de9aaa-82ce-4818-9e2a-f2be5e023bc9
312024-03-11T14:45:48.843ZINFOcrucible-agent (datafile): region 42de9aaa-82ce-4818-9e2a-f2be5e023bc9 state: Requested -> Created
322024-03-11T14:45:48.843ZINFOcrucible-agent (worker): applying SMF actions post create...
332024-03-11T14:45:48.844ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-42de9aaa-82ce-4818-9e2a-f2be5e023bc9
342024-03-11T14:45:48.848ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-42de9aaa-82ce-4818-9e2a-f2be5e023bc9
352024-03-11T14:45:48.855ZINFOcrucible-agent (worker): creating config property group
362024-03-11T14:45:48.859ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-42de9aaa-82ce-4818-9e2a-f2be5e023bc9
372024-03-11T14:45:48.859ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/42de9aaa-82ce-4818-9e2a-f2be5e023bc9
382024-03-11T14:45:48.859ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
392024-03-11T14:45:48.859ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
402024-03-11T14:45:48.859ZINFOcrucible-agent (worker): commit
412024-03-11T14:45:48.863ZINFOcrucible-agent (worker): ok!
422024-03-11T14:45:48.870ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/a0aa2b4f-d467-4c7d-8580-235e591932c5
432024-03-11T14:45:48.870ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
442024-03-11T14:45:48.870ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::15
452024-03-11T14:45:48.870ZINFOcrucible-agent (worker): reconfig required
462024-03-11T14:45:48.870ZINFOcrucible-agent (worker): using existing config property group
472024-03-11T14:45:48.870ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a0aa2b4f-d467-4c7d-8580-235e591932c5
482024-03-11T14:45:48.870ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a0aa2b4f-d467-4c7d-8580-235e591932c5
492024-03-11T14:45:48.870ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
502024-03-11T14:45:48.870ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
512024-03-11T14:45:48.870ZINFOcrucible-agent (worker): commit
522024-03-11T14:45:48.878ZINFOcrucible-agent (worker): ok!
532024-03-11T14:45:48.879ZINFOcrucible-agent (worker): SMF ok!
542024-03-11T14:45:48.986ZINFOcrucible-agent (dropshot): request completed latency_us = 755 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33550 req_id = b34eae03-5b74-4463-bb8a-85b3f812c83b response_code = 200 uri = /crucible/0/regions
552024-03-11T14:46:21.247ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:35736
562024-03-11T14:46:21.247ZINFOcrucible-agent (dropshot): request completed latency_us = 78 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35736 req_id = 926fd1de-a7b1-4b57-a1c6-363110543a97 response_code = 200 uri = /crucible/0/regions/42de9aaa-82ce-4818-9e2a-f2be5e023bc9
572024-03-11T14:46:21.247ZINFOcrucible-agent (datafile): path is "/data/regions/42de9aaa-82ce-4818-9e2a-f2be5e023bc9"
582024-03-11T14:46:21.259ZINFOcrucible-agent (datafile): dataset is oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible/regions/42de9aaa-82ce-4818-9e2a-f2be5e023bc9
592024-03-11T14:46:21.267ZINFOcrucible-agent (datafile): region 42de9aaa-82ce-4818-9e2a-f2be5e023bc9 state: Created -> Tombstoned
602024-03-11T14:46:21.267ZINFOcrucible-agent (worker): applying SMF actions before removal...
612024-03-11T14:46:21.267ZINFOcrucible-agent (dropshot): request completed latency_us = 19458 local_addr = [fd00:1122:3344:101::15]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:35736 req_id = eab9f2ee-a287-49f6-8100-2d4166d8efad response_code = 204 uri = /crucible/0/regions/42de9aaa-82ce-4818-9e2a-f2be5e023bc9
622024-03-11T14:46:21.268ZINFOcrucible-agent (dropshot): request completed latency_us = 590 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35736 req_id = aba578a5-b939-43e8-a6c6-2fc55046aeae response_code = 200 uri = /crucible/0/regions/42de9aaa-82ce-4818-9e2a-f2be5e023bc9
632024-03-11T14:46:21.268ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-42de9aaa-82ce-4818-9e2a-f2be5e023bc9 (instance states: (Some(Online), None))
642024-03-11T14:46:21.273ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/a0aa2b4f-d467-4c7d-8580-235e591932c5
652024-03-11T14:46:21.273ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
662024-03-11T14:46:21.273ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::15
672024-03-11T14:46:21.273ZINFOcrucible-agent (worker): reconfig required
682024-03-11T14:46:21.273ZINFOcrucible-agent (worker): using existing config property group
692024-03-11T14:46:21.273ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a0aa2b4f-d467-4c7d-8580-235e591932c5
702024-03-11T14:46:21.273ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a0aa2b4f-d467-4c7d-8580-235e591932c5
712024-03-11T14:46:21.273ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
722024-03-11T14:46:21.273ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
732024-03-11T14:46:21.273ZINFOcrucible-agent (worker): commit
742024-03-11T14:46:21.277ZINFOcrucible-agent (worker): ok!
752024-03-11T14:46:21.277ZINFOcrucible-agent (worker): SMF ok!
762024-03-11T14:46:21.287ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_616b26df-e62a-4c68-b506-f4a923d8aaf7/crucible/regions/42de9aaa-82ce-4818-9e2a-f2be5e023bc9" region = 42de9aaa-82ce-4818-9e2a-f2be5e023bc9
772024-03-11T14:46:21.327ZINFOcrucible-agent (dropshot): request completed latency_us = 89 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35736 req_id = d3fefbeb-3e6e-4cac-a4af-b85338a9a32a response_code = 200 uri = /crucible/0/regions/42de9aaa-82ce-4818-9e2a-f2be5e023bc9
782024-03-11T14:46:21.336ZINFOcrucible-agent (datafile): region 42de9aaa-82ce-4818-9e2a-f2be5e023bc9 state: Tombstoned -> Destroyed
792024-03-11T14:46:21.405ZINFOcrucible-agent (dropshot): request completed latency_us = 32 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35736 req_id = 74d91dcc-387b-4df6-8b38-5d6fb99aa7bb response_code = 200 uri = /crucible/0/regions/42de9aaa-82ce-4818-9e2a-f2be5e023bc9