Max level shown:
12023-12-11T21:18:19.167ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:64488
22023-12-11T21:18:19.214ZINFOcrucible-agent (datafile): region a4213a54-d0fe-4881-ab5c-a7fb7a607c13 state: Requested
32023-12-11T21:18:19.217ZINFOcrucible-agent (dropshot): request completed latency_us = 17489 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:64488 req_id = b5b05ba7-852d-4cec-84db-a7cff2430308 response_code = 200 uri = /crucible/0/regions
42023-12-11T21:18:19.217ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52023-12-11T21:18:19.230ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/a4213a54-d0fe-4881-ab5c-a7fb7a607c13
62023-12-11T21:18:19.230ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/a4213a54-d0fe-4881-ab5c-a7fb7a607c13
72023-12-11T21:18:19.284ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a4213a54-d0fe-4881-ab5c-a7fb7a607c13"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None } at "/data/regions/a4213a54-d0fe-4881-ab5c-a7fb7a607c13" region = a4213a54-d0fe-4881-ab5c-a7fb7a607c13
82023-12-11T21:18:19.463ZINFOcrucible-agent (dropshot): request completed latency_us = 83 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:64488 req_id = 7d8f6935-756d-4546-a19b-03d4f8d76d5b response_code = 200 uri = /crucible/0/regions
92023-12-11T21:18:19.556ZINFOcrucible-agent (worker): region files created ok region = a4213a54-d0fe-4881-ab5c-a7fb7a607c13
102023-12-11T21:18:19.556ZINFOcrucible-agent (datafile): region a4213a54-d0fe-4881-ab5c-a7fb7a607c13 state: Requested -> Created
112023-12-11T21:18:19.557ZINFOcrucible-agent (worker): applying SMF actions post create...
122023-12-11T21:18:19.559ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a4213a54-d0fe-4881-ab5c-a7fb7a607c13
132023-12-11T21:18:19.565ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a4213a54-d0fe-4881-ab5c-a7fb7a607c13
142023-12-11T21:18:19.588ZINFOcrucible-agent (worker): creating config property group
152023-12-11T21:18:19.591ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a4213a54-d0fe-4881-ab5c-a7fb7a607c13
162023-12-11T21:18:19.599ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a4213a54-d0fe-4881-ab5c-a7fb7a607c13
172023-12-11T21:18:19.601ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
182023-12-11T21:18:19.601ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
192023-12-11T21:18:19.601ZINFOcrucible-agent (worker): commit
202023-12-11T21:18:19.605ZINFOcrucible-agent (worker): ok!
212023-12-11T21:18:19.609ZINFOcrucible-agent (worker): SMF ok!
222023-12-11T21:18:19.889ZINFOcrucible-agent (dropshot): request completed latency_us = 76 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:64488 req_id = 2552d369-6f84-415c-9bb6-cd510c394c32 response_code = 200 uri = /crucible/0/regions
232023-12-11T21:18:23.162ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:55146
242023-12-11T21:18:23.193ZINFOcrucible-agent (datafile): region 2f9e2ece-358d-428e-9249-d6b367013227 state: Requested
252023-12-11T21:18:23.193ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
262023-12-11T21:18:23.193ZINFOcrucible-agent (dropshot): request completed latency_us = 216 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55146 req_id = bebd3b29-41e4-4a2a-8668-33ec19d0695c response_code = 200 uri = /crucible/0/regions
272023-12-11T21:18:23.204ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/2f9e2ece-358d-428e-9249-d6b367013227
282023-12-11T21:18:23.204ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/2f9e2ece-358d-428e-9249-d6b367013227
292023-12-11T21:18:23.277ZINFOcrucible-agent (worker): creating region Region { id: RegionId("2f9e2ece-358d-428e-9249-d6b367013227"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None } at "/data/regions/2f9e2ece-358d-428e-9249-d6b367013227" region = 2f9e2ece-358d-428e-9249-d6b367013227
302023-12-11T21:18:23.344ZINFOcrucible-agent (worker): region files created ok region = 2f9e2ece-358d-428e-9249-d6b367013227
312023-12-11T21:18:23.344ZINFOcrucible-agent (datafile): region 2f9e2ece-358d-428e-9249-d6b367013227 state: Requested -> Created
322023-12-11T21:18:23.344ZINFOcrucible-agent (worker): applying SMF actions post create...
332023-12-11T21:18:23.345ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-2f9e2ece-358d-428e-9249-d6b367013227
342023-12-11T21:18:23.348ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-2f9e2ece-358d-428e-9249-d6b367013227
352023-12-11T21:18:23.355ZINFOcrucible-agent (worker): creating config property group
362023-12-11T21:18:23.358ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-2f9e2ece-358d-428e-9249-d6b367013227
372023-12-11T21:18:23.358ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/2f9e2ece-358d-428e-9249-d6b367013227
382023-12-11T21:18:23.358ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
392023-12-11T21:18:23.358ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
402023-12-11T21:18:23.358ZINFOcrucible-agent (worker): commit
412023-12-11T21:18:23.362ZINFOcrucible-agent (worker): ok!
422023-12-11T21:18:23.372ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/a4213a54-d0fe-4881-ab5c-a7fb7a607c13
432023-12-11T21:18:23.372ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
442023-12-11T21:18:23.372ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::16
452023-12-11T21:18:23.372ZINFOcrucible-agent (worker): reconfig required
462023-12-11T21:18:23.372ZINFOcrucible-agent (worker): using existing config property group
472023-12-11T21:18:23.372ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a4213a54-d0fe-4881-ab5c-a7fb7a607c13
482023-12-11T21:18:23.372ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a4213a54-d0fe-4881-ab5c-a7fb7a607c13
492023-12-11T21:18:23.372ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
502023-12-11T21:18:23.372ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
512023-12-11T21:18:23.372ZINFOcrucible-agent (worker): commit
522023-12-11T21:18:23.377ZINFOcrucible-agent (worker): ok!
532023-12-11T21:18:23.377ZINFOcrucible-agent (worker): SMF ok!
542023-12-11T21:18:23.462ZINFOcrucible-agent (dropshot): request completed latency_us = 94 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55146 req_id = f874a059-c86a-4316-ac44-47c54e119e1a response_code = 200 uri = /crucible/0/regions
552023-12-11T21:18:54.224ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:46279
562023-12-11T21:18:54.229ZINFOcrucible-agent (dropshot): request completed latency_us = 5030 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:46279 req_id = b84981da-6677-4673-a3ef-b2d7c11f33bd response_code = 200 uri = /crucible/0/regions/2f9e2ece-358d-428e-9249-d6b367013227
572023-12-11T21:18:54.230ZINFOcrucible-agent (datafile): path is "/data/regions/2f9e2ece-358d-428e-9249-d6b367013227"
582023-12-11T21:18:54.241ZINFOcrucible-agent (datafile): dataset is oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/2f9e2ece-358d-428e-9249-d6b367013227
592023-12-11T21:18:54.250ZINFOcrucible-agent (datafile): region 2f9e2ece-358d-428e-9249-d6b367013227 state: Created -> Tombstoned
602023-12-11T21:18:54.250ZINFOcrucible-agent (worker): applying SMF actions before removal...
612023-12-11T21:18:54.250ZINFOcrucible-agent (dropshot): request completed latency_us = 20466 local_addr = [fd00:1122:3344:101::16]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:46279 req_id = 3b6d5d81-6d5a-4856-9e32-235179ad6f10 response_code = 204 uri = /crucible/0/regions/2f9e2ece-358d-428e-9249-d6b367013227
622023-12-11T21:18:54.250ZINFOcrucible-agent (dropshot): request completed latency_us = 27 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:46279 req_id = edb25606-5363-4d78-b849-969a5fa0f7cc response_code = 200 uri = /crucible/0/regions/2f9e2ece-358d-428e-9249-d6b367013227
632023-12-11T21:18:54.251ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-2f9e2ece-358d-428e-9249-d6b367013227 (instance states: (Some(Online), None))
642023-12-11T21:18:54.279ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/a4213a54-d0fe-4881-ab5c-a7fb7a607c13
652023-12-11T21:18:54.280ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
662023-12-11T21:18:54.280ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::16
672023-12-11T21:18:54.280ZINFOcrucible-agent (worker): reconfig required
682023-12-11T21:18:54.280ZINFOcrucible-agent (worker): using existing config property group
692023-12-11T21:18:54.280ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a4213a54-d0fe-4881-ab5c-a7fb7a607c13
702023-12-11T21:18:54.280ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a4213a54-d0fe-4881-ab5c-a7fb7a607c13
712023-12-11T21:18:54.280ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
722023-12-11T21:18:54.280ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
732023-12-11T21:18:54.280ZINFOcrucible-agent (worker): commit
742023-12-11T21:18:54.290ZINFOcrucible-agent (worker): ok!
752023-12-11T21:18:54.291ZINFOcrucible-agent (worker): SMF ok!
762023-12-11T21:18:54.302ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/2f9e2ece-358d-428e-9249-d6b367013227" region = 2f9e2ece-358d-428e-9249-d6b367013227
772023-12-11T21:18:54.365ZINFOcrucible-agent (datafile): region 2f9e2ece-358d-428e-9249-d6b367013227 state: Tombstoned -> Destroyed
782023-12-11T21:18:54.383ZINFOcrucible-agent (dropshot): request completed latency_us = 807 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:46279 req_id = 76b4a96e-f241-47be-875d-1dfc05f08f08 response_code = 200 uri = /crucible/0/regions/2f9e2ece-358d-428e-9249-d6b367013227