Max level shown:
12025-02-06T05:22:21.089ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:35660
22025-02-06T05:22:21.129ZINFOcrucible-agent (datafile): region 736c9e29-d627-4242-8d47-2771d51ae5e4 state: Requested
32025-02-06T05:22:21.132ZINFOcrucible-agent (dropshot): request completed latency_us = 15625 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:35660 req_id = eb7852af-ecd8-4593-ae2d-099a3ee9e040 response_code = 200 uri = /crucible/0/regions
42025-02-06T05:22:21.138ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52025-02-06T05:22:21.152ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_2efdc4c8-a829-4558-a4a5-339458899198/crucible/regions/736c9e29-d627-4242-8d47-2771d51ae5e4
62025-02-06T05:22:21.152ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_2efdc4c8-a829-4558-a4a5-339458899198/crucible/regions/736c9e29-d627-4242-8d47-2771d51ae5e4
72025-02-06T05:22:21.209ZINFOcrucible-agent (worker): creating region Region { id: RegionId("736c9e29-d627-4242-8d47-2771d51ae5e4"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/736c9e29-d627-4242-8d47-2771d51ae5e4" region = 736c9e29-d627-4242-8d47-2771d51ae5e4
82025-02-06T05:22:21.211ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "736c9e29-d627-4242-8d47-2771d51ae5e4" "--data" "/data/regions/736c9e29-d627-4242-8d47-2771d51ae5e4" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 736c9e29-d627-4242-8d47-2771d51ae5e4
92025-02-06T05:22:21.315ZINFOcrucible-agent (dropshot): request completed latency_us = 86 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:35660 req_id = 727c3b2a-6668-4376-9f31-78128274bbb6 response_code = 200 uri = /crucible/0/regions
102025-02-06T05:22:21.573ZINFOcrucible-agent (worker): region files created ok region = 736c9e29-d627-4242-8d47-2771d51ae5e4
112025-02-06T05:22:21.573ZINFOcrucible-agent (datafile): region 736c9e29-d627-4242-8d47-2771d51ae5e4 state: Requested -> Created
122025-02-06T05:22:21.573ZINFOcrucible-agent (worker): applying SMF actions post create...
132025-02-06T05:22:21.574ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-736c9e29-d627-4242-8d47-2771d51ae5e4
142025-02-06T05:22:21.582ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-736c9e29-d627-4242-8d47-2771d51ae5e4
152025-02-06T05:22:21.599ZINFOcrucible-agent (dropshot): request completed latency_us = 44 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:35660 req_id = d3ad812a-d9b0-4614-ad22-29db016dc560 response_code = 200 uri = /crucible/0/regions
162025-02-06T05:22:21.601ZINFOcrucible-agent (worker): creating config property group
172025-02-06T05:22:21.608ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-736c9e29-d627-4242-8d47-2771d51ae5e4
182025-02-06T05:22:21.608ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/736c9e29-d627-4242-8d47-2771d51ae5e4
192025-02-06T05:22:21.612ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
202025-02-06T05:22:21.612ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
212025-02-06T05:22:21.612ZINFOcrucible-agent (worker): commit
222025-02-06T05:22:21.626ZINFOcrucible-agent (worker): ok!
232025-02-06T05:22:21.632ZINFOcrucible-agent (worker): SMF ok!
242025-02-06T05:22:26.143ZINFOcrucible-agent (datafile): region ca7886b2-f93d-441f-9bdd-e6a836cecd62 state: Requested
252025-02-06T05:22:26.145ZINFOcrucible-agent (dropshot): request completed latency_us = 2356 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:35660 req_id = 6560ea43-4d65-42e7-ac1e-433f469dc73f response_code = 200 uri = /crucible/0/regions
262025-02-06T05:22:26.145ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
272025-02-06T05:22:26.154ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_2efdc4c8-a829-4558-a4a5-339458899198/crucible/regions/ca7886b2-f93d-441f-9bdd-e6a836cecd62
282025-02-06T05:22:26.154ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_2efdc4c8-a829-4558-a4a5-339458899198/crucible/regions/ca7886b2-f93d-441f-9bdd-e6a836cecd62
292025-02-06T05:22:26.206ZINFOcrucible-agent (worker): creating region Region { id: RegionId("ca7886b2-f93d-441f-9bdd-e6a836cecd62"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/ca7886b2-f93d-441f-9bdd-e6a836cecd62" region = ca7886b2-f93d-441f-9bdd-e6a836cecd62
302025-02-06T05:22:26.206ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "ca7886b2-f93d-441f-9bdd-e6a836cecd62" "--data" "/data/regions/ca7886b2-f93d-441f-9bdd-e6a836cecd62" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = ca7886b2-f93d-441f-9bdd-e6a836cecd62
312025-02-06T05:22:26.275ZINFOcrucible-agent (worker): region files created ok region = ca7886b2-f93d-441f-9bdd-e6a836cecd62
322025-02-06T05:22:26.275ZINFOcrucible-agent (datafile): region ca7886b2-f93d-441f-9bdd-e6a836cecd62 state: Requested -> Created
332025-02-06T05:22:26.275ZINFOcrucible-agent (worker): applying SMF actions post create...
342025-02-06T05:22:26.276ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/736c9e29-d627-4242-8d47-2771d51ae5e4
352025-02-06T05:22:26.276ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
362025-02-06T05:22:26.276ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::17
372025-02-06T05:22:26.276ZINFOcrucible-agent (worker): reconfig required
382025-02-06T05:22:26.276ZINFOcrucible-agent (worker): using existing config property group
392025-02-06T05:22:26.276ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-736c9e29-d627-4242-8d47-2771d51ae5e4
402025-02-06T05:22:26.276ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/736c9e29-d627-4242-8d47-2771d51ae5e4
412025-02-06T05:22:26.276ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
422025-02-06T05:22:26.276ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432025-02-06T05:22:26.276ZINFOcrucible-agent (worker): commit
442025-02-06T05:22:26.280ZINFOcrucible-agent (worker): ok!
452025-02-06T05:22:26.280ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-ca7886b2-f93d-441f-9bdd-e6a836cecd62
462025-02-06T05:22:26.283ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-ca7886b2-f93d-441f-9bdd-e6a836cecd62
472025-02-06T05:22:26.290ZINFOcrucible-agent (worker): creating config property group
482025-02-06T05:22:26.293ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-ca7886b2-f93d-441f-9bdd-e6a836cecd62
492025-02-06T05:22:26.293ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/ca7886b2-f93d-441f-9bdd-e6a836cecd62
502025-02-06T05:22:26.293ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
512025-02-06T05:22:26.294ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
522025-02-06T05:22:26.294ZINFOcrucible-agent (worker): commit
532025-02-06T05:22:26.297ZINFOcrucible-agent (worker): ok!
542025-02-06T05:22:26.301ZINFOcrucible-agent (worker): SMF ok!
552025-02-06T05:22:26.479ZINFOcrucible-agent (dropshot): request completed latency_us = 611 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:35660 req_id = be02e423-8a35-433c-9ae7-6a9db73edbfc response_code = 200 uri = /crucible/0/regions
562025-02-06T05:22:50.994ZINFOcrucible-agent (dropshot): request completed latency_us = 77 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35660 req_id = 43f3b34b-0bad-4cdb-bae7-01dbd7f965ff response_code = 200 uri = /crucible/0/regions/ca7886b2-f93d-441f-9bdd-e6a836cecd62
572025-02-06T05:22:50.997ZINFOcrucible-agent (datafile): path is "/data/regions/ca7886b2-f93d-441f-9bdd-e6a836cecd62"
582025-02-06T05:22:51.008ZINFOcrucible-agent (datafile): dataset is oxp_2efdc4c8-a829-4558-a4a5-339458899198/crucible/regions/ca7886b2-f93d-441f-9bdd-e6a836cecd62
592025-02-06T05:22:51.017ZINFOcrucible-agent (datafile): region ca7886b2-f93d-441f-9bdd-e6a836cecd62 state: Created -> Tombstoned
602025-02-06T05:22:51.018ZINFOcrucible-agent (worker): applying SMF actions before removal...
612025-02-06T05:22:51.019ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-ca7886b2-f93d-441f-9bdd-e6a836cecd62 (instance states: (Some(Online), None))
622025-02-06T05:22:51.020ZINFOcrucible-agent (dropshot): request completed latency_us = 23087 local_addr = [fd00:1122:3344:101::17]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:35660 req_id = f9f69fa0-899e-422e-9654-0bc56a1901b0 response_code = 204 uri = /crucible/0/regions/ca7886b2-f93d-441f-9bdd-e6a836cecd62
632025-02-06T05:22:51.023ZINFOcrucible-agent (dropshot): request completed latency_us = 39 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35660 req_id = d079624c-2b62-4f2e-9199-207808ce522a response_code = 200 uri = /crucible/0/regions/ca7886b2-f93d-441f-9bdd-e6a836cecd62
642025-02-06T05:22:51.023ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/736c9e29-d627-4242-8d47-2771d51ae5e4
652025-02-06T05:22:51.024ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
662025-02-06T05:22:51.024ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::17
672025-02-06T05:22:51.024ZINFOcrucible-agent (worker): reconfig required
682025-02-06T05:22:51.024ZINFOcrucible-agent (worker): using existing config property group
692025-02-06T05:22:51.024ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-736c9e29-d627-4242-8d47-2771d51ae5e4
702025-02-06T05:22:51.024ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/736c9e29-d627-4242-8d47-2771d51ae5e4
712025-02-06T05:22:51.024ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
722025-02-06T05:22:51.024ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
732025-02-06T05:22:51.024ZINFOcrucible-agent (worker): commit
742025-02-06T05:22:51.029ZINFOcrucible-agent (worker): ok!
752025-02-06T05:22:51.029ZINFOcrucible-agent (worker): SMF ok!
762025-02-06T05:22:51.040ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_2efdc4c8-a829-4558-a4a5-339458899198/crucible/regions/ca7886b2-f93d-441f-9bdd-e6a836cecd62" region = ca7886b2-f93d-441f-9bdd-e6a836cecd62
772025-02-06T05:22:51.096ZINFOcrucible-agent (datafile): region ca7886b2-f93d-441f-9bdd-e6a836cecd62 state: Tombstoned -> Destroyed
782025-02-06T05:22:51.174ZINFOcrucible-agent (dropshot): request completed latency_us = 71 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35660 req_id = 39a35c30-378c-495b-a15d-d9fdd1c38aaf response_code = 200 uri = /crucible/0/regions/ca7886b2-f93d-441f-9bdd-e6a836cecd62