Max level shown:
12025-01-28T17:53:31.048ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::a]:59832
22025-01-28T17:53:31.092ZINFOcrucible-agent (datafile): region e7f38630-0864-40b7-af99-2d3436c900ee state: Requested
32025-01-28T17:53:31.099ZINFOcrucible-agent (dropshot): request completed latency_us = 21355 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:59832 req_id = 1edd81a3-5e25-4c7e-a6fb-92fe41c2a1ad response_code = 200 uri = /crucible/0/regions
42025-01-28T17:53:31.099ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52025-01-28T17:53:31.112ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_37b8192c-9236-4941-bad3-2ab9840de50c/crucible/regions/e7f38630-0864-40b7-af99-2d3436c900ee
62025-01-28T17:53:31.112ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_37b8192c-9236-4941-bad3-2ab9840de50c/crucible/regions/e7f38630-0864-40b7-af99-2d3436c900ee
72025-01-28T17:53:31.159ZINFOcrucible-agent (worker): creating region Region { id: RegionId("e7f38630-0864-40b7-af99-2d3436c900ee"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/e7f38630-0864-40b7-af99-2d3436c900ee" region = e7f38630-0864-40b7-af99-2d3436c900ee
82025-01-28T17:53:31.162ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "e7f38630-0864-40b7-af99-2d3436c900ee" "--data" "/data/regions/e7f38630-0864-40b7-af99-2d3436c900ee" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = e7f38630-0864-40b7-af99-2d3436c900ee
92025-01-28T17:53:31.435ZINFOcrucible-agent (dropshot): request completed latency_us = 715 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:59832 req_id = 68b5e61f-a864-4a1e-b566-f9bf2f52848a response_code = 200 uri = /crucible/0/regions
102025-01-28T17:53:31.464ZINFOcrucible-agent (worker): region files created ok region = e7f38630-0864-40b7-af99-2d3436c900ee
112025-01-28T17:53:31.464ZINFOcrucible-agent (datafile): region e7f38630-0864-40b7-af99-2d3436c900ee state: Requested -> Created
122025-01-28T17:53:31.464ZINFOcrucible-agent (worker): applying SMF actions post create...
132025-01-28T17:53:31.465ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-e7f38630-0864-40b7-af99-2d3436c900ee
142025-01-28T17:53:31.471ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-e7f38630-0864-40b7-af99-2d3436c900ee
152025-01-28T17:53:31.486ZINFOcrucible-agent (worker): creating config property group
162025-01-28T17:53:31.497ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-e7f38630-0864-40b7-af99-2d3436c900ee
172025-01-28T17:53:31.497ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/e7f38630-0864-40b7-af99-2d3436c900ee
182025-01-28T17:53:31.500ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192025-01-28T17:53:31.500ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
202025-01-28T17:53:31.500ZINFOcrucible-agent (worker): commit
212025-01-28T17:53:31.505ZINFOcrucible-agent (worker): ok!
222025-01-28T17:53:31.508ZINFOcrucible-agent (worker): SMF ok!
232025-01-28T17:53:31.837ZINFOcrucible-agent (dropshot): request completed latency_us = 67 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:59832 req_id = c50a4aa6-202a-492a-926d-0c48e25eae9a response_code = 200 uri = /crucible/0/regions
242025-01-28T17:53:36.738ZINFOcrucible-agent (datafile): region 02e7b20f-63f1-4dc8-823c-5df03598634f state: Requested
252025-01-28T17:53:36.738ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
262025-01-28T17:53:36.738ZINFOcrucible-agent (dropshot): request completed latency_us = 257 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:59832 req_id = 4996d86e-3e20-4b79-962d-1625269ec355 response_code = 200 uri = /crucible/0/regions
272025-01-28T17:53:36.748ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_37b8192c-9236-4941-bad3-2ab9840de50c/crucible/regions/02e7b20f-63f1-4dc8-823c-5df03598634f
282025-01-28T17:53:36.748ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_37b8192c-9236-4941-bad3-2ab9840de50c/crucible/regions/02e7b20f-63f1-4dc8-823c-5df03598634f
292025-01-28T17:53:36.800ZINFOcrucible-agent (worker): creating region Region { id: RegionId("02e7b20f-63f1-4dc8-823c-5df03598634f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/02e7b20f-63f1-4dc8-823c-5df03598634f" region = 02e7b20f-63f1-4dc8-823c-5df03598634f
302025-01-28T17:53:36.800ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "02e7b20f-63f1-4dc8-823c-5df03598634f" "--data" "/data/regions/02e7b20f-63f1-4dc8-823c-5df03598634f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 02e7b20f-63f1-4dc8-823c-5df03598634f
312025-01-28T17:53:36.885ZINFOcrucible-agent (worker): region files created ok region = 02e7b20f-63f1-4dc8-823c-5df03598634f
322025-01-28T17:53:36.885ZINFOcrucible-agent (datafile): region 02e7b20f-63f1-4dc8-823c-5df03598634f state: Requested -> Created
332025-01-28T17:53:36.885ZINFOcrucible-agent (worker): applying SMF actions post create...
342025-01-28T17:53:36.885ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-02e7b20f-63f1-4dc8-823c-5df03598634f
352025-01-28T17:53:36.889ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-02e7b20f-63f1-4dc8-823c-5df03598634f
362025-01-28T17:53:36.896ZINFOcrucible-agent (worker): creating config property group
372025-01-28T17:53:36.900ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-02e7b20f-63f1-4dc8-823c-5df03598634f
382025-01-28T17:53:36.900ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/02e7b20f-63f1-4dc8-823c-5df03598634f
392025-01-28T17:53:36.900ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
402025-01-28T17:53:36.900ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
412025-01-28T17:53:36.900ZINFOcrucible-agent (worker): commit
422025-01-28T17:53:36.904ZINFOcrucible-agent (worker): ok!
432025-01-28T17:53:36.909ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/e7f38630-0864-40b7-af99-2d3436c900ee
442025-01-28T17:53:36.909ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
452025-01-28T17:53:36.909ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::1a
462025-01-28T17:53:36.909ZINFOcrucible-agent (worker): reconfig required
472025-01-28T17:53:36.909ZINFOcrucible-agent (worker): using existing config property group
482025-01-28T17:53:36.909ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-e7f38630-0864-40b7-af99-2d3436c900ee
492025-01-28T17:53:36.909ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/e7f38630-0864-40b7-af99-2d3436c900ee
502025-01-28T17:53:36.909ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
512025-01-28T17:53:36.909ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
522025-01-28T17:53:36.909ZINFOcrucible-agent (worker): commit
532025-01-28T17:53:36.916ZINFOcrucible-agent (worker): ok!
542025-01-28T17:53:36.917ZINFOcrucible-agent (worker): SMF ok!
552025-01-28T17:53:37.034ZINFOcrucible-agent (dropshot): request completed latency_us = 86 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:59832 req_id = f4107a68-0714-4d9b-9758-1370827bffd1 response_code = 200 uri = /crucible/0/regions
562025-01-28T17:54:02.351ZINFOcrucible-agent (dropshot): request completed latency_us = 291 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:59832 req_id = 9e4432f0-83b8-485e-ad8a-9d463d0665dc response_code = 200 uri = /crucible/0/regions/02e7b20f-63f1-4dc8-823c-5df03598634f
572025-01-28T17:54:02.354ZINFOcrucible-agent (datafile): path is "/data/regions/02e7b20f-63f1-4dc8-823c-5df03598634f"
582025-01-28T17:54:02.365ZINFOcrucible-agent (datafile): dataset is oxp_37b8192c-9236-4941-bad3-2ab9840de50c/crucible/regions/02e7b20f-63f1-4dc8-823c-5df03598634f
592025-01-28T17:54:02.376ZINFOcrucible-agent (datafile): region 02e7b20f-63f1-4dc8-823c-5df03598634f state: Created -> Tombstoned
602025-01-28T17:54:02.376ZINFOcrucible-agent (worker): applying SMF actions before removal...
612025-01-28T17:54:02.377ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-02e7b20f-63f1-4dc8-823c-5df03598634f (instance states: (Some(Online), None))
622025-01-28T17:54:02.378ZINFOcrucible-agent (dropshot): request completed latency_us = 23802 local_addr = [fd00:1122:3344:101::1a]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:59832 req_id = 37872e7a-3452-4db2-8f3f-5fd6525e5282 response_code = 204 uri = /crucible/0/regions/02e7b20f-63f1-4dc8-823c-5df03598634f
632025-01-28T17:54:02.382ZINFOcrucible-agent (dropshot): request completed latency_us = 27 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:59832 req_id = f3816eb5-4609-4f70-820d-808317a35336 response_code = 200 uri = /crucible/0/regions/02e7b20f-63f1-4dc8-823c-5df03598634f
642025-01-28T17:54:02.384ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/e7f38630-0864-40b7-af99-2d3436c900ee
652025-01-28T17:54:02.384ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
662025-01-28T17:54:02.384ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::1a
672025-01-28T17:54:02.384ZINFOcrucible-agent (worker): reconfig required
682025-01-28T17:54:02.384ZINFOcrucible-agent (worker): using existing config property group
692025-01-28T17:54:02.384ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-e7f38630-0864-40b7-af99-2d3436c900ee
702025-01-28T17:54:02.385ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/e7f38630-0864-40b7-af99-2d3436c900ee
712025-01-28T17:54:02.385ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
722025-01-28T17:54:02.385ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
732025-01-28T17:54:02.385ZINFOcrucible-agent (worker): commit
742025-01-28T17:54:02.392ZINFOcrucible-agent (worker): ok!
752025-01-28T17:54:02.392ZINFOcrucible-agent (worker): SMF ok!
762025-01-28T17:54:02.403ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_37b8192c-9236-4941-bad3-2ab9840de50c/crucible/regions/02e7b20f-63f1-4dc8-823c-5df03598634f" region = 02e7b20f-63f1-4dc8-823c-5df03598634f
772025-01-28T17:54:02.460ZINFOcrucible-agent (datafile): region 02e7b20f-63f1-4dc8-823c-5df03598634f state: Tombstoned -> Destroyed
782025-01-28T17:54:02.478ZINFOcrucible-agent (dropshot): request completed latency_us = 87 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:59832 req_id = 4675b91a-08b8-4c19-915c-140bfabe6ce3 response_code = 200 uri = /crucible/0/regions/02e7b20f-63f1-4dc8-823c-5df03598634f