Max level shown:
12026-05-15T02:54:01.682ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::b]:54778
22026-05-15T02:54:01.739ZINFOcrucible-agent (datafile): region f61b4a11-643e-4c33-8868-f096c6c239e8 state: Requested
32026-05-15T02:54:01.742ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-05-15T02:54:01.742ZINFOcrucible-agent (dropshot): request completed latency_us = 21946 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:54778 req_id = 9f70ef63-33af-43bc-8e45-cfbcbe062fca response_code = 200 uri = /crucible/0/regions
52026-05-15T02:54:01.752ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_7ad20e42-1a49-4451-b37a-7c0d0e1cbefb/crucible/regions/f61b4a11-643e-4c33-8868-f096c6c239e8
62026-05-15T02:54:01.752ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_7ad20e42-1a49-4451-b37a-7c0d0e1cbefb/crucible/regions/f61b4a11-643e-4c33-8868-f096c6c239e8
72026-05-15T02:54:01.801ZINFOcrucible-agent (worker): creating region Region { id: RegionId("f61b4a11-643e-4c33-8868-f096c6c239e8"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/f61b4a11-643e-4c33-8868-f096c6c239e8" region = f61b4a11-643e-4c33-8868-f096c6c239e8
82026-05-15T02:54:01.804ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "f61b4a11-643e-4c33-8868-f096c6c239e8" "--data" "/data/regions/f61b4a11-643e-4c33-8868-f096c6c239e8" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = f61b4a11-643e-4c33-8868-f096c6c239e8
92026-05-15T02:54:01.996ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:54778 req_id = 08ce45d9-344e-4503-b957-3e7cb1c604a9 response_code = 200 uri = /crucible/0/regions
102026-05-15T02:54:02.156ZINFOcrucible-agent (worker): region files created ok region = f61b4a11-643e-4c33-8868-f096c6c239e8
112026-05-15T02:54:02.156ZINFOcrucible-agent (datafile): region f61b4a11-643e-4c33-8868-f096c6c239e8 state: Requested -> Created
122026-05-15T02:54:02.156ZINFOcrucible-agent (worker): applying SMF actions post create...
132026-05-15T02:54:02.156ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-f61b4a11-643e-4c33-8868-f096c6c239e8
142026-05-15T02:54:02.164ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-f61b4a11-643e-4c33-8868-f096c6c239e8
152026-05-15T02:54:02.176ZINFOcrucible-agent (worker): creating config property group
162026-05-15T02:54:02.177ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-f61b4a11-643e-4c33-8868-f096c6c239e8
172026-05-15T02:54:02.178ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f61b4a11-643e-4c33-8868-f096c6c239e8
182026-05-15T02:54:02.178ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192026-05-15T02:54:02.178ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
202026-05-15T02:54:02.178ZINFOcrucible-agent (worker): commit
212026-05-15T02:54:02.180ZINFOcrucible-agent (worker): ok!
222026-05-15T02:54:02.183ZINFOcrucible-agent (worker): SMF ok!
232026-05-15T02:54:02.381ZINFOcrucible-agent (dropshot): request completed latency_us = 67 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:54778 req_id = 73f5cbae-bc8d-4a91-af8f-1c80e35eff6d response_code = 200 uri = /crucible/0/regions
242026-05-15T02:54:08.102ZINFOcrucible-agent (datafile): region b3395560-b337-426e-a855-0fcfedba0c9f state: Requested
252026-05-15T02:54:08.102ZINFOcrucible-agent (dropshot): request completed latency_us = 209 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:54778 req_id = f1fe0a8c-8df5-4f27-b0e3-89ae20c9d3c8 response_code = 200 uri = /crucible/0/regions
262026-05-15T02:54:08.102ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
272026-05-15T02:54:08.109ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_7ad20e42-1a49-4451-b37a-7c0d0e1cbefb/crucible/regions/b3395560-b337-426e-a855-0fcfedba0c9f
282026-05-15T02:54:08.109ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_7ad20e42-1a49-4451-b37a-7c0d0e1cbefb/crucible/regions/b3395560-b337-426e-a855-0fcfedba0c9f
292026-05-15T02:54:08.148ZINFOcrucible-agent (worker): creating region Region { id: RegionId("b3395560-b337-426e-a855-0fcfedba0c9f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/b3395560-b337-426e-a855-0fcfedba0c9f" region = b3395560-b337-426e-a855-0fcfedba0c9f
302026-05-15T02:54:08.148ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "b3395560-b337-426e-a855-0fcfedba0c9f" "--data" "/data/regions/b3395560-b337-426e-a855-0fcfedba0c9f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = b3395560-b337-426e-a855-0fcfedba0c9f
312026-05-15T02:54:08.203ZINFOcrucible-agent (worker): region files created ok region = b3395560-b337-426e-a855-0fcfedba0c9f
322026-05-15T02:54:08.203ZINFOcrucible-agent (datafile): region b3395560-b337-426e-a855-0fcfedba0c9f state: Requested -> Created
332026-05-15T02:54:08.203ZINFOcrucible-agent (worker): applying SMF actions post create...
342026-05-15T02:54:08.204ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-b3395560-b337-426e-a855-0fcfedba0c9f
352026-05-15T02:54:08.205ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-b3395560-b337-426e-a855-0fcfedba0c9f
362026-05-15T02:54:08.208ZINFOcrucible-agent (worker): creating config property group
372026-05-15T02:54:08.210ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-b3395560-b337-426e-a855-0fcfedba0c9f
382026-05-15T02:54:08.210ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b3395560-b337-426e-a855-0fcfedba0c9f
392026-05-15T02:54:08.210ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
402026-05-15T02:54:08.210ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
412026-05-15T02:54:08.210ZINFOcrucible-agent (worker): commit
422026-05-15T02:54:08.212ZINFOcrucible-agent (worker): ok!
432026-05-15T02:54:08.218ZINFOcrucible-agent (worker): SMF ok!
442026-05-15T02:54:08.348ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:54778 req_id = c35eb13b-39ad-4436-b60f-8b151d9af86c response_code = 200 uri = /crucible/0/regions
452026-05-15T02:54:32.838ZINFOcrucible-agent (dropshot): request completed latency_us = 73 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:54778 req_id = e76e3762-d0bf-4986-8082-25dd7d5cc37d response_code = 200 uri = /crucible/0/regions/b3395560-b337-426e-a855-0fcfedba0c9f
462026-05-15T02:54:32.841ZINFOcrucible-agent (datafile): path is "/data/regions/b3395560-b337-426e-a855-0fcfedba0c9f"
472026-05-15T02:54:32.850ZINFOcrucible-agent (datafile): dataset is oxp_7ad20e42-1a49-4451-b37a-7c0d0e1cbefb/crucible/regions/b3395560-b337-426e-a855-0fcfedba0c9f
482026-05-15T02:54:32.858ZINFOcrucible-agent (datafile): region b3395560-b337-426e-a855-0fcfedba0c9f state: Created -> Tombstoned
492026-05-15T02:54:32.858ZINFOcrucible-agent (worker): applying SMF actions before removal...
502026-05-15T02:54:32.858ZINFOcrucible-agent (dropshot): request completed latency_us = 17461 local_addr = [fd00:1122:3344:101::13]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:54778 req_id = ffc4d292-95b3-414e-9e63-a3f3d120af4c response_code = 204 uri = /crucible/0/regions/b3395560-b337-426e-a855-0fcfedba0c9f
512026-05-15T02:54:32.859ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-b3395560-b337-426e-a855-0fcfedba0c9f (instance states: (Some(Online), None))
522026-05-15T02:54:32.861ZINFOcrucible-agent (dropshot): request completed latency_us = 48 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:54778 req_id = 9340deac-a3d0-4e4d-9b78-0bdc78a70350 response_code = 200 uri = /crucible/0/regions/b3395560-b337-426e-a855-0fcfedba0c9f
532026-05-15T02:54:32.863ZINFOcrucible-agent (worker): SMF ok!
542026-05-15T02:54:32.871ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_7ad20e42-1a49-4451-b37a-7c0d0e1cbefb/crucible/regions/b3395560-b337-426e-a855-0fcfedba0c9f" region = b3395560-b337-426e-a855-0fcfedba0c9f
552026-05-15T02:54:32.928ZINFOcrucible-agent (datafile): region b3395560-b337-426e-a855-0fcfedba0c9f state: Tombstoned -> Destroyed
562026-05-15T02:54:33.003ZINFOcrucible-agent (dropshot): request completed latency_us = 46 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:54778 req_id = d64c0c91-898f-408c-a638-ac3db808ce50 response_code = 200 uri = /crucible/0/regions/b3395560-b337-426e-a855-0fcfedba0c9f