Max level shown:
12025-02-23T14:23:25.675ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::b]:58645
22025-02-23T14:23:25.715ZINFOcrucible-agent (datafile): region dd28bcae-cae0-4d61-b15e-8f9f41758bff state: Requested
32025-02-23T14:23:25.720ZINFOcrucible-agent (dropshot): request completed latency_us = 17183 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:58645 req_id = ff2e4d84-428c-43ca-95dc-234f04760dec response_code = 200 uri = /crucible/0/regions
42025-02-23T14:23:25.720ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52025-02-23T14:23:25.733ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_14020d00-0a95-4460-937f-e7ccd7cacd12/crucible/regions/dd28bcae-cae0-4d61-b15e-8f9f41758bff
62025-02-23T14:23:25.733ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_14020d00-0a95-4460-937f-e7ccd7cacd12/crucible/regions/dd28bcae-cae0-4d61-b15e-8f9f41758bff
72025-02-23T14:23:25.810ZINFOcrucible-agent (worker): creating region Region { id: RegionId("dd28bcae-cae0-4d61-b15e-8f9f41758bff"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/dd28bcae-cae0-4d61-b15e-8f9f41758bff" region = dd28bcae-cae0-4d61-b15e-8f9f41758bff
82025-02-23T14:23:25.813ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "dd28bcae-cae0-4d61-b15e-8f9f41758bff" "--data" "/data/regions/dd28bcae-cae0-4d61-b15e-8f9f41758bff" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = dd28bcae-cae0-4d61-b15e-8f9f41758bff
92025-02-23T14:23:25.943ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:58645 req_id = 27914814-97f8-4ed5-a66f-4437d18c58d4 response_code = 200 uri = /crucible/0/regions
102025-02-23T14:23:26.238ZINFOcrucible-agent (worker): region files created ok region = dd28bcae-cae0-4d61-b15e-8f9f41758bff
112025-02-23T14:23:26.238ZINFOcrucible-agent (datafile): region dd28bcae-cae0-4d61-b15e-8f9f41758bff state: Requested -> Created
122025-02-23T14:23:26.238ZINFOcrucible-agent (worker): applying SMF actions post create...
132025-02-23T14:23:26.239ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-dd28bcae-cae0-4d61-b15e-8f9f41758bff
142025-02-23T14:23:26.251ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-dd28bcae-cae0-4d61-b15e-8f9f41758bff
152025-02-23T14:23:26.275ZINFOcrucible-agent (worker): creating config property group
162025-02-23T14:23:26.285ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-dd28bcae-cae0-4d61-b15e-8f9f41758bff
172025-02-23T14:23:26.285ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/dd28bcae-cae0-4d61-b15e-8f9f41758bff
182025-02-23T14:23:26.292ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192025-02-23T14:23:26.292ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
202025-02-23T14:23:26.292ZINFOcrucible-agent (worker): commit
212025-02-23T14:23:26.306ZINFOcrucible-agent (worker): ok!
222025-02-23T14:23:26.317ZINFOcrucible-agent (worker): SMF ok!
232025-02-23T14:23:26.556ZINFOcrucible-agent (dropshot): request completed latency_us = 50 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:58645 req_id = 693bce12-6c3b-4f68-be53-2cd72dd2e281 response_code = 200 uri = /crucible/0/regions
242025-02-23T14:23:32.159ZINFOcrucible-agent (datafile): region 0f0f208c-6022-45ce-be68-e73199d862ff state: Requested
252025-02-23T14:23:32.162ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
262025-02-23T14:23:32.162ZINFOcrucible-agent (dropshot): request completed latency_us = 3026 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:58645 req_id = 20e78de6-b62d-4534-b0ac-5d770aef1af3 response_code = 200 uri = /crucible/0/regions
272025-02-23T14:23:32.171ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_14020d00-0a95-4460-937f-e7ccd7cacd12/crucible/regions/0f0f208c-6022-45ce-be68-e73199d862ff
282025-02-23T14:23:32.171ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_14020d00-0a95-4460-937f-e7ccd7cacd12/crucible/regions/0f0f208c-6022-45ce-be68-e73199d862ff
292025-02-23T14:23:32.244ZINFOcrucible-agent (worker): creating region Region { id: RegionId("0f0f208c-6022-45ce-be68-e73199d862ff"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/0f0f208c-6022-45ce-be68-e73199d862ff" region = 0f0f208c-6022-45ce-be68-e73199d862ff
302025-02-23T14:23:32.245ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "0f0f208c-6022-45ce-be68-e73199d862ff" "--data" "/data/regions/0f0f208c-6022-45ce-be68-e73199d862ff" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 0f0f208c-6022-45ce-be68-e73199d862ff
312025-02-23T14:23:32.359ZINFOcrucible-agent (worker): region files created ok region = 0f0f208c-6022-45ce-be68-e73199d862ff
322025-02-23T14:23:32.359ZINFOcrucible-agent (datafile): region 0f0f208c-6022-45ce-be68-e73199d862ff state: Requested -> Created
332025-02-23T14:23:32.359ZINFOcrucible-agent (worker): applying SMF actions post create...
342025-02-23T14:23:32.360ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-0f0f208c-6022-45ce-be68-e73199d862ff
352025-02-23T14:23:32.366ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-0f0f208c-6022-45ce-be68-e73199d862ff
362025-02-23T14:23:32.378ZINFOcrucible-agent (worker): creating config property group
372025-02-23T14:23:32.383ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-0f0f208c-6022-45ce-be68-e73199d862ff
382025-02-23T14:23:32.383ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/0f0f208c-6022-45ce-be68-e73199d862ff
392025-02-23T14:23:32.383ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
402025-02-23T14:23:32.383ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
412025-02-23T14:23:32.383ZINFOcrucible-agent (worker): commit
422025-02-23T14:23:32.390ZINFOcrucible-agent (worker): ok!
432025-02-23T14:23:32.396ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/dd28bcae-cae0-4d61-b15e-8f9f41758bff
442025-02-23T14:23:32.396ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
452025-02-23T14:23:32.396ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::15
462025-02-23T14:23:32.396ZINFOcrucible-agent (worker): reconfig required
472025-02-23T14:23:32.396ZINFOcrucible-agent (worker): using existing config property group
482025-02-23T14:23:32.397ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-dd28bcae-cae0-4d61-b15e-8f9f41758bff
492025-02-23T14:23:32.397ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/dd28bcae-cae0-4d61-b15e-8f9f41758bff
502025-02-23T14:23:32.397ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
512025-02-23T14:23:32.397ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
522025-02-23T14:23:32.397ZINFOcrucible-agent (worker): commit
532025-02-23T14:23:32.407ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:58645 req_id = 5609463c-23ff-48d6-9ce4-a1f7c55c2b7f response_code = 200 uri = /crucible/0/regions
542025-02-23T14:23:32.412ZINFOcrucible-agent (worker): ok!
552025-02-23T14:23:32.412ZINFOcrucible-agent (worker): SMF ok!
562025-02-23T14:23:59.101ZINFOcrucible-agent (dropshot): request completed latency_us = 58 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:58645 req_id = a6a15a00-b7f5-4779-97b1-02a7a3257f14 response_code = 200 uri = /crucible/0/regions/0f0f208c-6022-45ce-be68-e73199d862ff
572025-02-23T14:23:59.105ZINFOcrucible-agent (datafile): path is "/data/regions/0f0f208c-6022-45ce-be68-e73199d862ff"
582025-02-23T14:23:59.117ZINFOcrucible-agent (datafile): dataset is oxp_14020d00-0a95-4460-937f-e7ccd7cacd12/crucible/regions/0f0f208c-6022-45ce-be68-e73199d862ff
592025-02-23T14:23:59.127ZINFOcrucible-agent (datafile): region 0f0f208c-6022-45ce-be68-e73199d862ff state: Created -> Tombstoned
602025-02-23T14:23:59.127ZINFOcrucible-agent (worker): applying SMF actions before removal...
612025-02-23T14:23:59.128ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-0f0f208c-6022-45ce-be68-e73199d862ff (instance states: (Some(Online), None))
622025-02-23T14:23:59.131ZINFOcrucible-agent (dropshot): request completed latency_us = 26374 local_addr = [fd00:1122:3344:101::15]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:58645 req_id = add8d9d2-bb7a-438d-ae80-0beae0f9b745 response_code = 204 uri = /crucible/0/regions/0f0f208c-6022-45ce-be68-e73199d862ff
632025-02-23T14:23:59.134ZINFOcrucible-agent (dropshot): request completed latency_us = 36 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:58645 req_id = 96179060-bcef-45e7-98b9-534b35beefcd response_code = 200 uri = /crucible/0/regions/0f0f208c-6022-45ce-be68-e73199d862ff
642025-02-23T14:23:59.138ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/dd28bcae-cae0-4d61-b15e-8f9f41758bff
652025-02-23T14:23:59.138ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
662025-02-23T14:23:59.138ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::15
672025-02-23T14:23:59.138ZINFOcrucible-agent (worker): reconfig required
682025-02-23T14:23:59.138ZINFOcrucible-agent (worker): using existing config property group
692025-02-23T14:23:59.138ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-dd28bcae-cae0-4d61-b15e-8f9f41758bff
702025-02-23T14:23:59.138ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/dd28bcae-cae0-4d61-b15e-8f9f41758bff
712025-02-23T14:23:59.138ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
722025-02-23T14:23:59.138ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
732025-02-23T14:23:59.138ZINFOcrucible-agent (worker): commit
742025-02-23T14:23:59.145ZINFOcrucible-agent (worker): ok!
752025-02-23T14:23:59.145ZINFOcrucible-agent (worker): SMF ok!
762025-02-23T14:23:59.157ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_14020d00-0a95-4460-937f-e7ccd7cacd12/crucible/regions/0f0f208c-6022-45ce-be68-e73199d862ff" region = 0f0f208c-6022-45ce-be68-e73199d862ff
772025-02-23T14:23:59.221ZINFOcrucible-agent (datafile): region 0f0f208c-6022-45ce-be68-e73199d862ff state: Tombstoned -> Destroyed
782025-02-23T14:23:59.224ZINFOcrucible-agent (dropshot): request completed latency_us = 86 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:58645 req_id = 654af9de-7a1f-486e-81ef-85bac08a114b response_code = 200 uri = /crucible/0/regions/0f0f208c-6022-45ce-be68-e73199d862ff