Max level shown:
12026-03-19T08:37:17.147ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:61210
22026-03-19T08:37:17.196ZINFOcrucible-agent (datafile): region f719ec68-76ca-4d35-b9c3-9502fcbffe57 state: Requested
32026-03-19T08:37:17.201ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-03-19T08:37:17.201ZINFOcrucible-agent (dropshot): request completed latency_us = 23958 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:61210 req_id = 3efc7a37-6157-4505-8851-db884611da99 response_code = 200 uri = /crucible/0/regions
52026-03-19T08:37:17.211ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_adb49832-6aec-4c24-9355-4560c49a15c7/crucible/regions/f719ec68-76ca-4d35-b9c3-9502fcbffe57
62026-03-19T08:37:17.211ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_adb49832-6aec-4c24-9355-4560c49a15c7/crucible/regions/f719ec68-76ca-4d35-b9c3-9502fcbffe57
72026-03-19T08:37:17.270ZINFOcrucible-agent (worker): creating region Region { id: RegionId("f719ec68-76ca-4d35-b9c3-9502fcbffe57"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/f719ec68-76ca-4d35-b9c3-9502fcbffe57" region = f719ec68-76ca-4d35-b9c3-9502fcbffe57
82026-03-19T08:37:17.272ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "f719ec68-76ca-4d35-b9c3-9502fcbffe57" "--data" "/data/regions/f719ec68-76ca-4d35-b9c3-9502fcbffe57" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = f719ec68-76ca-4d35-b9c3-9502fcbffe57
92026-03-19T08:37:17.525ZINFOcrucible-agent (dropshot): request completed latency_us = 68 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:61210 req_id = 92f8a441-2472-4dfe-a777-cbbd4a5ed98a response_code = 200 uri = /crucible/0/regions
102026-03-19T08:37:17.676ZINFOcrucible-agent (worker): region files created ok region = f719ec68-76ca-4d35-b9c3-9502fcbffe57
112026-03-19T08:37:17.676ZINFOcrucible-agent (datafile): region f719ec68-76ca-4d35-b9c3-9502fcbffe57 state: Requested -> Created
122026-03-19T08:37:17.677ZINFOcrucible-agent (worker): applying SMF actions post create...
132026-03-19T08:37:17.677ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-f719ec68-76ca-4d35-b9c3-9502fcbffe57
142026-03-19T08:37:17.692ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-f719ec68-76ca-4d35-b9c3-9502fcbffe57
152026-03-19T08:37:17.720ZINFOcrucible-agent (worker): creating config property group
162026-03-19T08:37:17.725ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-f719ec68-76ca-4d35-b9c3-9502fcbffe57
172026-03-19T08:37:17.726ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f719ec68-76ca-4d35-b9c3-9502fcbffe57
182026-03-19T08:37:17.726ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192026-03-19T08:37:17.726ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
202026-03-19T08:37:17.726ZINFOcrucible-agent (worker): commit
212026-03-19T08:37:17.730ZINFOcrucible-agent (worker): ok!
222026-03-19T08:37:17.737ZINFOcrucible-agent (worker): SMF ok!
232026-03-19T08:37:18.112ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:61210 req_id = 6fe6b1a3-de47-46fb-af01-fe97f3c8b9fe response_code = 200 uri = /crucible/0/regions
242026-03-19T08:37:25.011ZINFOcrucible-agent (datafile): region 089185cc-756c-4b31-9730-be532fe29992 state: Requested
252026-03-19T08:37:25.012ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
262026-03-19T08:37:25.012ZINFOcrucible-agent (dropshot): request completed latency_us = 237 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:61210 req_id = 141cec11-f9f1-4409-96f2-646e0edf8e1e response_code = 200 uri = /crucible/0/regions
272026-03-19T08:37:25.018ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_adb49832-6aec-4c24-9355-4560c49a15c7/crucible/regions/089185cc-756c-4b31-9730-be532fe29992
282026-03-19T08:37:25.018ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_adb49832-6aec-4c24-9355-4560c49a15c7/crucible/regions/089185cc-756c-4b31-9730-be532fe29992
292026-03-19T08:37:25.092ZINFOcrucible-agent (worker): creating region Region { id: RegionId("089185cc-756c-4b31-9730-be532fe29992"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/089185cc-756c-4b31-9730-be532fe29992" region = 089185cc-756c-4b31-9730-be532fe29992
302026-03-19T08:37:25.092ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "089185cc-756c-4b31-9730-be532fe29992" "--data" "/data/regions/089185cc-756c-4b31-9730-be532fe29992" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 089185cc-756c-4b31-9730-be532fe29992
312026-03-19T08:37:25.210ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:61210 req_id = 0c849f8a-ac22-4c92-8c77-2b4e2b17a5dd response_code = 200 uri = /crucible/0/regions
322026-03-19T08:37:25.236ZINFOcrucible-agent (worker): region files created ok region = 089185cc-756c-4b31-9730-be532fe29992
332026-03-19T08:37:25.236ZINFOcrucible-agent (datafile): region 089185cc-756c-4b31-9730-be532fe29992 state: Requested -> Created
342026-03-19T08:37:25.236ZINFOcrucible-agent (worker): applying SMF actions post create...
352026-03-19T08:37:25.236ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-089185cc-756c-4b31-9730-be532fe29992
362026-03-19T08:37:25.245ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-089185cc-756c-4b31-9730-be532fe29992
372026-03-19T08:37:25.261ZINFOcrucible-agent (worker): creating config property group
382026-03-19T08:37:25.266ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-089185cc-756c-4b31-9730-be532fe29992
392026-03-19T08:37:25.267ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/089185cc-756c-4b31-9730-be532fe29992
402026-03-19T08:37:25.267ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
412026-03-19T08:37:25.267ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
422026-03-19T08:37:25.267ZINFOcrucible-agent (worker): commit
432026-03-19T08:37:25.273ZINFOcrucible-agent (worker): ok!
442026-03-19T08:37:25.289ZINFOcrucible-agent (worker): SMF ok!
452026-03-19T08:37:25.760ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:61210 req_id = 528b906c-10bc-45d9-8514-623ba5e465b3 response_code = 200 uri = /crucible/0/regions
462026-03-19T08:37:51.275ZINFOcrucible-agent (dropshot): request completed latency_us = 80 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:61210 req_id = e90c3e26-0ab3-4708-9f22-9dc9adc8755f response_code = 200 uri = /crucible/0/regions/089185cc-756c-4b31-9730-be532fe29992
472026-03-19T08:37:51.279ZINFOcrucible-agent (datafile): path is "/data/regions/089185cc-756c-4b31-9730-be532fe29992"
482026-03-19T08:37:51.291ZINFOcrucible-agent (datafile): dataset is oxp_adb49832-6aec-4c24-9355-4560c49a15c7/crucible/regions/089185cc-756c-4b31-9730-be532fe29992
492026-03-19T08:37:51.298ZINFOcrucible-agent (datafile): region 089185cc-756c-4b31-9730-be532fe29992 state: Created -> Tombstoned
502026-03-19T08:37:51.298ZINFOcrucible-agent (worker): applying SMF actions before removal...
512026-03-19T08:37:51.298ZINFOcrucible-agent (dropshot): request completed latency_us = 19013 local_addr = [fd00:1122:3344:101::15]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:61210 req_id = 721aa332-72c6-45b0-a73b-fc72d7241b59 response_code = 204 uri = /crucible/0/regions/089185cc-756c-4b31-9730-be532fe29992
522026-03-19T08:37:51.299ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-089185cc-756c-4b31-9730-be532fe29992 (instance states: (Some(Online), None))
532026-03-19T08:37:51.301ZINFOcrucible-agent (dropshot): request completed latency_us = 30 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:61210 req_id = 4c6329f7-abc8-4d8e-b015-1809174f4c7f response_code = 200 uri = /crucible/0/regions/089185cc-756c-4b31-9730-be532fe29992
542026-03-19T08:37:51.311ZINFOcrucible-agent (worker): SMF ok!
552026-03-19T08:37:51.318ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_adb49832-6aec-4c24-9355-4560c49a15c7/crucible/regions/089185cc-756c-4b31-9730-be532fe29992" region = 089185cc-756c-4b31-9730-be532fe29992
562026-03-19T08:37:51.372ZINFOcrucible-agent (datafile): region 089185cc-756c-4b31-9730-be532fe29992 state: Tombstoned -> Destroyed
572026-03-19T08:37:51.400ZINFOcrucible-agent (dropshot): request completed latency_us = 43 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:61210 req_id = acf4c6ee-6dac-478e-803c-d91c5465ffe3 response_code = 200 uri = /crucible/0/regions/089185cc-756c-4b31-9730-be532fe29992