Max level shown:
12025-11-03T20:24:48.216ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::b]:64764
22025-11-03T20:24:48.274ZINFOcrucible-agent (datafile): region e72141c9-aa64-4a87-8624-da41ec585450 state: Requested
32025-11-03T20:24:48.282ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-11-03T20:24:48.282ZINFOcrucible-agent (dropshot): request completed latency_us = 25929 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64764 req_id = b64ed047-bd83-44e2-82b5-46c6a37cfe09 response_code = 200 uri = /crucible/0/regions
52025-11-03T20:24:48.295ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_d988f988-2649-4d71-a2a0-0f194156e959/crucible/regions/e72141c9-aa64-4a87-8624-da41ec585450
62025-11-03T20:24:48.295ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_d988f988-2649-4d71-a2a0-0f194156e959/crucible/regions/e72141c9-aa64-4a87-8624-da41ec585450
72025-11-03T20:24:48.366ZINFOcrucible-agent (worker): creating region Region { id: RegionId("e72141c9-aa64-4a87-8624-da41ec585450"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/e72141c9-aa64-4a87-8624-da41ec585450" region = e72141c9-aa64-4a87-8624-da41ec585450
82025-11-03T20:24:48.369ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "e72141c9-aa64-4a87-8624-da41ec585450" "--data" "/data/regions/e72141c9-aa64-4a87-8624-da41ec585450" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = e72141c9-aa64-4a87-8624-da41ec585450
92025-11-03T20:24:48.475ZINFOcrucible-agent (dropshot): request completed latency_us = 99 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64764 req_id = 441a0950-015f-450f-a810-3af9baf51577 response_code = 200 uri = /crucible/0/regions
102025-11-03T20:24:48.795ZINFOcrucible-agent (worker): region files created ok region = e72141c9-aa64-4a87-8624-da41ec585450
112025-11-03T20:24:48.795ZINFOcrucible-agent (datafile): region e72141c9-aa64-4a87-8624-da41ec585450 state: Requested -> Created
122025-11-03T20:24:48.795ZINFOcrucible-agent (worker): applying SMF actions post create...
132025-11-03T20:24:48.796ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-e72141c9-aa64-4a87-8624-da41ec585450
142025-11-03T20:24:48.803ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64764 req_id = a3ed6c46-3b6f-421e-987c-f46505c6e788 response_code = 200 uri = /crucible/0/regions
152025-11-03T20:24:48.809ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-e72141c9-aa64-4a87-8624-da41ec585450
162025-11-03T20:24:48.834ZINFOcrucible-agent (worker): creating config property group
172025-11-03T20:24:48.838ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-e72141c9-aa64-4a87-8624-da41ec585450
182025-11-03T20:24:48.839ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/e72141c9-aa64-4a87-8624-da41ec585450
192025-11-03T20:24:48.839ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
202025-11-03T20:24:48.839ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
212025-11-03T20:24:48.839ZINFOcrucible-agent (worker): commit
222025-11-03T20:24:48.844ZINFOcrucible-agent (worker): ok!
232025-11-03T20:24:48.849ZINFOcrucible-agent (worker): SMF ok!
242025-11-03T20:24:56.452ZINFOcrucible-agent (datafile): region e46fe8af-2d00-4d6c-9908-b288f3ca4b1c state: Requested
252025-11-03T20:24:56.453ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
262025-11-03T20:24:56.453ZINFOcrucible-agent (dropshot): request completed latency_us = 227 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64764 req_id = 2042271f-74cd-4007-9b0f-9cc40a7b2229 response_code = 200 uri = /crucible/0/regions
272025-11-03T20:24:56.459ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_d988f988-2649-4d71-a2a0-0f194156e959/crucible/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
282025-11-03T20:24:56.459ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_d988f988-2649-4d71-a2a0-0f194156e959/crucible/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
292025-11-03T20:24:56.588ZINFOcrucible-agent (worker): creating region Region { id: RegionId("e46fe8af-2d00-4d6c-9908-b288f3ca4b1c"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c" region = e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
302025-11-03T20:24:56.588ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "e46fe8af-2d00-4d6c-9908-b288f3ca4b1c" "--data" "/data/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
312025-11-03T20:24:56.665ZINFOcrucible-agent (dropshot): request completed latency_us = 86 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64764 req_id = a065778c-bf45-4fe6-902a-eb5b24a54cb0 response_code = 200 uri = /crucible/0/regions
322025-11-03T20:24:56.862ZINFOcrucible-agent (worker): region files created ok region = e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
332025-11-03T20:24:56.862ZINFOcrucible-agent (datafile): region e46fe8af-2d00-4d6c-9908-b288f3ca4b1c state: Requested -> Created
342025-11-03T20:24:56.862ZINFOcrucible-agent (worker): applying SMF actions post create...
352025-11-03T20:24:56.862ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
362025-11-03T20:24:56.870ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
372025-11-03T20:24:56.896ZINFOcrucible-agent (worker): creating config property group
382025-11-03T20:24:56.914ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
392025-11-03T20:24:56.915ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
402025-11-03T20:24:56.915ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
412025-11-03T20:24:56.915ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
422025-11-03T20:24:56.915ZINFOcrucible-agent (worker): commit
432025-11-03T20:24:56.933ZINFOcrucible-agent (worker): ok!
442025-11-03T20:24:56.940ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64764 req_id = ff0b7a44-4483-4e27-88f9-e277ebd7ba5d response_code = 200 uri = /crucible/0/regions
452025-11-03T20:24:56.948ZINFOcrucible-agent (worker): SMF ok!
462025-11-03T20:24:57.181ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::c]:44115
472025-11-03T20:24:57.181ZINFOcrucible-agent (dropshot): request completed latency_us = 71 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:44115 req_id = 2d761507-4530-4693-9f9d-720012e1bfd8 response_code = 200 uri = /crucible/0/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
482025-11-03T20:24:57.250ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::b]:45096
492025-11-03T20:24:57.250ZINFOcrucible-agent (dropshot): request completed latency_us = 38 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:45096 req_id = 8af40def-03dd-422a-80ee-8cce1ca3cdc4 response_code = 200 uri = /crucible/0/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
502025-11-03T20:25:31.850ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::b]:55353
512025-11-03T20:25:31.851ZINFOcrucible-agent (dropshot): request completed latency_us = 52 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:55353 req_id = 31fab277-5552-4ed7-a2e2-7a40bf3dd97a response_code = 200 uri = /crucible/0/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
522025-11-03T20:25:31.855ZINFOcrucible-agent (datafile): path is "/data/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c"
532025-11-03T20:25:31.866ZINFOcrucible-agent (datafile): dataset is oxp_d988f988-2649-4d71-a2a0-0f194156e959/crucible/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
542025-11-03T20:25:31.873ZINFOcrucible-agent (datafile): region e46fe8af-2d00-4d6c-9908-b288f3ca4b1c state: Created -> Tombstoned
552025-11-03T20:25:31.873ZINFOcrucible-agent (worker): applying SMF actions before removal...
562025-11-03T20:25:31.873ZINFOcrucible-agent (dropshot): request completed latency_us = 18668 local_addr = [fd00:1122:3344:101::13]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:55353 req_id = 36a69b62-fc52-4670-9aaf-0cfb5e708db4 response_code = 204 uri = /crucible/0/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
572025-11-03T20:25:31.874ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-e46fe8af-2d00-4d6c-9908-b288f3ca4b1c (instance states: (Some(Online), None))
582025-11-03T20:25:31.876ZINFOcrucible-agent (dropshot): request completed latency_us = 31 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:55353 req_id = 1166032f-454f-4b1a-ad7c-3c27fb667c19 response_code = 200 uri = /crucible/0/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
592025-11-03T20:25:31.891ZINFOcrucible-agent (worker): SMF ok!
602025-11-03T20:25:31.900ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_d988f988-2649-4d71-a2a0-0f194156e959/crucible/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c" region = e46fe8af-2d00-4d6c-9908-b288f3ca4b1c
612025-11-03T20:25:31.982ZINFOcrucible-agent (datafile): region e46fe8af-2d00-4d6c-9908-b288f3ca4b1c state: Tombstoned -> Destroyed
622025-11-03T20:25:31.986ZINFOcrucible-agent (dropshot): request completed latency_us = 48 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:55353 req_id = 9f5bfc71-7874-4f83-999c-4dfcec595c73 response_code = 200 uri = /crucible/0/regions/e46fe8af-2d00-4d6c-9908-b288f3ca4b1c