Max level shown:
1 [ Jan 22 09:42:07 Enabled. ]
2 [ Jan 22 09:42:07 Rereading configuration. ]
3 [ Jan 22 09:42:12 Rereading configuration. ]
4 [ Jan 22 09:42:24 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_ad5cfcf5-0f28-4932-b8ad-4e30f2750980/crucible -l [fd00:1122:3344:101::15]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-01-22T09:42:24.154ZINFOcrucible-agent: dataset: "oxp_ad5cfcf5-0f28-4932-b8ad-4e30f2750980/crucible"
72025-01-22T09:42:24.156ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::15]:32345
82025-01-22T09:42:24.156ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-01-22T09:42:24.165ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-01-22T09:42:24.510ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::15]:32345
112025-01-22T09:43:02.740ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::c]:49137
122025-01-22T09:43:02.777ZINFOcrucible-agent (datafile): region 0615c17d-2984-4255-b429-532ca7cc92ff state: Requested
132025-01-22T09:43:02.782ZINFOcrucible-agent (dropshot): request completed latency_us = 17334 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:49137 req_id = fdbc77ca-a3b5-4cd5-9c06-95ba63e763cf response_code = 200 uri = /crucible/0/regions
142025-01-22T09:43:02.785ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152025-01-22T09:43:02.794ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_ad5cfcf5-0f28-4932-b8ad-4e30f2750980/crucible/regions/0615c17d-2984-4255-b429-532ca7cc92ff
162025-01-22T09:43:02.794ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_ad5cfcf5-0f28-4932-b8ad-4e30f2750980/crucible/regions/0615c17d-2984-4255-b429-532ca7cc92ff
172025-01-22T09:43:02.867ZINFOcrucible-agent (worker): creating region Region { id: RegionId("0615c17d-2984-4255-b429-532ca7cc92ff"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/0615c17d-2984-4255-b429-532ca7cc92ff" region = 0615c17d-2984-4255-b429-532ca7cc92ff
182025-01-22T09:43:02.869ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "0615c17d-2984-4255-b429-532ca7cc92ff" "--data" "/data/regions/0615c17d-2984-4255-b429-532ca7cc92ff" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 0615c17d-2984-4255-b429-532ca7cc92ff
192025-01-22T09:43:03.104ZINFOcrucible-agent (worker): region files created ok region = 0615c17d-2984-4255-b429-532ca7cc92ff
202025-01-22T09:43:03.104ZINFOcrucible-agent (datafile): region 0615c17d-2984-4255-b429-532ca7cc92ff state: Requested -> Created
212025-01-22T09:43:03.104ZINFOcrucible-agent (worker): applying SMF actions post create...
222025-01-22T09:43:03.104ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-0615c17d-2984-4255-b429-532ca7cc92ff
232025-01-22T09:43:03.112ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-0615c17d-2984-4255-b429-532ca7cc92ff
242025-01-22T09:43:03.127ZINFOcrucible-agent (dropshot): request completed latency_us = 81 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:49137 req_id = 055bd586-a47b-469c-93f6-0c3380f13f7f response_code = 200 uri = /crucible/0/regions
252025-01-22T09:43:03.128ZINFOcrucible-agent (worker): creating config property group
262025-01-22T09:43:03.144ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-0615c17d-2984-4255-b429-532ca7cc92ff
272025-01-22T09:43:03.145ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/0615c17d-2984-4255-b429-532ca7cc92ff
282025-01-22T09:43:03.147ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292025-01-22T09:43:03.147ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
302025-01-22T09:43:03.147ZINFOcrucible-agent (worker): commit
312025-01-22T09:43:03.152ZINFOcrucible-agent (worker): ok!
322025-01-22T09:43:03.156ZINFOcrucible-agent (worker): SMF ok!
332025-01-22T09:43:03.954ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:49137 req_id = b9de829f-95f8-470d-a57f-9cc4169b6529 response_code = 200 uri = /crucible/0/regions/0615c17d-2984-4255-b429-532ca7cc92ff
342025-01-22T09:43:03.957ZINFOcrucible-agent (datafile): path is "/data/regions/0615c17d-2984-4255-b429-532ca7cc92ff"
352025-01-22T09:43:03.968ZINFOcrucible-agent (datafile): dataset is oxp_ad5cfcf5-0f28-4932-b8ad-4e30f2750980/crucible/regions/0615c17d-2984-4255-b429-532ca7cc92ff
362025-01-22T09:43:03.978ZINFOcrucible-agent (datafile): region 0615c17d-2984-4255-b429-532ca7cc92ff state: Created -> Tombstoned
372025-01-22T09:43:03.978ZINFOcrucible-agent (worker): applying SMF actions before removal...
382025-01-22T09:43:03.979ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-0615c17d-2984-4255-b429-532ca7cc92ff (instance states: (Some(Online), None))
392025-01-22T09:43:03.980ZINFOcrucible-agent (dropshot): request completed latency_us = 22953 local_addr = [fd00:1122:3344:101::15]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:49137 req_id = c8e85cfd-a92a-4b8c-ab8d-ae8b2bdb6462 response_code = 204 uri = /crucible/0/regions/0615c17d-2984-4255-b429-532ca7cc92ff
402025-01-22T09:43:03.984ZINFOcrucible-agent (dropshot): request completed latency_us = 26 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:49137 req_id = 00c4c62f-01f9-4b19-88c9-3c1527128341 response_code = 200 uri = /crucible/0/regions/0615c17d-2984-4255-b429-532ca7cc92ff
412025-01-22T09:43:03.985ZINFOcrucible-agent (worker): SMF ok!
422025-01-22T09:43:03.997ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_ad5cfcf5-0f28-4932-b8ad-4e30f2750980/crucible/regions/0615c17d-2984-4255-b429-532ca7cc92ff" region = 0615c17d-2984-4255-b429-532ca7cc92ff
432025-01-22T09:43:04.067ZINFOcrucible-agent (datafile): region 0615c17d-2984-4255-b429-532ca7cc92ff state: Tombstoned -> Destroyed
442025-01-22T09:43:04.117ZINFOcrucible-agent (dropshot): request completed latency_us = 71 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:49137 req_id = 21b9b70e-391a-45ef-98ab-34daa842dfde response_code = 200 uri = /crucible/0/regions/0615c17d-2984-4255-b429-532ca7cc92ff
452025-01-22T09:43:06.149ZINFOcrucible-agent (datafile): region a4d3065f-ee77-4f0f-b508-7d9e268dece1 state: Requested
462025-01-22T09:43:06.149ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
472025-01-22T09:43:06.149ZINFOcrucible-agent (dropshot): request completed latency_us = 243 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:49137 req_id = e3d2bb66-a3f6-4705-97a4-ffeadd0bac16 response_code = 200 uri = /crucible/0/regions
482025-01-22T09:43:06.157ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_ad5cfcf5-0f28-4932-b8ad-4e30f2750980/crucible/regions/a4d3065f-ee77-4f0f-b508-7d9e268dece1
492025-01-22T09:43:06.157ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_ad5cfcf5-0f28-4932-b8ad-4e30f2750980/crucible/regions/a4d3065f-ee77-4f0f-b508-7d9e268dece1
502025-01-22T09:43:06.209ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a4d3065f-ee77-4f0f-b508-7d9e268dece1"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/a4d3065f-ee77-4f0f-b508-7d9e268dece1" region = a4d3065f-ee77-4f0f-b508-7d9e268dece1
512025-01-22T09:43:06.209ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "a4d3065f-ee77-4f0f-b508-7d9e268dece1" "--data" "/data/regions/a4d3065f-ee77-4f0f-b508-7d9e268dece1" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = a4d3065f-ee77-4f0f-b508-7d9e268dece1
522025-01-22T09:43:06.283ZINFOcrucible-agent (worker): region files created ok region = a4d3065f-ee77-4f0f-b508-7d9e268dece1
532025-01-22T09:43:06.283ZINFOcrucible-agent (datafile): region a4d3065f-ee77-4f0f-b508-7d9e268dece1 state: Requested -> Created
542025-01-22T09:43:06.283ZINFOcrucible-agent (worker): applying SMF actions post create...
552025-01-22T09:43:06.284ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a4d3065f-ee77-4f0f-b508-7d9e268dece1
562025-01-22T09:43:06.288ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a4d3065f-ee77-4f0f-b508-7d9e268dece1
572025-01-22T09:43:06.295ZINFOcrucible-agent (worker): creating config property group
582025-01-22T09:43:06.298ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a4d3065f-ee77-4f0f-b508-7d9e268dece1
592025-01-22T09:43:06.298ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a4d3065f-ee77-4f0f-b508-7d9e268dece1
602025-01-22T09:43:06.299ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
612025-01-22T09:43:06.299ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
622025-01-22T09:43:06.299ZINFOcrucible-agent (worker): commit
632025-01-22T09:43:06.302ZINFOcrucible-agent (worker): ok!
642025-01-22T09:43:06.306ZINFOcrucible-agent (worker): SMF ok!
652025-01-22T09:43:06.432ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:49137 req_id = 8aff26af-12ab-497f-955e-279febc87628 response_code = 200 uri = /crucible/0/regions