Max level shown:
1 [ Apr 18 21:38:14 Enabled. ]
2 [ Apr 18 21:38:14 Rereading configuration. ]
3 [ Apr 18 21:38:16 Rereading configuration. ]
4 [ Apr 18 21:38:21 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_8ddb2801-4d94-4455-8327-6dfcb84942a0/crucible -l [fd00:1122:3344:101::15]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-04-18T21:38:22.309ZINFOcrucible-agent: dataset: "oxp_8ddb2801-4d94-4455-8327-6dfcb84942a0/crucible"
72025-04-18T21:38:22.312ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::15]:32345
82025-04-18T21:38:22.312ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-04-18T21:38:22.328ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-04-18T21:38:22.453ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::15]:32345
112025-04-18T21:39:05.699ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::b]:42556
122025-04-18T21:39:05.733ZINFOcrucible-agent (datafile): region d789507d-c88a-4d46-933f-ae4f33608d23 state: Requested
132025-04-18T21:39:05.738ZINFOcrucible-agent (dropshot): request completed latency_us = 18383 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:42556 req_id = e4c297df-4de6-467f-aee9-42dc262035a0 response_code = 200 uri = /crucible/0/regions
142025-04-18T21:39:05.741ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152025-04-18T21:39:05.750ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_8ddb2801-4d94-4455-8327-6dfcb84942a0/crucible/regions/d789507d-c88a-4d46-933f-ae4f33608d23
162025-04-18T21:39:05.750ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_8ddb2801-4d94-4455-8327-6dfcb84942a0/crucible/regions/d789507d-c88a-4d46-933f-ae4f33608d23
172025-04-18T21:39:05.813ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d789507d-c88a-4d46-933f-ae4f33608d23"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d789507d-c88a-4d46-933f-ae4f33608d23" region = d789507d-c88a-4d46-933f-ae4f33608d23
182025-04-18T21:39:05.816ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d789507d-c88a-4d46-933f-ae4f33608d23" "--data" "/data/regions/d789507d-c88a-4d46-933f-ae4f33608d23" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = d789507d-c88a-4d46-933f-ae4f33608d23
192025-04-18T21:39:05.918ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:42556 req_id = 383a269f-79a9-4f92-b780-13c7d6ef068d response_code = 200 uri = /crucible/0/regions
202025-04-18T21:39:06.119ZINFOcrucible-agent (worker): region files created ok region = d789507d-c88a-4d46-933f-ae4f33608d23
212025-04-18T21:39:06.119ZINFOcrucible-agent (datafile): region d789507d-c88a-4d46-933f-ae4f33608d23 state: Requested -> Created
222025-04-18T21:39:06.119ZINFOcrucible-agent (worker): applying SMF actions post create...
232025-04-18T21:39:06.119ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d789507d-c88a-4d46-933f-ae4f33608d23
242025-04-18T21:39:06.124ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d789507d-c88a-4d46-933f-ae4f33608d23
252025-04-18T21:39:06.151ZINFOcrucible-agent (worker): creating config property group
262025-04-18T21:39:06.166ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d789507d-c88a-4d46-933f-ae4f33608d23
272025-04-18T21:39:06.166ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d789507d-c88a-4d46-933f-ae4f33608d23
282025-04-18T21:39:06.166ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292025-04-18T21:39:06.167ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
302025-04-18T21:39:06.167ZINFOcrucible-agent (worker): commit
312025-04-18T21:39:06.171ZINFOcrucible-agent (worker): ok!
322025-04-18T21:39:06.178ZINFOcrucible-agent (worker): SMF ok!
332025-04-18T21:39:06.629ZINFOcrucible-agent (dropshot): request completed latency_us = 48 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:42556 req_id = b00c38e6-061a-43e4-82e1-392072d9aee3 response_code = 200 uri = /crucible/0/regions
342025-04-18T21:39:06.935ZINFOcrucible-agent (dropshot): request completed latency_us = 77 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:42556 req_id = 313f0487-890e-4d0a-84cf-e5f14b2fc1a8 response_code = 200 uri = /crucible/0/regions/d789507d-c88a-4d46-933f-ae4f33608d23
352025-04-18T21:39:06.938ZINFOcrucible-agent (datafile): path is "/data/regions/d789507d-c88a-4d46-933f-ae4f33608d23"
362025-04-18T21:39:06.950ZINFOcrucible-agent (datafile): dataset is oxp_8ddb2801-4d94-4455-8327-6dfcb84942a0/crucible/regions/d789507d-c88a-4d46-933f-ae4f33608d23
372025-04-18T21:39:06.960ZINFOcrucible-agent (datafile): region d789507d-c88a-4d46-933f-ae4f33608d23 state: Created -> Tombstoned
382025-04-18T21:39:06.960ZINFOcrucible-agent (worker): applying SMF actions before removal...
392025-04-18T21:39:06.960ZINFOcrucible-agent (dropshot): request completed latency_us = 22801 local_addr = [fd00:1122:3344:101::15]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:42556 req_id = 4709aae2-58ad-4836-83c2-646774131ac4 response_code = 204 uri = /crucible/0/regions/d789507d-c88a-4d46-933f-ae4f33608d23
402025-04-18T21:39:06.962ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-d789507d-c88a-4d46-933f-ae4f33608d23 (instance states: (Some(Online), None))
412025-04-18T21:39:06.963ZINFOcrucible-agent (dropshot): request completed latency_us = 22 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:42556 req_id = 580de9a7-a74b-4407-89ab-007f562bb418 response_code = 200 uri = /crucible/0/regions/d789507d-c88a-4d46-933f-ae4f33608d23
422025-04-18T21:39:06.966ZINFOcrucible-agent (worker): SMF ok!
432025-04-18T21:39:06.978ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_8ddb2801-4d94-4455-8327-6dfcb84942a0/crucible/regions/d789507d-c88a-4d46-933f-ae4f33608d23" region = d789507d-c88a-4d46-933f-ae4f33608d23
442025-04-18T21:39:07.046ZINFOcrucible-agent (datafile): region d789507d-c88a-4d46-933f-ae4f33608d23 state: Tombstoned -> Destroyed
452025-04-18T21:39:07.047ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:42556 req_id = bc605417-a640-4ef9-bda5-2d5096f33c34 response_code = 200 uri = /crucible/0/regions/d789507d-c88a-4d46-933f-ae4f33608d23
462025-04-18T21:39:09.436ZINFOcrucible-agent (datafile): region 415e16ab-5e9c-47a7-8814-378caf27b92b state: Requested
472025-04-18T21:39:09.436ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
482025-04-18T21:39:09.436ZINFOcrucible-agent (dropshot): request completed latency_us = 206 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:42556 req_id = c619fe1e-3263-4850-981f-c24991d5a958 response_code = 200 uri = /crucible/0/regions
492025-04-18T21:39:09.445ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_8ddb2801-4d94-4455-8327-6dfcb84942a0/crucible/regions/415e16ab-5e9c-47a7-8814-378caf27b92b
502025-04-18T21:39:09.445ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_8ddb2801-4d94-4455-8327-6dfcb84942a0/crucible/regions/415e16ab-5e9c-47a7-8814-378caf27b92b
512025-04-18T21:39:09.502ZINFOcrucible-agent (worker): creating region Region { id: RegionId("415e16ab-5e9c-47a7-8814-378caf27b92b"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/415e16ab-5e9c-47a7-8814-378caf27b92b" region = 415e16ab-5e9c-47a7-8814-378caf27b92b
522025-04-18T21:39:09.504ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "415e16ab-5e9c-47a7-8814-378caf27b92b" "--data" "/data/regions/415e16ab-5e9c-47a7-8814-378caf27b92b" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 415e16ab-5e9c-47a7-8814-378caf27b92b
532025-04-18T21:39:09.579ZINFOcrucible-agent (worker): region files created ok region = 415e16ab-5e9c-47a7-8814-378caf27b92b
542025-04-18T21:39:09.579ZINFOcrucible-agent (datafile): region 415e16ab-5e9c-47a7-8814-378caf27b92b state: Requested -> Created
552025-04-18T21:39:09.579ZINFOcrucible-agent (worker): applying SMF actions post create...
562025-04-18T21:39:09.581ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-415e16ab-5e9c-47a7-8814-378caf27b92b
572025-04-18T21:39:09.585ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-415e16ab-5e9c-47a7-8814-378caf27b92b
582025-04-18T21:39:09.594ZINFOcrucible-agent (worker): creating config property group
592025-04-18T21:39:09.599ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-415e16ab-5e9c-47a7-8814-378caf27b92b
602025-04-18T21:39:09.599ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/415e16ab-5e9c-47a7-8814-378caf27b92b
612025-04-18T21:39:09.599ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
622025-04-18T21:39:09.599ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
632025-04-18T21:39:09.600ZINFOcrucible-agent (worker): commit
642025-04-18T21:39:09.605ZINFOcrucible-agent (worker): ok!
652025-04-18T21:39:09.611ZINFOcrucible-agent (worker): SMF ok!
662025-04-18T21:39:09.800ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:42556 req_id = c14b04b8-b472-41de-ae35-f65724473958 response_code = 200 uri = /crucible/0/regions