Max level shown:
12025-12-12T06:10:58.966ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:33523
22025-12-12T06:10:58.966ZINFOcrucible-agent (datafile): region deaa4220-8e31-4f1a-a45f-f15b50229eff state: Requested
32025-12-12T06:10:58.966ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-12-12T06:10:58.966ZINFOcrucible-agent (dropshot): request completed latency_us = 635 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33523 req_id = 01eb36af-85cb-456c-a7de-b97dcb898598 response_code = 200 uri = /crucible/0/regions
52025-12-12T06:10:58.975ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_b53e8dea-faa3-4c8c-b280-ddce60306ea6/crucible/regions/deaa4220-8e31-4f1a-a45f-f15b50229eff
62025-12-12T06:10:58.975ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_b53e8dea-faa3-4c8c-b280-ddce60306ea6/crucible/regions/deaa4220-8e31-4f1a-a45f-f15b50229eff
72025-12-12T06:10:59.036ZINFOcrucible-agent (worker): creating region Region { id: RegionId("deaa4220-8e31-4f1a-a45f-f15b50229eff"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/deaa4220-8e31-4f1a-a45f-f15b50229eff" region = deaa4220-8e31-4f1a-a45f-f15b50229eff
82025-12-12T06:10:59.037ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "deaa4220-8e31-4f1a-a45f-f15b50229eff" "--data" "/data/regions/deaa4220-8e31-4f1a-a45f-f15b50229eff" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = deaa4220-8e31-4f1a-a45f-f15b50229eff
92025-12-12T06:10:59.127ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33523 req_id = 903fb97a-3e8e-49d5-a859-c4ea009d2e87 response_code = 200 uri = /crucible/0/regions
102025-12-12T06:10:59.155ZINFOcrucible-agent (worker): region files created ok region = deaa4220-8e31-4f1a-a45f-f15b50229eff
112025-12-12T06:10:59.155ZINFOcrucible-agent (datafile): region deaa4220-8e31-4f1a-a45f-f15b50229eff state: Requested -> Created
122025-12-12T06:10:59.155ZINFOcrucible-agent (worker): applying SMF actions post create...
132025-12-12T06:10:59.175ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-deaa4220-8e31-4f1a-a45f-f15b50229eff
142025-12-12T06:10:59.191ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-deaa4220-8e31-4f1a-a45f-f15b50229eff
152025-12-12T06:10:59.210ZINFOcrucible-agent (worker): creating config property group
162025-12-12T06:10:59.216ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-deaa4220-8e31-4f1a-a45f-f15b50229eff
172025-12-12T06:10:59.216ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/deaa4220-8e31-4f1a-a45f-f15b50229eff
182025-12-12T06:10:59.216ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-12-12T06:10:59.216ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
202025-12-12T06:10:59.216ZINFOcrucible-agent (worker): commit
212025-12-12T06:10:59.224ZINFOcrucible-agent (worker): ok!
222025-12-12T06:10:59.231ZINFOcrucible-agent (worker): SMF ok!
232025-12-12T06:10:59.615ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33523 req_id = 378551f4-255b-4b8a-b91a-55a57056b397 response_code = 200 uri = /crucible/0/regions
242025-12-12T06:11:00.035ZINFOcrucible-agent (dropshot): request completed latency_us = 52 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33523 req_id = abf7c635-630d-4089-b9db-07043b361df9 response_code = 200 uri = /crucible/0/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e
252025-12-12T06:11:00.041ZINFOcrucible-agent (datafile): path is "/data/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e"
262025-12-12T06:11:00.047ZINFOcrucible-agent (datafile): dataset is oxp_b53e8dea-faa3-4c8c-b280-ddce60306ea6/crucible/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e
272025-12-12T06:11:00.055ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b53e8dea-faa3-4c8c-b280-ddce60306ea6/crucible/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e@14c08207-559e-4bc1-a2a3-1986ba50d651
282025-12-12T06:11:00.055ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b53e8dea-faa3-4c8c-b280-ddce60306ea6/crucible/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e", "14c08207-559e-4bc1-a2a3-1986ba50d651"]
292025-12-12T06:11:00.063ZINFOcrucible-agent (dropshot): request completed latency_us = 22180 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33523 req_id = 23ac9407-3715-4be7-811d-9eff006448ab response_code = 200 uri = /crucible/0/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e/snapshots/14c08207-559e-4bc1-a2a3-1986ba50d651
302025-12-12T06:11:00.071ZINFOcrucible-agent (datafile): path is "/data/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e"
312025-12-12T06:11:00.077ZINFOcrucible-agent (datafile): dataset is oxp_b53e8dea-faa3-4c8c-b280-ddce60306ea6/crucible/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e
322025-12-12T06:11:00.083ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b53e8dea-faa3-4c8c-b280-ddce60306ea6/crucible/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e@14c08207-559e-4bc1-a2a3-1986ba50d651
332025-12-12T06:11:00.083ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b53e8dea-faa3-4c8c-b280-ddce60306ea6/crucible/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e", "14c08207-559e-4bc1-a2a3-1986ba50d651"]
342025-12-12T06:11:00.089ZINFOcrucible-agent (datafile): requesting running snapshot 12d4049e-0c53-425f-9cb0-8b839f4b0a0e-14c08207-559e-4bc1-a2a3-1986ba50d651 state: Requested
352025-12-12T06:11:00.090ZINFOcrucible-agent (worker): applying SMF actions for region 12d4049e-0c53-425f-9cb0-8b839f4b0a0e running snapshot 14c08207-559e-4bc1-a2a3-1986ba50d651 (state Requested)...
362025-12-12T06:11:00.090ZINFOcrucible-agent (dropshot): request completed latency_us = 18848 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33523 req_id = ace23725-8970-4b58-918b-7f6a53444c34 response_code = 200 uri = /crucible/0/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e/snapshots/14c08207-559e-4bc1-a2a3-1986ba50d651/run
372025-12-12T06:11:00.094ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-12d4049e-0c53-425f-9cb0-8b839f4b0a0e-14c08207-559e-4bc1-a2a3-1986ba50d651
382025-12-12T06:11:00.098ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-12d4049e-0c53-425f-9cb0-8b839f4b0a0e-14c08207-559e-4bc1-a2a3-1986ba50d651
392025-12-12T06:11:00.119ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-12d4049e-0c53-425f-9cb0-8b839f4b0a0e-14c08207-559e-4bc1-a2a3-1986ba50d651
402025-12-12T06:11:00.120ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e/.zfs/snapshot/14c08207-559e-4bc1-a2a3-1986ba50d651
412025-12-12T06:11:00.120ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
422025-12-12T06:11:00.120ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
432025-12-12T06:11:00.120ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
442025-12-12T06:11:00.120ZINFOcrucible-agent (worker): commit
452025-12-12T06:11:00.129ZINFOcrucible-agent (worker): ok!
462025-12-12T06:11:00.135ZINFOcrucible-agent (worker): SMF ok!
472025-12-12T06:11:00.135ZINFOcrucible-agent (datafile): region 12d4049e-0c53-425f-9cb0-8b839f4b0a0e running snapshot 14c08207-559e-4bc1-a2a3-1986ba50d651 state: Requested -> Created
482025-12-12T06:11:00.280ZINFOcrucible-agent (datafile): path is "/data/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e"
492025-12-12T06:11:00.287ZINFOcrucible-agent (datafile): dataset is oxp_b53e8dea-faa3-4c8c-b280-ddce60306ea6/crucible/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e
502025-12-12T06:11:00.294ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b53e8dea-faa3-4c8c-b280-ddce60306ea6/crucible/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e@14c08207-559e-4bc1-a2a3-1986ba50d651
512025-12-12T06:11:00.294ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b53e8dea-faa3-4c8c-b280-ddce60306ea6/crucible/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e", "14c08207-559e-4bc1-a2a3-1986ba50d651"]
522025-12-12T06:11:00.299ZINFOcrucible-agent (dropshot): request completed latency_us = 19173 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33523 req_id = df6ae0be-af0f-4d14-ae91-448c60bd9fce response_code = 200 uri = /crucible/0/regions/12d4049e-0c53-425f-9cb0-8b839f4b0a0e/snapshots/14c08207-559e-4bc1-a2a3-1986ba50d651/run