Max level shown:
1 [ Feb 25 03:48:28 Enabled. ]
2 [ Feb 25 03:48:28 Rereading configuration. ]
3 [ Feb 25 03:48:31 Rereading configuration. ]
4 [ Feb 25 03:48:36 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_28cbc31a-0341-47b3-9708-10d85c028aed/crucible -l [fd00:1122:3344:101::15]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-02-25T03:48:36.354ZINFOcrucible-agent: dataset: "oxp_28cbc31a-0341-47b3-9708-10d85c028aed/crucible"
72025-02-25T03:48:36.357ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::15]:32345
82025-02-25T03:48:36.357ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-02-25T03:48:36.374ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-02-25T03:48:36.502ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::15]:32345
112025-02-25T03:49:20.941ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::b]:53686
122025-02-25T03:49:20.964ZINFOcrucible-agent (datafile): region 8d6dcb5e-d274-490f-9f0b-00eb66b03dae state: Requested
132025-02-25T03:49:20.964ZINFOcrucible-agent (dropshot): request completed latency_us = 11340 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:53686 req_id = 5bedb8f3-e66b-4008-8cd2-2f47dfcd3cec response_code = 200 uri = /crucible/0/regions
142025-02-25T03:49:20.967ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152025-02-25T03:49:20.977ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_28cbc31a-0341-47b3-9708-10d85c028aed/crucible/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae
162025-02-25T03:49:20.977ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_28cbc31a-0341-47b3-9708-10d85c028aed/crucible/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae
172025-02-25T03:49:21.043ZINFOcrucible-agent (worker): creating region Region { id: RegionId("8d6dcb5e-d274-490f-9f0b-00eb66b03dae"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae" region = 8d6dcb5e-d274-490f-9f0b-00eb66b03dae
182025-02-25T03:49:21.045ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "8d6dcb5e-d274-490f-9f0b-00eb66b03dae" "--data" "/data/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 8d6dcb5e-d274-490f-9f0b-00eb66b03dae
192025-02-25T03:49:21.155ZINFOcrucible-agent (worker): region files created ok region = 8d6dcb5e-d274-490f-9f0b-00eb66b03dae
202025-02-25T03:49:21.155ZINFOcrucible-agent (datafile): region 8d6dcb5e-d274-490f-9f0b-00eb66b03dae state: Requested -> Created
212025-02-25T03:49:21.155ZINFOcrucible-agent (worker): applying SMF actions post create...
222025-02-25T03:49:21.155ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-8d6dcb5e-d274-490f-9f0b-00eb66b03dae
232025-02-25T03:49:21.159ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-8d6dcb5e-d274-490f-9f0b-00eb66b03dae
242025-02-25T03:49:21.170ZINFOcrucible-agent (worker): creating config property group
252025-02-25T03:49:21.174ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-8d6dcb5e-d274-490f-9f0b-00eb66b03dae
262025-02-25T03:49:21.174ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae
272025-02-25T03:49:21.177ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
282025-02-25T03:49:21.177ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
292025-02-25T03:49:21.177ZINFOcrucible-agent (worker): commit
302025-02-25T03:49:21.187ZINFOcrucible-agent (worker): ok!
312025-02-25T03:49:21.191ZINFOcrucible-agent (worker): SMF ok!
322025-02-25T03:49:21.337ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:53686 req_id = 542c97f1-519c-4553-8c4c-a2ba783cc7d8 response_code = 200 uri = /crucible/0/regions
332025-02-25T03:49:21.853ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:53686 req_id = aa8ed25e-3464-4eb2-84ab-4f34b54e96c1 response_code = 200 uri = /crucible/0/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae
342025-02-25T03:49:21.856ZINFOcrucible-agent (datafile): path is "/data/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae"
352025-02-25T03:49:21.867ZINFOcrucible-agent (datafile): dataset is oxp_28cbc31a-0341-47b3-9708-10d85c028aed/crucible/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae
362025-02-25T03:49:21.878ZINFOcrucible-agent (datafile): region 8d6dcb5e-d274-490f-9f0b-00eb66b03dae state: Created -> Tombstoned
372025-02-25T03:49:21.878ZINFOcrucible-agent (worker): applying SMF actions before removal...
382025-02-25T03:49:21.879ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-8d6dcb5e-d274-490f-9f0b-00eb66b03dae (instance states: (Some(Online), None))
392025-02-25T03:49:21.881ZINFOcrucible-agent (dropshot): request completed latency_us = 24959 local_addr = [fd00:1122:3344:101::15]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:53686 req_id = 5cd21194-3cf1-4dcb-82a4-9cf055dec658 response_code = 204 uri = /crucible/0/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae
402025-02-25T03:49:21.884ZINFOcrucible-agent (dropshot): request completed latency_us = 33 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:53686 req_id = a5f399d0-fbab-4c3d-b4da-270eb3ffc8c4 response_code = 200 uri = /crucible/0/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae
412025-02-25T03:49:21.884ZINFOcrucible-agent (worker): SMF ok!
422025-02-25T03:49:21.896ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_28cbc31a-0341-47b3-9708-10d85c028aed/crucible/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae" region = 8d6dcb5e-d274-490f-9f0b-00eb66b03dae
432025-02-25T03:49:21.960ZINFOcrucible-agent (dropshot): request completed latency_us = 76 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:53686 req_id = 0553f1ee-63d5-4195-913e-7d689eaa94ff response_code = 200 uri = /crucible/0/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae
442025-02-25T03:49:21.969ZINFOcrucible-agent (datafile): region 8d6dcb5e-d274-490f-9f0b-00eb66b03dae state: Tombstoned -> Destroyed
452025-02-25T03:49:22.126ZINFOcrucible-agent (dropshot): request completed latency_us = 51 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:53686 req_id = bae5fd84-f4ea-438e-a913-3efc3e2277ea response_code = 200 uri = /crucible/0/regions/8d6dcb5e-d274-490f-9f0b-00eb66b03dae
462025-02-25T03:59:35.755ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::b]:63204
472025-02-25T03:59:35.755ZINFOcrucible-agent (datafile): region c1c92982-3a9a-47db-b249-7bb9f9d4b6c9 state: Requested
482025-02-25T03:59:35.758ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492025-02-25T03:59:35.758ZINFOcrucible-agent (dropshot): request completed latency_us = 3172 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:63204 req_id = 6f9b8156-af1a-40dc-ae8a-ad9d9f39b3d9 response_code = 200 uri = /crucible/0/regions
502025-02-25T03:59:35.768ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_28cbc31a-0341-47b3-9708-10d85c028aed/crucible/regions/c1c92982-3a9a-47db-b249-7bb9f9d4b6c9
512025-02-25T03:59:35.768ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_28cbc31a-0341-47b3-9708-10d85c028aed/crucible/regions/c1c92982-3a9a-47db-b249-7bb9f9d4b6c9
522025-02-25T03:59:35.840ZINFOcrucible-agent (worker): creating region Region { id: RegionId("c1c92982-3a9a-47db-b249-7bb9f9d4b6c9"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/c1c92982-3a9a-47db-b249-7bb9f9d4b6c9" region = c1c92982-3a9a-47db-b249-7bb9f9d4b6c9
532025-02-25T03:59:35.842ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "c1c92982-3a9a-47db-b249-7bb9f9d4b6c9" "--data" "/data/regions/c1c92982-3a9a-47db-b249-7bb9f9d4b6c9" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = c1c92982-3a9a-47db-b249-7bb9f9d4b6c9
542025-02-25T03:59:35.918ZINFOcrucible-agent (worker): region files created ok region = c1c92982-3a9a-47db-b249-7bb9f9d4b6c9
552025-02-25T03:59:35.918ZINFOcrucible-agent (datafile): region c1c92982-3a9a-47db-b249-7bb9f9d4b6c9 state: Requested -> Created
562025-02-25T03:59:35.918ZINFOcrucible-agent (worker): applying SMF actions post create...
572025-02-25T03:59:35.919ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-c1c92982-3a9a-47db-b249-7bb9f9d4b6c9
582025-02-25T03:59:35.925ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-c1c92982-3a9a-47db-b249-7bb9f9d4b6c9
592025-02-25T03:59:35.939ZINFOcrucible-agent (worker): creating config property group
602025-02-25T03:59:35.943ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-c1c92982-3a9a-47db-b249-7bb9f9d4b6c9
612025-02-25T03:59:35.943ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/c1c92982-3a9a-47db-b249-7bb9f9d4b6c9
622025-02-25T03:59:35.943ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632025-02-25T03:59:35.943ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
642025-02-25T03:59:35.943ZINFOcrucible-agent (worker): commit
652025-02-25T03:59:35.947ZINFOcrucible-agent (worker): ok!
662025-02-25T03:59:35.951ZINFOcrucible-agent (worker): SMF ok!
672025-02-25T03:59:36.009ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:63204 req_id = 0d9e98c2-210e-4d41-b388-6df6a9303b8c response_code = 200 uri = /crucible/0/regions