Computer-assisted cognitive training can help patients affected by several illnesses alleviate their cognitive deficits, or healthy people improve their mental performance. Adapting the difficulty of the exercises to how individuals perform in their execution is crucial to improve the effectiveness of cognitive training activities. We propose the use of Reinforcement Learning to learn how to automatically adapt the difficulty of computerized exercises for cognitive training. We illustrate a method to be initially used to learn difficulty-variation policies tailored for specific categories of trainees, and then to refine these policies for single individuals. We present the results of two user studies that provide evidence for the effectiveness of our method.