Second-hand supervised learning in Hebbian perceptrons

Abstract

We study the problem of supervised learning of a rule when there is an intermediary, or teacher, between the rule, or book, and the student. Both student and teacher learn using the Hebbian learning rule, but the student receives its examples exclusively from the teacher and the teacher from the book. We show that, in this situation, the rate at which the student learns the book's contents decreases with the presence of the teacher. Depending on the absolute training of the teacher (α) and the relative training of the student (t_s=α_s/α), we observe a crossover between the regime in which it is more advantageous to teach the student using ‘questions’ from the training set used by the teacher, and the regime in which additional ‘questions’ need to be presented to obtain a better performance.