An apparatus, a method, a method of manufacturing and apparatus, and a method of constructing an integrated circuit are provided. The apparatus includes a teacher network; a student network; a plurality of knowledge bridges between the teacher network and the student network, where each of the plurality of knowledge bridges provides a hint about a function being learned, and where a hint includes a mean square error or a probability; and a loss function device connected to the plurality of knowledge bridges and the student network. The method includes training a teacher network; providing hints to a student network by a plurality of knowledge bridges between the teacher network and the student network; and determining a loss function from outputs of the plurality of knowledge bridges and the student network.
展开▼
