Abstract

Modular is a powerful and inherently hierarchical concept in the human brain to process a large variety of complex tasks. Converging evidence has shown several advantages to hierarchically modular network organizations in the human brain such as interpretability and evolvability of network function. Inspired by previous neuroscience studies, we propose MNN-CH, a novel modular neural network that is constructed with explored category hierarchy. The basic idea is learning to learn an optimized category hierarchy to decompose complex patterns. And specific patterns are imposed into corresponding modules to realize a transparent design of the neural network. Specifically, for a given classification task, each class or superclass is first represented as a prototype. Afterward, the category hierarchy is initially determined by investigating class similarity and gather similar ones to train each branch neural network (i.e., modular) separately. Finally, an error-driven prototype learning is introduced to refine the category hierarchy by updating the class-superclass affiliation. Experiment results on several image classification datasets show that our model has a good performance, especially in complex tasks. Beyond, we conduct an analysis to illustrate the tree-manner interpretability of the modular neural network.