A Witnessing Duality for ReLU Network DepthResearch Paper
How many hidden layers a ReLU network needs to compute exactly is a basic question about the expressiveness of depth. Exact representation questions of this kind are studied for real, rational and integer weights. For integer weights, Haase, Hertrich and Loho (arXiv:2302.12553) proved a logarithmic depth lower bound by a modular argument…