On the VC Dimension of Autoregressive Linear Threshold FunctionsResearch Paper
An autoregressive model applies one fixed next-token rule to the current sequence, appends the produced token, and repeats. Intermediate tokens act as computation steps, which is one explanation of why chain-of-thought generation increases the power of language models. Joshi et al. introduced a learning framework for this process: a fixed generator runs for …