Multiple sources of evidence support the idea that reading and visual word identification build upon statistical regularities in the (written) language. However, statistical learning is surely not a language-specific engine, and seems to be deeply embedded into the visual system. This begs the question: How special is letter and word processing, really? In this talk, I'll show that some fundamental mechanism underlying word representation and processing, widely thought to be reading-specific, apply in fact to many other types of visual objects. I'll also show that the distribution of information across wordforms guides eye movements, in a way that is simply explained by a fundamental, non-language-specific computational principle — efficient information take-up. These findings nicely integrate with other recent work in the lab with developing readers, adults exposed to artificial lexicons and rats trying to make sense of letter strings. Overall, we find clear traces of sensitivity to statistical structure in both linguistic and non-linguistic materials, and linguistic and non-linguistic animals — with striking similarities. Of course, this body of evidence doesn't necessarily mean that the whole lexical-semantic system can be explained by statistical learning. However, it does suggest that the ramifications of perception within language are more widespread than we would typically think, and perhaps we can explain very much of our linguistic behaviour without assuming language-specific computations.