Gary Stix in Scientific American:
Recursion can refer to a procedure that references itself or similar procedures, or to a structure that contains an embedded structure of the same type. It is important because it allows one to build entities of theoretically unlimited complexity by progressively embedding structures within structures.
Some scholars think that language may be built on the use of recursive building blocks. Isn’t that a fundamental tenet of the modern linguistics pioneered by Noam Chomsky.
Yes, Chomsky’s view of language is that it is recursive, and this gives language its potential for infinite complexity—or what he has also called “discrete infinity.” In recent formulations, this is achieved through an “unbounded Merge,” in which elements are merged, and the merged elements can then themselves be merged—a process that can be repeated without bounds. To Chomsky, though, language is essentially internal thought, a mental monologue, known as I-language, and not a means of communication. The structure of language is therefore a by-product of internal thought. This implies a common structure, called “universal grammar,” that underlies all languages. But there is growing doubt as to whether such a structure exists.
More here.