I think we need to entertain the possibility that a large portion (99%?) of what we humans write and speak is generated by something equivalent to a LLM -- even when we feel like we are being creative.

Being truly creative requires (a) rejecting "conventional wisdom", and (b) evaluating the new conclusion and its implications to distinguish between fanciful results (no value judgment here) and actual insight. It is helpful if you can communicate a new insight in a way that others can benefit from it. (You can let people make what they will of the fanciful stuff!)

How can an AI system be programmed to reject conventional wisdom and evaluate the implications thereof. In essence, one needs an anarchic spirit -- something Chomsky would surely appreciate.