All thought is in language. Plato says (in a number of his dialogues, for instance, in the Cratylus) that thought is the conversation of the soul with (phases of) itself. What comes into thought when this conversation takes place is another matter. In the first instance, the dialogic aspect is determined in large part by the memory of past experiences as these are brought to the forefront of one’s consciousness; secondarily, by external stimuli.
Here is a contemporary example. Standing outside the New York Public Library at the entrance to the Research Library on Fifth Avenue and Forty-Second Street in Manhattan and waiting to be allowed in, I turn around to see the two sculptured lions on their pedestals that guard the building on the Fifth Avenue side. This immediately summons forth the Latin phrase my beloved wife Marianne (a Latinist and medievalist) taught me long ago, Hic sunt leones ‘here are lions’, which was to be found on ancient maps to signify that the cartographer did not have knowledge of what lay beyond the boundary at that point (and assumed that wild beasts lurked).
Clearly, the Latin phrase was triggered by the stone lions outside the Library. But it could not have been part of my thought in an immediately summonable linguistic form without the cherished memory of the person who taught the phrase to me.