neuralese

Noun

neuralese (uncountable)

1. (artificial intelligence, machine learning) An optimized internal language used by an artificial intelligence, generally inscrutable and alien to human understanding.
   • However, DCP agents instead communicate with an automatically induced protocol of unstructured, real-valued recurrent state vectors—an artificial language we might call “neuralese,” which superficially bears little resemblance to natural language, and thus frustrates attempts at direct...2017 July, Jacob Andreas, Anca Dragan, Dan Klein, “Translating Neuralese”, in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, →DOI, page 232
   • We call this “neuralese” because unlike English words, these high-dimensional vectors are likely quite difficult for humans to interpret. In the past, researchers could get a good idea what LLMs were thinking simply by reading its chain of thought. Now researchers have to ask the model to translate...2025 April 3, Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, Romeo Dean, “AI 2027”, in AI 2027