The Wayward Axolotl

This is a little discussion I had with Claude, the LLM agent created by Anthropic. My prompts are in block-quotes. The rest of the text is from Claude. For more on the nature of knowledge, see Theories of Knowledge and Knowledge and Reality . ✦ ✦ ✦ Explain how the principle of information compression relates to LLMs. Information Compression and LLMs The relationship is deep and arguably foundational to what LLMs are. The core idea Information compression, in the technical sense, is about finding compact representations that preserve the essential structure of data while discarding redundancy. A good compressor must learn the underlying patterns — the grammar, regularities, and dependencies — of the data it’s compressing. LLMs are, in a meaningful sense, learned compressors of human language and knowledge. Training as compression When an LLM is trained on a vast corpus of text, it’s forced to distill billions of documents into a fi...

Talking with Ethan Howell about knowledge. Moderated by Eric Claussen .

Great minds discuss ideas; average minds discuss events; small minds discuss people. — Eleanor Roosevelt We can divide thought and discourse into three levels of abstraction that roughly correspond to the levels in that famous quote: Thought/discourse about people and ordinary events. Thought/discourse about entities and events that are beyond direct personal experience. Thought/discourse about abstract systems, processes and theories. At the first level, we think/talk about objects and events that we can directly engage with. At the second level, we think/talk about objects and events that are removed from personal experience, because they are too big, too small, too far away, etc. However, we are still talking about objects and events. At the third level, we think/talk about systems, processes and theories. We are either thinking/talking about abstract patterns that are divorced from any specific object or event, or we are think...

Discussion with Garrick about Adam Lanza, eulavism, primitivism, etc.

To almost everyone, it is perfectly clear and obvious what should be done. The difficulty is that these people seem to divide almost evenly on the problem, with large numbers thinking that the opposing half is just being silly. — Robert Nozick This essay is about how not to solve Newcomb’s paradox . I will not explain the paradox here. I will use the formulation from Wikipedia, but I have also presented my own version of the problem here . (In my version, the amounts and the labels are different.) The following table shows the relationship between the prediction and the outcome: Table 1 Prediction Choice Outcome A & B A & B $1,000 A & B B $0 B A & B $1,001,000 ...

Alex sent me some questions about dishonesty in the academy. I assume that they were inspired by the essay Why Most Academic Reasearch is Fake . In this post, I will answer his questions. What ‘big important problems’ are ignored by academic research? There are many examples of academic fields ignoring core issues so that they can focus on minutia. The notion of utility, which is widely used in philosophy and economics, is almost entirely taken for granted. Economists assume that utility can be measured with monetary metrics, such as GDP. Important questions, such as “What is utility?” and “Can we quantify utility?” are ignored, despite the notion of utility being essential to both moral philosophy and economics. Evolutionary theory is not well-defined in academic biology, despite being the core theory. Meanwhile, evolutionary biologists spend their time contriving Rube Goldberg mechanisms to avoid the implications of evolutionary theory, rather than trying ...