Thinking about LLMs again because, unfortunately, I needed to think about LLMs again and I think I found a way to say something.

LLMs are a technique for matching input requests with semantically connected outputs from their training set, where 'semantically connected' is determined by how language in different contexts in the training data. That's what they do: find related language to what 'should come after' the prompt.

Which means that with enough input related to anything in the training set, you'll start getting the related training data (mixed and matched at the token level to varying degrees). Bomb making instructions? If they're in the training set, there's a way to extract them. Hacking tips and tricks? Of course. Other customers private data? If the LLM can access it, it can be triggered to share it.

Which means, fundamentally, the only way to control LLM output is... choosing the LLM training data. Apart from all of the odious business practices of the current suppliers, why are we trying to legislate LLM output when what we really need to enforce is ethical LLM training input? The rest is just noise; even banning LLMs makes no sense, at the end of the day they're just applied matrix multiplication and we've seen how 'banning maths' goes historically a few times already. #llm #MathsBan #IllegalNumbers