Discussion
Something that's been nagging at me about the current wave of LLMs and other machine learning techniques is how utterly *crap* they are at actually learning.
Don't get me wrong, the machine vision stuff and suchlike things are pretty amazing and all, but they are so, *so* bad at actual learning.
For example, apparently Elon's touting that Tesla now has ten billion miles of supervised driving and was supposed to be FSD by that milestone. And it isn't.
Granted, there's a lot of stupid in the data gathering (why this moron thinks that monoscopic video is a good way to monitor what's going on in a three dimensional world is... a subject for another rant) but even still. *I* learned to drive more reliably than these systems did and I think I clocked maybe 500 miles total in practice driving. Probably less, tbh, to be well past the point that these current systems are right now.
Same's true for the LLMs. Seventy bajillion parameters from every written word they could scrape up, with millions of hours of human tagging assistance and trillions of CPU hours, and they're still not quite there. Most humans get to this point of literacy with a few hundred books (at most) and maybe a couple thousand hours of actual reading time.
So yeah. Machine learning has done some great things, but it's still really pathetically inefficient.
@wordshaper we actually just recorded a podcast about learning and one of the points I made is a classic psych point, that humans are incredible at learning from a relatively small number of examples. It is truly one of our remarkable abilities
@wordshaper culture is fascinating not just in terms of understanding the concrete realities around us but because you can see it as a vehicle for distributing an extraordinarily scalable set of structural patterns from which people extrapolate to many unimagined novel behaviors and situations (eg, cog scientist Cristine Legare gives the example, culture is how you know how to file into a room you've never seen, with people you've never met, and successfully take your seat in an auditorium)
@wordshaper humans do not even need written language for this! We learn IMMENSE patterns of behavior and expectations without books at all 😄
@grimalkina Culture is objectively wild! In a lot of ways it's like external brain-support, providing some measure of higher-level functionality and system guidance that we don't necessarily have to fully learn to use.
I have no doubt that at some point we (the computer scientist "we", that I'm very generously including myself in 😊 will figure out some digital analogs for some parts of. The current ML stuff... isn't that.
@grimalkina Machine learning's still neat! And I am *absolutely* looking forward to actual real productive uses of this -- a portable X-ray machine that does first-pass checking for a whole bunch of known diseases would be *amazing* for a large part of the world -- I'm just, I think, mostly annoyed we're using the word "learning" in all this (and "intelligence") and I'm thinking that should always get a reflexive Inigo Montoya GIF tossed at it. :)
@wordshaper yeah I get that, but you know funny enough as a human learning person the "learning" term really doesn't bother me. I think across all the experience I've had with stats, predictive modeling, ml and edtech and schools I really mostly saw people correctly take it to just mean "this changes in response to information" which honestly is a better mindset than a static one ("intelligence" privileges a static mind idea maybe bc of our cultural biases! An interesting testable question)
@grimalkina I think what bothers me about the (in my view, which is of course inarguably 100% correct 😊 use of "learning" is that it carries implications that are inaccurate and leads people to making assumptions and decisions based on that inaccurate implication. Not even in a malicious way, pareidolia is absolutely a thing, but secondary decisions are made based on primary misunderstandings, and once those get hardened into a system... it's difficult to fix, because systems are persistent.
Install bonfire.mavnn.eu
Get the full app experience
