Generative AI, traditional ML, and systems of magic

GenAI and traditional ML can both seem like magic when leveraged properly, but that doesn't absolve practitioners from understanding how they work. I think of ML as a lot like the highly structured type of magic often depicted in fantasy novels, video games, and Dungeons & Dragons. There are rules to magic and strict formulas to follow – if you say these words exactly right, move your hands like this, then throw a ball of sulphur and bat poop, you get a fireball. If you don't get it right, either nothing happens or something goes very wrong, depending on the tone of the system and the sadism of the person running your game (side note: who wants to play Mork Borg with me?). Wizards need to study for years to master new spells, but once mastered that spell is going to do exactly what it says on the tin, every time, without fail.

This is a bit like training a traditional ML model – for somatic components substitute an algorithm, for material components substitute training data, and for verbal components substitute the code that invokes and combines both. Get everything right, wait for it to run, and poof – you have a trained model. Sure, not every model is deterministic and its predictions won't always be accurate, but you knew that going in, and you can generally estimate the quality of its responses through testing.

If traditional ML is reliable, formulaic wizardry that requires years of study to master, generative AI is bypassing all that boring work by summoning a demon and trying to convince them to do it for you.

(Substitute a fey trickster, long-forgotten powerful spirit, or unknowable alien intelligence if you're uncomfortable with the demon metaphor – what's summoned doesn't really matter.)

I don't mean this in the Faustian sense of bartering your soul for favors, but in the simpler sense that you are going to have to ask that summoned being to do something without understanding how it thinks or what its intentions are, and what you get back depends on your phrasing in unpredictable ways.

You start out with what you thought was a simple question and then you have to keep trying different wording and adding more specifics until you coax out a response that looks generally right – although you can't really be sure unless you verify the answer yourself. The internet abounds with examples of very confident, completely made-up and/or wrong answers coming out of LLMs because the internet already abounded with that stuff and that's what the LLMs learned from. So you can never know if changing your wording or saying please or offering a sacrifice of the Soylent you brought for lunch would result in something more helpful or at least closer to correct.

Sounds pretty similar to wrangling with an omniscient (at least in terms of the internet) and inhuman intellect for precious knowledge that it may or may not want to give up. It might be telling the truth; it might be deliberately misleading you because you triggered something in its pattern recognition that warranted a troll response from one of the nastier back alleys of the dark web. It might be giving you a brilliant synthesis of multiple legitimate and hard to find sources or it might be feeding you conspiracy theories.

This is just not at all like the nice, safe, practical magic of traditional ML – put in the work, follow the formula, get the response.

I've seen – and even kind of like – the metaphor that you should treat an LLM like you'd treat an inexperienced but very eager intern. I would extend this by imagining you hired that intern because you were impressed with their educational background, but you misread "The Demonweb Pits, Layer 66 of The Abyss" as "Yale".

So by all means, wake an eldritch being from its millennia-long slumber to ask it what kind of birthday present you should get for your nephew who answers all of your questions with "skibidi Ohio" when he bothers to look up from playing Roblox on his iPad if you think that's the best and most efficient path forward. I'll be over here invoking Grundy's Grandiose Grecommendation Engine as soon as I can get my hands on that kid's toenail trimmings and figure out how to pronounce these words of mystical power.