Generative AI and LLMs continue to provide the least controversial answer to any question I ask them. For my purposes, this makes them little more than a calculator for words, a generator of historical fiction short stories.
As I mentioned two years ago, this doesn’t make LLMs useless, but it does greatly shrink their usefulness – to those places where you want a general idea of the consensus…whether or not it’s correct, accurate, or legal. Just an average doesn’t necessarily represent any individual datapoint.
For, the more training data the generative AI providers shovel into their models, the greater the drift from credibility toward absurdity the generated consensus.
It’s one thing to train the models on all the scientific research. It’s another to train on all the books ever published (copyright issues aside for the moment). It’s quite another to train it on Reddit and Twitter. It’s yet another thing all together to treat all data equal independent of parody, satire, or propaganda.
Again, there are use cases for this (e.g. getting familiar with the basics of a topic in record time), but the moment you expect quality, credibility, or specifics…it collapses like a toy giraffe.
A toy giraffe that, when a person engages with it, can only – collapse.
As a metaphor for new technologies, this toy giraffe’s message is worth considering, “we break when any pressure is applied.”
General purpose LLMs will only get worse the more data they digest. Special purpose LLMs only trained on a specific context, a specific vertical, a rigidly curated & edited set of sources may achieve the level of expert these applications are hyped up to be.
But we may never know they exist because the most valuable use cases – national defense, cybersecurity, fraud detection – will never need (or desire) the visibility the general purpose LLMs require.
There was a time when the internet was mostly silly technology experiments. Where that it worked at all was the win. All the projects were primarily for delight of the creator and delight for others. There was no advertising. No polish. Simply a neopunk aesthetic of “let’s just do it, it’ll be fun.” As the internet has matured and my own children have occasionally referred to me as ‘Mr Business Business’. I’ve longed for a more unstable time, a more optimistic time, a more experimental time.
I thank the creators of the projects below for showing me the we can have internet silliness in 2024.
Heard of ShotSpotter? Microphones are installed across cities in the United States by police to detect gunshots, purported to not be very accurate. This is that, but for music. This is culture surveillance. No one notices, no one consents. But it’s not about catching criminals. It’s about catching vibes.
I loaded a substantial sample of my writing into Google’s Notebook LM and pressed the button to make a short podcast between to AI bots.
Yes, it’s absolutely bizarre to hear bots discuss how amazing kubb is (among other things).
Along the way, I also asked the Notebook what the three most controversial themes of my writing were:
The Ethics of Data Collection and Privacy in a Technology-Driven World
The Pitfalls of Growth as the Sole Metric of Success, Especially for Small Businesses
The Importance of Disconnecting from Technology and Embracing Offline Experiences
as well as what I’m missing:
Limited Discussion of Specific Technologies: This absence suggests that while van Buren recognizes the importance of technology in his work, he doesn't view it as the primary driver of innovation or success. His focus remains on human-centered aspects of work and life, suggesting a conscious effort to maintain a balanced relationship with technology rather than solely pursuing the latest advancements.
“We estimate raising Dogger Bank would cost £97.5bn, but would bring present value benefits of £622bn. Under the government’s standard method of cost-benefit analysis, this project would get a go-ahead, with a cost-benefit ratio of 6.2.”
Humans, hunter-gathering Neanderthals, lived in Doggerland 10,000BC – 7,500BC.
Geological surveys have suggested Doggerland stretched from what is now the east coast of Great Britain to what are now the Netherlands, the western coast of Germany and the peninsula of Jutland.[2] It was probably a rich habitat with human habitation in the Mesolithic period.
The archaeological potential of the area was first identified in the early 20th century, and interest intensified in 1931 when a fishing trawler operating east of the Wash dragged up a barbed antler point that was subsequently dated to a time when the area was tundra.
UK Archeaologists from Europes Lost Frontiers project are using seabed mapping (originally created by oil and gas companies) to create 3D renderings of 17,000 square miles of Doggerland. Including rivers, lakes systems. Extracting core samples via survey ships to hopefully capture DNA from the plants and animals that lived there – including wooly rhinoceroses in addition to the wooly mammoths and hunter-gathering Neanderthals.
Vessels have since dragged up remains of mammoths, lions and other animals, and a few prehistoric tools and spearpoints, arrowheads,
Evidence gathered allows study of past environments, ecological change, and human transition from hunter-gatherer to farming communities
It was flooded by rising sea levels around 6500–6200 BCE.
What caused the sea level to rise 180feet? (6 feet every 100 years for 3,000yrs)
The three Storegga Slides are amongst the largest known submarine landslides.
They occurred at the edge of Norway’s continental shelf in the Norwegian Sea, approximately 6225–6170 BCE.
The collapse involved an estimated 180 mi length of coastal shelf, with a total volume of 840 cu mi of debris, which caused a tsunami in the North Atlantic Ocean.
The flooding immediately turned Doggerland into an island, then, eventually the sea swallowed the island.
Right now, I’m working on a series, called ‘For Starters – The important stuff early-stage entrepreneurs too often leave until it’s too late’ it’s over at Substack, cuz I want to better understand the benefits of their distribution model.
“If you follow the recipes to the dot as printed in the book, sometimes it’s not going to work anyhow…The way ingredients behave in one part of the world might not be the same as how they behave where you are, for natural reasons…Are you getting discouraged? Well don’t…Recipes are there to give you a base to start from, inspiration…and also to explain the technical base on which you can then build….You will have to use common sense.”
“Being able to follow a recipe is like being able to read music, and you should feel free to make it your own a little, because nobody will mind if you like your broccoli a little more cooked than I do….”
“A recipe can’t tell you exactly how much liquid to add because of the fat content of the milk, the amount of protein in the flour, even the weather can affect the moistness of the dough.”
At the beginning of summer, I walked into the neighborhood library and the librarian asked me why I had the recipe for bread on my shirt.
“I’ve committed to entering a loaf of bread into the Minnesota State Fair.”
Yes in fact, a few weeks earlier, I selected a recipe out of Ken Forkish’s Flour Water Salt Yeast to master and have been baking 2 loaves of bread every week since.
No, I haven’t been happy with any of them. Thank you for asking.
In an attempt to get happier, I switched yeast (somewhat better), then I switched flour (much worse), then I introduced a kitchen mixer (much much worse), then I switched flour again (somewhat better).
As much as Forkish’s recipe is far more sophisticated than my t-shirt – or the title of his own book – turns out it’s also a long way from the level of sophistication required for success with this flour, this yeast, in this oven, in my kitchen, in summer, at this altitude.
At beer club, if you’ve a question about anything in your most recent batch the first response you’ll get from the more experience members, “Did you bring your notes?”
So, like anyone screwing around doing science, I’m taking notes. I’m documenting what seems to work in each batch and documenting how I diverge, inadvertently or otherwise, from what the recipes states.
In the end, if I’m successful, I’ll have this one bread recipe adapted/developed, and if written out comprehensively for someone else (even future me), it will likely be >4 pages (it’s already 2 pages). Four pages is quite a bit longer than four words.
This massive discrepancy in length is obvious to anyone having developed a recipe – especially one targeting commercial food equipment at scale. Or anyone having built and refined anything from zero. There are a number of non-obvious details that are only a concern if the goal is: make it repeatable.
The Ninety-Ninety Rule aims to remind us there are always non-obvious details, context, and decisions that are only encountered once we’re in the middle of an effort.
The Coastline Paradox reminds us of a similar phenomenon, if you were to carefully walk the entirety of any coastline, the distance walked will be longer than any measurement of the coastline. Which is to say, the complexity of a situation is far higher when you’re in the middle of it than when you’re an observer.
Years ago, Merlin Mann recorded (by my assessment) a classic NSFW rant entitled, Make Believe Help. In it, he drags all the internet publications trading in reframing common sense as the latest life hack. For contrast, Merlin evokes an Old Butcher – an expert in the small details gained through hundreds, thousands, of repetitions.
These reps matter.
Making the same cut over and over. Each time doing it wrong in a different way, in a different spot. Then again. And again. And again.
An active practice.
Reps are only thing that will develop skills beyond what fits on a t-shirt.
Everything that’s not real world reps lacks necessary context, the necessary details, the appreciation for how all the factors fit into place. All the factors, not just the obvious ones.
Accelerating skill development requires creating an environment for the reps, for the practice, including a post-rep assessment;
Was the target achieved?
What went well?
What didn’t?
What was different this time?
What do we want to deliberately focus on in the next rep?
Any individual instance matters less than the accumulation of all the instances. There will always be another instance.
Of course, this is why we have coaches, advisors, structured classes, and multi-year specialized educational programs – all to accelerate the acquisition of greater context for a price. To help us get better at defining the problem accurately.
In my work with startups, I’m continually listening for hints of such an environment an real world reps – whether dogfooding or helping a customer solve a problem today. This is usually evident in if, and how casually, they talk about details and problems non-obvious to an external observer. A surprisingly small number have. Some resist even the mere suggestion. I get it. Real world reps can smell like low-value work that doesn’t scale, especially when your end goal is to abstract and automate away the pesky details. But here is where real competitive advantages lay, not to mention early revenue.
Admittedly, as this story of the Vienna Beef company reminds us, we can be successful for a very long time without fully understanding our own non-obvious details.
No, at this moment, I have no evidence this particular recipe would even do well in the Minnesota State Fair. Yet, I’ve committed to this recipe because it looked interesting, delicious, and slightly challenging. All of which could be part of my problem.
How many more reps can I get in before the drop off date?
“…The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds;…who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls…”
