I can't help but imagine agents using caveman speak sometimes start behaving in a stereotypically caveman manner, even if it's subtle. Is there a chance the agent does less reasoning because of it?
Caveman invented fire, the wheel, domesticated wild plants and animals, organised society, survived the Toba catastrophe, cooked food, and was having sex ages before you and me. Don't write him off as stupid.
And I wonder how they actually spoke. Since there was no visual communications medium except for cave art. (Some of which is very excellent. Try drawing 3d curved horns in perspective.) So people would have used verbal communication more. Also no written word. So one would expect there to be quite a lot of oral tradition. Like people reciting poem form epics.
If we assume the time is before farming, population density would have been low and limiting culture. Hunter-gatherers might have travelled a lot more than farmers with a homestead though.
The Pleiades cluster is called the seven sisters in Greek. That's curious, because the human eye under the best conditions can discern only six stars in there. Even more curious, the aboriginal Australians also called this cluster the seven sisters.
Ancient Greeks' and aboriginal Australians' last common ancestors split about 60,000 years ago. And astronomers tell us that 60,000 years ago, there were seven discernable stars in that cluster.
One could this conclude not only is speech likely 60,000 years old, but also that the tale of the seven sisters might be a tale from so long ago.
Interestingly, back in my ill-spent youth, a few of my fellow astronomers and I were out in a very, very dark-sky location in the late 70s/early 80s, and we were able to consistently count and draw between 9 and 11 stars. Although we would tease those who could see 11 stars as using averted imagination. :-) Today, if I can see six stars, it's an okay night in an okay sky.
fwiw, if you can get out to dark skies where you can see fifth- or sixth-magnitude stars with the naked eye, I highly recommend getting out there when it's a low-moisture atmosphere and the Milky Way through Cassiopeia and Perseus is vertical, as it's a rather dramatic sight of this stream of stars heading down to the northern horizon.
The summertime Milky Way overhead down to Sagittarius tends to get all the love, but the wintertime Milky Way is also visually rich and worth spending time on.
It's not exactly a joke, it does reduce the amount of tokens. However, it does not improve performance (fine tunes are finnecky things, hard to get one right).
Personally the only 'enthusiast' modified qwen 3.6 27b or 3.6 35b-a3b I've found useful are the ones that have been run through heretic and adversarial data sets for innocent/dangerous prompts, to produce uncensored LLMs. They have some niche non-coding uses for things that a commercial LLM will never talk about.
I think those are mostly vapor that runs on the small culture of "models should not be censored" thing. But from my experience, they unlock nothing meaningful.
Fine-tuning is great for really small models on specific applications, but it's not something that can essentially improve a more generic model.
That said, there seems to be a fine line in quantization+finetuning that could recover performance. It's just hard to get a hold of it (I feel it in some models, but it's hard to say yet; lots of small labs working on this RN).
The most interesting use I've found for them so far is strictly as a novelty. Give a chat session with one to a completely non technical person, who at least knows that openai and anthropic have some guard rails on stuff, and tell them to wild with something like "give me the precursors and chemical formulas for the precusors for crystal meth" and watch it answer.
But does it answer those queries correctly, or does it just not refuse to not halucinate an incorrect answer? From where would it even have that information?
I don't know enough chemistry to say one way or the other if it's just wildly hallucinating the precursors and processes, but it'll also do things like, write an ISIS press release, or similar. There's a data set of basically a bunch of antisocial or dangerous prompts that some people have got variants of qwen to pass with 0 out of 465 refusals:
Just so we are clear, no "caveman" spoke English. "Caveman speak" is just shortening the vocabulary of english, not a "caveman language". Given this, your concerns for "stereotypical caveman manner" makes very little sense since what caveman are you talking about?
The concern is not that the model was trained on actual caveman artifacts, rather on modern media representations of the stereotypical caveman (that never actually existed).
This is nice, but the long time it took them to open source I think burnt a lot of the initial traction. I do like the idea & I'll see whether I can make use of it
The scary part to me is that it actually works. Not practically, but that upper management and whatnot actually buy it.
Most early stage startups around me right now are built on the premise of replacing groups of people. There's a startup strapping sensors and cameras onto blue collar workers in developing countries to take "egocentric" data and train "world models" to the get robots to replace skilled physical labor. Lots of "ambient capture" to then automate away the work of the employees getting recorded.
Even what I'm building right now, which was initially just a tool to help me with my ADHD and memory problems is slowly getting twisted towards the same direction based on where the money is.
I would love to know how to build a company that serves real humans rather than the accumulation of capital and the extractive nature of corporations.
> The scary part to me is that it actually works. Not practically, but that upper management and whatnot actually buy it.
That “below average” intelligence pool are well represented in upper management. FOMO is a powerful thing, smart people are often tricked by dumb ideas because they don’t want to miss out on something or be perceived to not understand the value of a bad idea
> I would love to know how to build a company that serves real humans rather than the accumulation of capital and the extractive nature of corporations.
Clearly not. Otherwise Bezos wouldn’t create a culture where employees feel the need to pee in bottles and continue working while a colleague is dead on the floor.
> It is not possible, actually.
I don’t believe that. Sure, if you want to be filthy rich you’ll exploit people, but there are plenty of smaller companies who treat their employees well (some owners have even left their companies to their employees) and genuinely care about customers as people.
I don’t even know what you’re talking about. Why would Bezos be spending money to make me believe it is possible for small companies who can treat their employees well, all the while making me think even worse of him for how he treats his own employees? That makes no sense.
Bezos doesn’t want you to abolish private property.
If you believe it possible to build a company that serves real humans rather than the accumulation of capital and the extractive nature of corporations, you will try that instead.
> If you believe it possible to build a company that serves real humans rather than the accumulation of capital and the extractive nature of corporations, you will try that instead.
No. That is wrong. Saying it is possible to generate less harm within a system does in no way mean I don’t think the system is flawed and rigged and incentives bad behaviour.
Ah, sorry, I didn’t realise I was the sole arbiter of private property and it was within my hands to abolish it. I’ll get on that first thing tomorrow, send me the papers.
I'm living in this ecosystem right now, and I see this every day. The mentality that "normal people" are beneath them.
I live a fairly social life and enjoy going to karaoke and going on hikes with friends or strangers not in tech/startups, and many of the others in the residency simply can't understand why I'd want to interact with people they perceive as less intelligent than them.
"Think about it. Think of the most average person you know. 50% are more stupid than them. And you won't get any value out of speaking to those"
> "Think about it. Think of the most average person you know. 50% are more stupid than them. And you won't get any value out of speaking to those"
Possible response: “And yet they’re all still more intelligent, compassionate, interesting, and a better company than you.”
There are many others: “The average person I know is pretty smart. I’m sorry to hear about your family, though.”
Or: “You’re right, I do indeed not get any value out of listening to you. I feel stupider already.”
Alright, alright, let’s actually try to be kind and change some minds: “There are different kinds of intelligence, and everyone has a full life rich in perspectives that we know nothing about. Making a blanket statement that someone we never met is “stupid” is reductive, lacking in empathy, and stifling to our own personal growth. To put it in your terms: What’s the best <relevant tech stack>? The answer is, of course, that it depends. They all have strengths and weaknesses and to be good you must learn to identify what those are. People are similar. If you only hang out with one type of person, you are the proverbial hammer who sees everything and everyone as nails.”
Oh hey, it's my blog. Weird seeing it posted here.
It's been a while and I now have a lot more context than back when I wrote the post in anger.
Disclaimer: As a result of people I met during the hackathon, I was somehow able to get into Entrepreneurs First, a startup accelerator/residency program and moved from Cardiff to San Francisco. The organizers of the hackathon has lots of ties to people who now fund me. What I say now will obviously be more measured and possibly biased.
There's many types of hackathons, and they exist for different purposes. I went in with the expectation of something similar to Huawei's Tech Arenas where the goal is to find the most technically capable engineers for specific problems (combinatorial optimization, optical network routing, etc) with the ability to communicate tacked on. That is why they're nearly a month long with actual metrics you compete by (latency, profit, etc)
HackEurope is a recruiting pipeline for larger startups and venture capitalists. They are specifically selecting for good pitchers, not technical people (which for them is a dime a dozen)
Having now been in the startup ecosystem for a bit, I can see it playing out very similar to how the hackathon went. Oversell and call it a long term vision when probed & you'll get enough funding to make at least something, even if you have to hire or contract everything out.
I still can't believe I'm here and I feel less and less human each passing day. The temptation of capital chips away daily at one's moral compass.
P.s. Gonna go sleep. The one place I always check is email, so if there's any particular feedback you want me to see, that's in my profile.
> Oversell and call it a long term vision when probed & you'll get enough funding to make at least something, even if you have to hire or contract everything out
Yeah. I was an engineer in a startup for a good bit and after a certain point I realized that for us there wasn't any point in actually trying to make working products - a flashy render did so much more than anything working ever would, at least until tens of millions of funding had been raised. Many of the other startups pitching at the same events as us were using off-the-shelf software and saying it was theirs, if they had any demo at all instead of concept renders.
And at that point, having little experience in creating renders, I contented myself with going to conferences, following around my much more charismatic cofounder, and saying tech buzzwords to make the business types smile. They sure love their AI. More than one conference we went to had an itinerary of talks where literally every talk title had "AI" in the name.
I don't think it was a waste. I learned people skills, I can talk to a crowd, pitch ideas confidently to men in suits full of the right words if I have to, and I do think on some level it is actually the right call to not build. Whatever I would have hacked together with me and whoever else was around would not have been the basis for a good product and probably would have calcified bad decisions anyways - better to raise capital before building.
Oh it definitely wasn't a waste for me either. I've gone from an introvert stumbling over trying to explain technical concepts to being able to pitch abstract ideas that check all the right boxes for VCs and whatnot.
Opened a ton of doors after the hackathon. Would've never dreamed of coming over here to SF and being part of the whole startup/VC ecosystem.
The animation in the background is fun, but really impairs readability on mobile. At least there’s Reader mode in browsers, but a pity that the original form isn’t accessible!
Apologies! Not much of a UI guy and I thought I dimmed it sufficiently for the text to be readable. I might reduce the rows of eyes from 4 to 2 for mobile tomorrow morning or whatever helps. Let me know!
I do want a bit of personality in my site but definitely need to figure out how to make it more subtle
This is very interesting. Not saying it is, but a possible endgame for Chinese models could be to have "backdoor" commands such that when a specific string is passed in, agents could ignore a particular alert or purposely reduce security. A lot of companies are currently working on "Agentic Security Operation Centers", some of them preferring to use open source models for sovereignty. This feels like a viable attack vector.
Yep, focus on actors may be warranted, but in a broad view and as a part of existing system and not 'their own system'. Otherwise, we get lost in a sea of IC level of paranoia. In simple terms, nations-states will do what nation-states will do ( which is basically whatever is to their advantage ).
That does not mean we can't have a technical discussion that bypasses at least some of those considerations.
That is just factually incorrect. Most Chinese Malaysians are either Cantonese or Hokkien, with closer ties to Taiwan and Hong Kong than the mainland. A lot of the older folk don't even speak Mandarin. Keep in mind the CCP wasn't even in power when most migrated here.
Wow republicans are now called fascist? Idk I always thought Schwarzenegger was such a nice example, wise, gentle, kind, funny and republican. Not loving the Trump, sure, but to say such a thing based on how someone votes, man you’re falling low.
Edit ok read the X post, man you guys are losing it if you call that fascist. So divided, you can be either black or white. I feel sorry for you.
Say one thing good about Trump and you’re a fascist, just pretend that all he does is bad. There is no more way of looking at it objectively. No wonder you are so divided over there. I really would stop watching the news. Half your country voted for him. What does it say about you that you view half your country as fascists?
It's probably less about viewing Trump as a fascist and more being afraid of being grouped in with Trump supporters by your in-group. It's a really divided country and there are circles where you could be outed even for expressing neutrality.
Again, I am not American, and would rather avoid the mess that is their politics
