I'm not who you responded to and I don't have any info on Google. Nor can I explain in detail due to NDAs. But multiple major players are working on something along the lines of what the parent is alluding to.
The "edge" AI landscape (in particular, what you can do with ~5W) is going to be nuts in about 18 months.
My uneducated guess says, not much. For running massive models you still need a ton of high-bandwidth interconnects between many individual chips/GPUs/etc since you need to do math across a few TB worth of weights. That's simply going to require more power (and more die area in I/O, and therefore more cost). Being able to run small models in tiny power envelopes is incredibly useful to people, but I believe it will be covering a different niche than what datacenters can provide. Likewise, you'll still need crazy amounts of high-end memory to populate whatever goes in these datacenters.
The only thing that will crash prices is reduced demand (duh) or, more interestingly, increased production. In particular, if CXMT is able to get their DDR5 fabs up to a reasonably high yield, that could add some downward price pressure (as could government subsidies). As well, if Micron/Kingston/Hynix think that CXMT is going to start cutting into their market share, they might be willing to either increases supply or drop prices. Unfortunately CXMT looks to be taking quite a while to get their new fab up to max capacity so that may take a year+ before anything manifests.
If you're interested in following the (publicly available) info on these sorts of things, check out what companies like Axelera, DeepX, and MemoryX are doing today and have on their roadmaps, as well as the sorts of chips/SoCs Qualcomm, Kinara (now NXP), and Ambarella currently have announced (or have on the market). And remember, that pretty much all of these chips on the market today were in initial development more or less when ChatGPT first launched. If you knew what you knew today (or a year ago) about what requirements current- and next-generation models would have (from a silicon perspective), what might you do differently? Think for instance, host system interconnects, amount and speed of on-package or on-die memory, image/video decode capabilities, int8 vs fp8 vs fp16 vs bf16 compute units, etc. And, consider that most "AI" stuff in development a few years ago was all 15nm or 12nm - because who was gonna pay big money to get fab capacity at 3nm to run some object detection models? So most of the stuff on the market today is on very old nodes and therefore not super power efficient.
How is “all RAM sold in consumer PCs must be registered ECC” equivalent to age verification laws? I’m not saying I agree with the premise but that is a bit of a leap
> ECC DIMMs are available only for the memory speeds guaranteed to work by the manufacturers of CPUs, like Intel and AMD, e.g. for up to DDR5-5600 in UDIMMs or DDR5-6400 in RDIMMs.
This is not true[1]. These DIMMs are not guaranteed to work with all Threadripper CPUs, but they are guaranteed themselves to be stable at their advertised frequencies, which are higher than JEDEC. (And from my experience, 9000-series threadripper is perfectly happy to drive 8-channel 1Rx8 at 7200MT/s.)
AMD (and possibly Intel, I don’t have any experience with their recent offerings) also lets you overclock memory completely on WRX90. So if you buy JEDEC sticks, you’re welcome to try to push them as hard as you can.
There are absolutely fast DDR5 ECC DIMMs[1]. Just break out your wallet (and piggy bank, and rainy day fund, and..)
The main reason there aren’t more options here is that a) high-frequency memory usually isn’t sufficiently stable, unless binned, and b) there’s very, very, very little demand.
I’ve been running a Supermicro X10SLH-LN6TF as my router, and it’s very well suited for the job. Power use is very high for a home router, but reasonable for a 1U router - especially considering the era it’s from and its capabilities (routing 5-8Gbps per stream depending on firewall config, able to handle a couple active streams, and line rate routing with no firewall).
I wouldn’t suggest anyone buy something like this today for home use, but 6 years ago when I purchased it there were not many 10GbE routers to be had period, let alone for the price ($300 roughly if I remember correctly ). Even considering the cost of power over the last 6 years, I still come out much cheaper than if I’d bought something more power efficient new at the time.
FWIW, I got 10 Gbit at home 2 years ago and built my router for $100 using a ThinkCentre M720q Tiny running on a Core i3-8100T (a mini-PC from 2019).
It draws around 15W idle, going up to 30W at full bore. For comparison, the 10G rental router from the ISP drew 15W no matter what (idle or full bore routing). This is with a 10GBase-T SFP+ which probably influences the power draw vs using a DAC.
It has no issues maxing out at 9.4 Gbit routing with a single or multiple streams of iperf3 with your basic IPv4 NAT+some port forwards config.
It looks that some O2 sensors that don't require a reference have been used (titania sensors) but even though they have some advantages, they are less precise and mostly obsolete.
> a place where you have sufficient access to the vehicle to drain it
Probably the only valid argument for people who park on the street.
> the right equipment
One $5 wrench, one $10 filter wrench (optional). One set of ramps ($40), or jack stands ($30) if you already have a jack. One drain pan, $10 (or free if you're resourceful). Total cost max $65. Cheaper if you look for deals, buy used, borrow from a friend. If you can't afford $65 once to save money in the long run while owning a car, you probably should've bought a cheaper car.
> the right disposal solutions
Every oil change requires a jug of oil to be purchased. You can drain your used oil into this jug and then dispose of it along with your other household hazardous waste. This is not hard.
> Most people who have cars do not have that.
I might believe this for a place to do an oil change, maybe. I struggle to believe most, but I would believe many. Aside from that, if you don't have those things, you are choosing not to have them.
Which is kind of the point. None of these things are hard, at all. The majority of car owners 100 years ago could adjust their own timing, clean distributor points, replace belts, etc. because if they couldn't, they'd be calling for a tow truck every few hundred miles. Those are all harder, and things have only gotten easier with time. If you can't do them, you are choosing not to, because there's an even easier solution - spending more money and getting someone else to do it for you.
In ~300k km worth of diy oil changes, I’ve yet to change a crush washer, and yet to have a drain plug leak.
I always replace them on friends’ Toyotas, because they seem more important, but on every car I’ve owned it hasn’t mattered. And if you take the least amount of effort to google “how to change oil on ________” (fill in the blank for your year, make, model), some forum or video will probably tell you exactly what steps to take, including whether or not changing a washer is necessary.
Costs me 15 cents per washer delivered, why bother risk it? The world doesn't need more cancerous used motor oil on the ground.
After downloading a service manual and doing many things myself it became very apparent that mechanics barely bother to do work the right way despite it coming at virtually zero extra effort.
It's quite rare to see them use a torque wrench on many bolts and if you ask them why "they know it by feel", cool, but why not use a torque wrench to the proper specs anyway? It's not any harder.
My favorite way to do it is in the auto parts store parking lot. They will help you cart your full drain pan back to their oil recycling receptacle and some will even prop the back door open for you to walk it straight there. The bonus being if you do happen to spill a bit you're not stuck having to power wash your own driveway. I've got a process down to where I can pull it off with one pair of nitrile gloves, one rag, and one trash bag, to keep any residue from the drain pan staining anything.
I am typing this on my return flight home from a business trip in Delhi. There are many other areas the Indian government needs to be focussing on first.
I had a similar view to you ~2 weeks ago. Spending some time there very quickly made me realize that there’s a lot of other things that are much more pressing.
I’ve tested this recently (this post week). Had a dns entry up and pointing to an nginx server for ~12 hours, zero requests. 17 seconds after the letsencrypt cert was issued, the floodgates opened. Over a dozen of requests per second.
I don't think it's necessarily specific to LE but rather to public certificate transparency logs. LE being free and easy to automate means it's very widely used these days, but if you theoretically go to a "pay" root CA and get a cert that covers thing.com and www.thing.com , the same probing will happen on the same time scale.
The "edge" AI landscape (in particular, what you can do with ~5W) is going to be nuts in about 18 months.
reply