Posting this two days before earnings seems to indicate that there is going to be some bad capex news.
zer00eyz 1 days ago [-]
Ya dont say.
Negative Free Cash Flow are the words every one on HN needs to learn.
Every thing before now, you will look back upon as the happy time. When every year storage, ram and compute grew and the prices went down. Your cloud provider made so much money they paid cash for hardware.
Now they borrow at 8% and net margins on ram/storage were around $0.10/GB ... today they sit at over a dollar. (SEE micron's s10's if you want to check the math on that).
Forget circular financing, its a circular firing squad because your going to have to figure out where the hell your going to run any of your gear.
4x to 10x for ram +8% for financing +30percent AWS margins. Good luck everyone its gonna get rough out there.
le-mark 24 hours ago [-]
Or hyperscalers revenue will continue to disappoint, there is no $60 trillion market for ai and there will be a glut of capacity for years to come!
jacquesm 23 hours ago [-]
The capital (heh) mistake in the whole assumption that these companies are worth that much is that they'll crash the economy in their zeal to get there. And when the economy goes down the drain so do the expenditures because in the end the companies that could spend money on AI are taking that money out of the economy to put it in the pockets of a tiny fraction of society. You can only do that if you are willing to leave the rest without food on the table and that sort of move is simply not very wise.
lp92 5 hours ago [-]
Yeah that's not going to happen. They will take a page from the DRAM manufacturers and fix prices. And they will get away with it too.
vkou 19 hours ago [-]
There is definitely a $60 trillion market for AI, just not by next year.
jacquesm 24 hours ago [-]
I am very lucky to have stocked up on RAM before this happened (256G) but I wished I had doubled or quadrupled that, an upgrade simply isn't going to happen now.
downrightmike 19 hours ago [-]
-10B on VR, -(The Compny's Future) on AI
islewis 24 hours ago [-]
> The Ariel variant of GB200 NVL72 pairs each NVIDIA Blackwell GPU with an NVIDIA Grace CPU and 2x NVIDIA ConnectX-7 400G NICs at a 1:1 CPU-to-GPU ratio, letting us deploy 800G of I/O per GPU earlier than the rest of the industry
If the 1:1 ratio was purely for I/O (big assumption, very likely wrong), I am curious why they did not just go with ConnectX-8? Other hyperscalers have been shipping NVL72 clusters with ConnectX-8 on roughly the same timeline, and it offers the same 800Gb/s of throughput.
Managing a custom Ariel variant (which seems to be a Meta-only build) is not insignificant complexity, I wonder if they had another need for that 2x CPU-GPU ratio in their workloads relative to the stock GB200s.
EDIT: I looked, the gb200s NVL72s never have shipped with CX-8s, I made this up. This decision could have totally been due to compute availability and timing
wmf 19 hours ago [-]
IIRC one of the analysts said 1:1 CPU:GPU was for recommendation algorithms and 1:2 is better for LLMs.
xeubie 1 days ago [-]
It seems that Meta didn't mention one aspect of their "next-generation" AI training system: their incompetent web crawler hammers websites while completely ignoring robots.txt. Meta's 57.141.0.0/24 made up about 90% of all requests to my server this past week, and it didn't stop after I blocked it. They continue to make a request every 2 seconds despite me responding with a 403 every time. Truly an engineering marvel that Meta has bestowed upon us.
edit: I just checked and they are back to hammering my server at 105 requests per minute. These are the Meta-owned ranges I'm blocking now, in case it helps anyone: 57.141.0.0/16, 57.142.0.0/15, 57.144.0.0/14, 57.148.0.0/15
imawakegnxoxo 15 hours ago [-]
This sounds suspiciously close to what used to be called a DDoS attack
So they can do something which used to be considered stalking
Every single thing this company does would get an individual thrown in jail
Why do the zuck and his goon squad get a free pass to literally be criminals?
throw0101a 11 hours ago [-]
> These are the Meta-owned ranges I'm blocking now, in case it helps anyone: 57.141.0.0/16, 57.142.0.0/15, 57.144.0.0/14, 57.148.0.0/15
What's Meta's ASN? May be simpler to just null-route the entire organization.
jacquesm 24 hours ago [-]
I've blocked Meta, OpenAI and Anthropic and yet they seem to find ways to get back in. It's pretty annoying.
ctoth 1 days ago [-]
nft add table inet filter
nft add set inet filter meta_crawl '{ type ipv4_addr; flags interval; }'
nft add element inet filter meta_crawl '{ 57.141.0.0/24 }'
nft add rule inet filter input ip saddr @meta_crawl tcp dport {80,443} reject with tcp reset
xeubie 1 days ago [-]
I did finally block them at the firewall level last night. They are constantly trying to connect anyway.
titzer 23 hours ago [-]
When you're a celebrity, they let you do it.
marcyb5st 18 hours ago [-]
Poison the responses with massive ai slop dumps.
lp92 5 hours ago [-]
That's probably the best way to be honest.
peddling-brink 17 hours ago [-]
Make them block you.
emmelaich 21 hours ago [-]
maybe just drop them rather than reject with tcp reset
jimmydoe 10 hours ago [-]
Those identified themselves are not the worst ones.
I have seen crawlers fake chrome and edge from a set of Brazil residential IPs crawling at 1.5m req/hour for 3 hours, and it comes back every four weeks at same time.
figmert 1 days ago [-]
> incompetent
I think that's not incompetence, I think that's by design.
solyin 24 hours ago [-]
How? What's the incentive structure for that design? There are no extra ads to be displayed here, so how does Meta get value from that design? Seems way more likely to be incompetence in my world view. Explain why that design is superior!
nixon_why69 20 hours ago [-]
Someone puts a high number of pages crawled per minute in their PSC. Doesn't matter if they're 403s.
Elsewhereindeed 20 hours ago [-]
[dead]
zamadatix 22 hours ago [-]
There are no extra ads to be had even by scanning sites with a robots.txt that allows them, because the value of pages could never have been in displaying ads but must have resided in the information on the pages itself.
abirch 23 hours ago [-]
With Muse, people can enter their bank account information and medical information so that it can work hard for them. For this, they need a lot of training material.
schleck8 23 hours ago [-]
LLM training data
water-drummer 18 hours ago [-]
Reverse Hanlon's Razor
imawakegnxoxo 15 hours ago [-]
Their apps and their hardware might be gorefests but you best believe their tracking software is rock solid with no possible failure modes
maxlin 18 hours ago [-]
bet it was written in PHP
alex1138 23 hours ago [-]
(Rhetorical question) Who would you trust scraping your website? Google, for who among all their issues I haven't seen complaints of "Google is ddosing the web", both pre and post LLM
Or Meta, run by someone of Zuckerberg's character that wants to dominate everything and it shows by how many requests per minute? He's one of the reasons sites now have to implement bot blocking technologies
jgalt212 23 hours ago [-]
I didn't want to have to put everything behind cloudflare, but this sort of nonsense made it the least bad path forward for us.
8 hours ago [-]
alberth 1 days ago [-]
Actual title: ”Inside Prometheus: Meta’s Journey to a Gigawatt-Scale Supercluster”
lmc 1 days ago [-]
Is this one of the pilot models they claim R&D tax credits for?
shell glob rather than RE? what's the world coming to!
clod_sux_dik 1 days ago [-]
[dead]
ma2kx 1 days ago [-]
Imagine your job is just replacing millions of Dollars worth of hardware every day. I mean there must be a reason the hold 8 GPUs per rack spare.
solyin 24 hours ago [-]
Yes the reason is they're pushing the GPUs hard so that they fail, and having hot spares saves the job until helping hands can go replace the bad component.
boredatoms 23 hours ago [-]
Theres a ton of people working at (neo)clouds doing exactly that
laweijfmvo 20 hours ago [-]
[dead]
Rendered at 22:17:51 GMT+0000 (Coordinated Universal Time) with Vercel.
Negative Free Cash Flow are the words every one on HN needs to learn.
Every thing before now, you will look back upon as the happy time. When every year storage, ram and compute grew and the prices went down. Your cloud provider made so much money they paid cash for hardware.
Now they borrow at 8% and net margins on ram/storage were around $0.10/GB ... today they sit at over a dollar. (SEE micron's s10's if you want to check the math on that).
Forget circular financing, its a circular firing squad because your going to have to figure out where the hell your going to run any of your gear.
4x to 10x for ram +8% for financing +30percent AWS margins. Good luck everyone its gonna get rough out there.
If the 1:1 ratio was purely for I/O (big assumption, very likely wrong), I am curious why they did not just go with ConnectX-8? Other hyperscalers have been shipping NVL72 clusters with ConnectX-8 on roughly the same timeline, and it offers the same 800Gb/s of throughput.
Managing a custom Ariel variant (which seems to be a Meta-only build) is not insignificant complexity, I wonder if they had another need for that 2x CPU-GPU ratio in their workloads relative to the stock GB200s.
EDIT: I looked, the gb200s NVL72s never have shipped with CX-8s, I made this up. This decision could have totally been due to compute availability and timing
edit: I just checked and they are back to hammering my server at 105 requests per minute. These are the Meta-owned ranges I'm blocking now, in case it helps anyone: 57.141.0.0/16, 57.142.0.0/15, 57.144.0.0/14, 57.148.0.0/15
So they can do something which used to be considered stalking
Every single thing this company does would get an individual thrown in jail
Why do the zuck and his goon squad get a free pass to literally be criminals?
What's Meta's ASN? May be simpler to just null-route the entire organization.
nft add set inet filter meta_crawl '{ type ipv4_addr; flags interval; }'
nft add element inet filter meta_crawl '{ 57.141.0.0/24 }'
nft add chain inet filter input '{ type filter hook input priority 0; }'
nft add rule inet filter input ip saddr @meta_crawl tcp dport {80,443} reject with tcp reset
I have seen crawlers fake chrome and edge from a set of Brazil residential IPs crawling at 1.5m req/hour for 3 hours, and it comes back every four weeks at same time.
I think that's not incompetence, I think that's by design.
Or Meta, run by someone of Zuckerberg's character that wants to dominate everything and it shows by how many requests per minute? He's one of the reasons sites now have to implement bot blocking technologies
https://news.ycombinator.com/item?id=49921118