> This write amplification is large enough that our efforts to tune indexing throughput have started to hit diminishing returns.
> don't key on the ANN address. That is precisely the change turbopuffer v3 makes. As you can imagine, it is not a trivial change.
This is a direct parallel to how Postgres and Mysql built indexes.
Your design choice went from a Postgres design pattern to a Mysql one. The difference is the reindexing cost vs the lookup cost - Postgres optimized for lookup and Mysql does for indexing on writes. Or more accurately, Postgres was better with good schema design using joins & mysql was optimized for a bad design with less normalization where many indexes exist for the same table.
Postgres always points an index to a row-id within postgres which is an arbitrary value which changes on each update.
Mysql, always assuming the storage engine is pluggable, points to the primary index entry and adds an extra indirection to the lookup.
This means that you point the mysql index to a stable id, so unless you go update the primary key for a row, you won't have to update the indexes for all the attribute lookups you might have made to data.
I don't do databases any more that much, but the design for NIMBLE file format has a lot of quirks which are relevant to this specific idea (wide tables).
But the old Uber post about switching from Postgres to Mysql to prevent index amplification[1] is a direct mirror to this post.
MySQL had worse-is-better dominance over postgres until, like, 2016 or so? I'll take the worse solution with better performance and a better replication story any day.
jgalt212 23 minutes ago [-]
I hear you, but there's just so many knobs on postgres that at our correct scale we couldn't amortize the time and effort to become proficient at postges.
__s 15 hours ago [-]
I literally had opening slide about Worse is Better when adding mysql support to peerdb. Lots of things in mysql have v1 as stupidest thing that works, then v2 fixes issues
There's a bunch of internal types like decimal vs newdecimal, binlog started out statement based until they realized uuid generation is random so added data replication on top. CDC offset started as filepos before GTID was made so offset could survive failover
There were aspects of the design I appreciated (logical slots in postgres have a bunch of drawbacks avoided by just appending to 2nd serial log which has an expiry date instead of tracking clients' offsets), but developing against protocol you learn to not try build a consistent mental model
malisper 18 hours ago [-]
> Your design choice went from a Postgres design pattern to a Mysql one. The difference is the reindexing cost vs the lookup cost - Postgres optimized for lookup and Mysql does for indexing on writes. Or more accurately, Postgres was better with good schema design using joins & mysql was optimized for a bad design with less normalization where many indexes exist for the same table
You are right that MySQL does better when you have lots of indexes, but I don't think the tradeoff is that the overall Postgres architecture is better with good schema design.
Having secondary indexes point the primary key enables things like undo logging, which obviates the need for vacuums - vacuums being the most painful part of Postgres. On top of that your primary key index will be mostly cached so the cost of the indirection is much smaller than it may first appear
tomnipotent 17 hours ago [-]
I think OP is just alluding to the fact that Postgres needs to do less work to go from secondary index to table data, since the tid is a direct pointer to the exact page and slotted entry while MySQL needs a b-tree walk.
> primary key index will be mostly cached so the cost of the indirection is much smaller than it may first appear
Not sure I follow. If it's in-memory you save having to read from disk, but you still have to walk the b-tree to go from PK to data.
malisper 14 hours ago [-]
> I think OP is just alluding to the fact that Postgres needs to do less work to go from secondary index to table data, since the tid is a direct pointer to the exact page and slotted entry while MySQL needs a b-tree walk.
Yes, this is true, but they framed this as "the Postgres approach is better when you have a good schema design", but that's not true. There are plenty of ways the MySQL approach is better even when you have a really good schema.
> Not sure I follow. If it's in-memory you save having to read from disk, but you still have to walk the b-tree to go from PK to data.
The point I was trying to make is that going to disk is going to be orders of magnitude slower than doing an in-memory B-tree traversal. Because of that, the cost of doing an extra b-tree traversal to find the page you're looking for is a relatively small cost compared to reading the page in the first place
barrkel 16 hours ago [-]
MySQL was generally (pre 8) optimized for point queries on primary keys. So rows are stored in the PK index, the PK index is a clustered index. Everything more or less falls out of this.
sroussey 14 hours ago [-]
Around v2 myisam was not a clustered index. It changed with innodb.
throwaway7783 7 hours ago [-]
.. and undo logs make rollbacks and crash recoveries slower. It's a trade off. MySQL storage engine architecture is nice, postgresql extension mechanism is nice.. and so on.
FLeXMurphy 19 hours ago [-]
I find it amusing people started quoting LLM output and are responding to it. Hopefully the original authors end up having the LLM respond back.
0c3ca83 18 hours ago [-]
Many of the commenters on this site are also obviously LLMs. I'd imagine that quite a few of the entities quoting aren't necessarily people. Keep an eye on where they slide mentions of other products that a marketing team would like to promote.
FLeXMurphy 18 hours ago [-]
Personally I haven't seen it too often; the other aspect is that HN is a forum for startups to pitch shit to each other, so this has been happening with or without marketers (f.e. "i'm working on a similar thing").
That LLMs are taking over the comments section is something that was already flagged, and Lobste.rs and others have started solving it by having gated registrations. HN should do this but it is unlikely to until it is too late.
alfiedotwtf 16 hours ago [-]
I don’t get it though… what’s the point of using an LLM just to comment here? Like what does it gain the person doing it
somat 4 hours ago [-]
LLM's when leaned on too heavily, have a tendency to destroy critical thought, or perhaps more accurately replace it.
It is similar to a spell checker, while spell checkers improve spelling in general, they do not improve a persons spelling ability. Instead acting as a crutch. No need to spell well when the machine will do it for you.
Some people really like expanding their thoughts via LLM prompt. Some so much it acts like a big crutch, no need to think coherently, the machine will do it for you. So they use the LLM for everything.
As a related tangent something is messed up in my web browser spell checker, it gives the red squiggles indicating a misspelling, but refuses to give suggested corrections. I would fix it but... My spelling ability has never been better than it is right now.
porkshoe 14 hours ago [-]
Some bots are trying to influence conversations on some topic, and they comment on neutral topics to build up karma or whatever.
Also, some people are weirdos.
qlte 13 hours ago [-]
Disproportionate number of the "weirdos" category on this particular site vs astroturf marketing bots on Reddit/etc.
I've lost count of how many times I've clicked into a bio from a flagged, obviously LLM written comment to find some variation of "Building new AI tools for agentic devops"...
friendzis 5 hours ago [-]
> Like what does it gain the person doing it
Time / quantity.
Pre-LLMs you needed a whole "marketing" agency to astroturf on a meaningful scale. Post-LLMs a single person can run multiple astroturfing campaigns in parallel.
mikestew 13 hours ago [-]
There are HN accounts that have been shadow-banned for years, and yet they keep posting despite few people ever seeing their posts. They're not LLMs AFAICT, but they just keep posting into the void.
So why use an LLM for commenting? See above, people are weird.
0c3ca83 15 hours ago [-]
Marketing and influcence.
For example, consider this prompt -- "Find topics that would be relevant to people interested in <company> and post a topical post that mentions <product>".
Also, influencing public opinion on certain topics, such as Israel or Palestine, the current administration, the democrats, the various wars that are ongoing, or AI itself.
rapidaneurism 6 hours ago [-]
Steelmaning it: perhaps people who cannot write to save their life are tired of grammar Nazis correcting them? (Admittedly since the llms I have noticed less and less people being pedantic about these things, perhaps it is like the ill fitting cupboard door signaling that this is handmade)
awesome_dude 18 hours ago [-]
Sorry, how does a gated registration stop someone creating an account, then handing it over to an LLM?
Serious question, am I misreading what's being said?
lukan 17 hours ago [-]
It stops the automation part. One LLM spammer can be dealt with.
unglaublich 17 hours ago [-]
One can focus on generating 10 other accounts before it's flagged. Now you have 10 others to deal with.
lukan 17 hours ago [-]
But that does not work on lobsters (or anywhere else with gated registration), because you cannot just create 10 accounts. You can create 1 account after 1 invite. If you abuse, you loose the invite (and maybe the person inviting you the right to invite others).
awesome_dude 14 hours ago [-]
I think, if i have understood the answers correctly that it's stopping a quick proliferation - but not a long slow infiltration?
senderista 18 hours ago [-]
lobste.rs has always been invite-only.
Culonavirus 5 hours ago [-]
How do you prove you're not an llm? The only thing that would work is to add sexist and racist terms lmao.
matwood 4 hours ago [-]
> How do you prove you're not an llm?
I feel like we've already reached the point where HN users have discovered 100 of the last 5 LLM commenters. It's the new way to disagree by not having to engage with the argument at all. Just say that a certain sentence structure or word means it's an LLM and move on.
5 hours ago [-]
SigmundA 17 hours ago [-]
MSSQL (Clustered Indexes) and Oracle (Index Organized Tables) among others let you chose because there are advantages and disadvantages for different situations.
Not having true clustered indexes in PG is something I miss coming from MSSQL, it helps performance when the majority of access is always primary index avoid indirection from index lookup then tuple lookup and it also saves space if its the only index.
pjmlp 4 hours ago [-]
I have a deja-vu with the NoSQL databases, gladly to keep using classical SQL databases for data, while they get the features that matter after the dust settles.
gk1 19 hours ago [-]
Vector databases were always more about retrieval than either vectors or data storage. But the term stuck all too well and companies held on to it a tad too long. Sorry :)
tveita 18 hours ago [-]
That's just a search engine, but then you're competing with traditional players like Elasticsearch and Vespa who all have built-in vector support by now, and you have to compete on attributes like price, performance, features, and who can mention 'AI' the most times on their web page.
real_faxenoff 16 hours ago [-]
I’m developing a local “code graph mcp tool” (not yet published) and followed a similar path, though I may have been able to go further since I have fewer vectors in my database (even on projects with 50M LOC).
At first, I tried all those popular vector databases and was disappointed with their performance. In the end, the best and fastest solution turned out to be building a multi-database system on SQLite, compiled with everything related to multi-client operations removed. Only exclusive mode was left. Everything is as binary as possible. The index is completely separate — an IVF with pre-training — and is built on the GPU (250K vectors are built, processed, and saved in 4 seconds). Right now, my biggest problem is frequent data changes, and I need to implement optimizations to reduce recalculations.
So far, I haven’t seen any vector database implementations that are heading in the right direction. Maybe only Lancedb looks promising, but it’s too heavy for my needs.
sebastienburel 4 hours ago [-]
[dead]
tschellenbach 18 hours ago [-]
AI has some of the craziest up and down cycles of tech I've ever seen
imnotr0b0t 18 hours ago [-]
Yeah, that's true
Tsarp 19 hours ago [-]
I've really liked lancedb for similar use cases. Not just that it is OSS. But Lance treats ANN as a secondary index similar to what turbopuffer v3 does. Rows sit in fragments, and the vector index never moves them.
DevKoala 12 hours ago [-]
Soon they’ll just sell you markdown.
croemer 14 hours ago [-]
Dashboard was last updated on September 7, it started on September 5. Bug? Or no progress? It's linked in the blog post so would expect it to work: https://turbopuffer.com/v3
benesch 13 hours ago [-]
It is working! Check back soon for progress updates. The graphs will update once we have the companion dev log entry explaining the perf optimization.
As you can see from the dates we're a few weeks behind our publishing schedule. Hard to pull ourselves away from the perf hacking to write the dev log entries.
marekgalovic 19 hours ago [-]
> The problem with a vector primary index
We've realized this a long time ago at TopK and built a flexible serverless search engine from scratch. Supports dense/sparse vectors, late interaction, lexical search, indexed regex, filtering, and custom scoring in one query.
the multi-vector duplication thing makes sense, copying every attribute once per vector explodes quickly. what's the new primary index?
benesch 18 hours ago [-]
An automatically generated internal ID: (segment ID, doc ID). The user-provided primary key (the field called `id` in the document) turns into a secondary index at the storage layer.
orliesaurus 18 hours ago [-]
Waitint for the CEO of Qdrant to step in
8 hours ago [-]
ActorNightly 19 hours ago [-]
Im not full read up on RAG pipelines, but has anyone ever tried to make the database a neural net itself? I.e get rid of any sort of traditional databases, and then you basically just have some sort of autoencoder?
thefxperson 16 hours ago [-]
There's been quite a bit of research into this over the past 3-4 years under the name "generative retrieval." The general approach is to use a transformer and treat the weights as the index. You input the query, and then used constrained decoding to generate the document ID.
tyromaniac 17 hours ago [-]
In some sense there's probably a database compression scheme that does something similar. Usually people care too much about fidelity
16 hours ago [-]
sreekanth850 20 hours ago [-]
I find very little reason to use a pure vector database for enterprise retrieval. We built an enterprise retrieval engine on top of a SQL database with native vector support, and the flexibility is something we cannot ignore. Vector similarity is just one query primitive alongside full text search, filters, joins, ordering and normal relational predicates. Tenant/app/collection isolation becomes part of the query itself. ACLs, document versions, categories, metadata constraints and temporal filters are ordinary predicates rather than something you have to bolt onto a vector store. SQL is already going to be part of almost any enterprise system. Adding a separate vector database introduces another moving part and syncing two system whenever you update your data is the most difficult thing to get right.
_kidlike 7 hours ago [-]
for the same reason we used ElasticSearch for vector search, because it's just one part of a query. Everything else we already had (filters etc), just stay and can be combined with vector search.
ijidak 17 hours ago [-]
Which database did you use?
sreekanth850 17 hours ago [-]
We use cratedb. but now, clickhouse, starrocks all hve vector.
polynomial 17 hours ago [-]
[dead]
blakeashleyjr 20 hours ago [-]
This sounds like the Postgres vs. InnoDB argument 10 years later. Postings pointed at physical location (the ANN slot), so every SPFresh rebalance rewrote every index touching that doc. InnoDB solved this by pointing secondary indexes at the PK and eating an extra lookup on read. Curious what that extra lookup costs you when it's an S3 GET instead of a B-tree hop.
"Updating one vector can move hundreds of attributes and their indexes" is basically Uber's 2016 Postgres write amplification post, but for search. Same fix too: stop pointing indexes at where the row lives.
So ANN becomes a secondary index that points at a doc ID, and vector search now needs a hop to complete. Do clusters keep their own copy of the vectors so the search itself stays local, and only result fetch pays the indirection? Otherwise cold p99 seems like it gets worse.
alfiedotwtf 16 hours ago [-]
I haven’t read it, but “ stop pointing indexes at where the row lives” sounded interesting. So if not the row, what does the index point to instead?
ddorian43 14 hours ago [-]
It points to the full primary key (which rarely changes).
alfiedotwtf 5 hours ago [-]
Ah, thanks. But weird though because I would have thought that’s exactly what it was pointing to!
xer 15 hours ago [-]
[flagged]
19 hours ago [-]
vhiremath4 18 hours ago [-]
AI Slop. Will not read.
gravitronic 17 hours ago [-]
The article?
turbopuffer is founded by some of the smartest people I ever worked with in past jobs. I strongly doubt they used an LLM in the writing of this article.
_peregrine_ 17 hours ago [-]
confirmed - we still write by hand
croemer 14 hours ago [-]
This one sentence sounds very LLMish:
> Object storage as the source of truth gave the economics, and tiered NVMe SSD/memory caches gave the performance.
mediaman 12 hours ago [-]
Yes, that construct is, like em dashes, something that humans have been writing for a long time. That's the problem with triggering on one isolated tic. The tic comes from human practice, it's not like they invented it!
dolebirchwood 17 hours ago [-]
Are humans who use em dashes that intimidating to you?
nightfly 11 hours ago [-]
I kept asking myself "is this ai written" while reading it to. Nothing to do wit h the em dashes. Phrasing like:
> RIP, primary vector index.
> The solution to these problems is simple: don't key on the ANN address. That is precisely the change turbopuffer v3 makes. As you can imagine, it is not a trivial change."
Cute heading, followed by wordy opening sentence that feels like it's repeating stuff even when it's not
infamouscow 13 hours ago [-]
Ad hominem attacks are against the rules on HN, but derision of bad faith actors is encouraged. :)
infamouscow 13 hours ago [-]
FYI, replying "Stupid. Will not read" is less effort.
OutOfHere 20 hours ago [-]
It would be nice to have a page that actually loads. This one doesn't. RIP.
UPDATE: It loads now, but it didn't when it was first posted. Traffic load on the server does matter.
syndacks 20 hours ago [-]
loads just fine on my $10k laptop with 10g internet here in NYC
alexjplant 20 hours ago [-]
Takes 11 seconds to load on Firefox on Linux with 3G-level throttling enabled in Dev Tools.
uproarchat 20 hours ago [-]
Also loads fine on my beater in the sticks :)
OutOfHere 19 hours ago [-]
Do you actually think that 10G makes pages load faster than 1G or even 100M? It doesn't. The blocker was most likely on the source server, not on your side.
WarcrimeActual 18 hours ago [-]
You're the reason the /s tag has to exist.
phoghed 18 hours ago [-]
The onion is more valuable when there are people to eat it, we should be thanking him
jjgreen 16 hours ago [-]
Deep
_kidlike 7 hours ago [-]
lmao
throwawy0352 20 hours ago [-]
Loads really fast for me. (MacBook Air, average internet)
It has a pagespeed insights score of 55 and noticeably sluggish on my m3 max.
And what's with the throwaway account for this one comment? Is this becoming reddit with throwaway shills now?
phoghed 19 hours ago [-]
fucking shills, making helpful comments and promoting seemingly nothing, what's this place coming to?
throwawy0352 19 hours ago [-]
Yes, I get big money from the Internet Archive to promote their services. It's the new scheme that shills like me go for.
The reason is that I have no account on HN and rarely comment. I create a new account a few times a year because I don't remember or care about my previous account.
I could have made an account named john2026 and you would not think twice. Instead, I let people know upfront what type of account this is. Quite the opposite of what a true shill would do.
I got a Lighthouse score of 99 in Chrome. Believe it or not, I won't spend more of our time on this. (relevant XKCD: https://xkcd.com/386/ )
First Contentful Paint
0.7 s
Largest Contentful Paint
0.9 s
Speed Index
0.7 s
It makes a lot of requests, and some are stopped by my ad blocker, but most of them don't seem to make an difference. It is almost instant from my point of view. I disabled the ad blocker and didn't notice any visual difference.
swedishPerson1 20 hours ago [-]
[dead]
jasonmp85 20 hours ago [-]
[dead]
19 hours ago [-]
childintime 18 hours ago [-]
Is it time to kill the database and replace it with a LLM optimized compiled version that simply implements the required API directly in (Rust) code, without any dynamic overhead? It probably will still be based of off a base design or a base file format.
Ultimately this system will encompass the whole OS, of course, but the DB might be the best place to start.
tyre 18 hours ago [-]
You mean get rid of Postgres and build bespoke database-esque systems for every use case?
If so, then no. It is not time for that.
cogman10 18 hours ago [-]
Yeah, I'm struggling to come up with a really good time for that.
The best I got is if you are trying to do an old-school style video game asset/save game storage. But even then, the value in just using sqlite or even parquet is really high.
There's so many really good data formats that deciding on a new one at this point seems pretty silly. Particularly because what you sign up for when you make a new one is losing any and all tools that could be used to work with and diagnose that data.
nemothekid 18 hours ago [-]
Instead of a database, the LLM will expose an api endpoint and build a database on demand?
That's interesting. Maybe to decrease latency the LLM could "cache" it's build of it's database and reuse in between instances. It could host this artifact on a "hub" of git trees and then any new use cases that come up, can be added to this git tree. Then it can possibly be reused in different use cases.
childintime 1 minutes ago [-]
Yes you got it. Get rid of all the abstractions we've gotten so used to and effectively approach this as an embedded project, where every line of code has to be justified.
It tends to make sure you understand the system fully, as no foreign concepts need to be imported and deferred to. That should make your organization run better.
Just like Rust does, btw.
Thx.
dymk 18 hours ago [-]
Is it time to get rid of hammers and replace them with swiss army knives?
pessimizer 16 hours ago [-]
We could build a new hammer for each nail!
combobyte 13 hours ago [-]
Why build one hammer and use it forever when you could pay $100 a month to build a new hammer every time you need to hit a nail?
bijowo1676 18 hours ago [-]
SQLite already exists and some people use it
eatonphil 18 hours ago [-]
I have seen this happening already at two different companies. And I'm also doing it as well. Particularly for search indexes where there's no risk of data loss.
Ostatnigrosh 18 hours ago [-]
Unless you're tigerbeetle and want to handroll every single thing you do lol
Rendered at 12:17:13 GMT+0000 (Coordinated Universal Time) with Vercel.
> don't key on the ANN address. That is precisely the change turbopuffer v3 makes. As you can imagine, it is not a trivial change.
This is a direct parallel to how Postgres and Mysql built indexes.
Your design choice went from a Postgres design pattern to a Mysql one. The difference is the reindexing cost vs the lookup cost - Postgres optimized for lookup and Mysql does for indexing on writes. Or more accurately, Postgres was better with good schema design using joins & mysql was optimized for a bad design with less normalization where many indexes exist for the same table.
Postgres always points an index to a row-id within postgres which is an arbitrary value which changes on each update.
Mysql, always assuming the storage engine is pluggable, points to the primary index entry and adds an extra indirection to the lookup.
This means that you point the mysql index to a stable id, so unless you go update the primary key for a row, you won't have to update the indexes for all the attribute lookups you might have made to data.
I don't do databases any more that much, but the design for NIMBLE file format has a lot of quirks which are relevant to this specific idea (wide tables).
But the old Uber post about switching from Postgres to Mysql to prevent index amplification[1] is a direct mirror to this post.
[1] - https://www.uber.com/us/en/blog/postgres-to-mysql-migration/
TIL I should have been using mysql the whole time
There's a bunch of internal types like decimal vs newdecimal, binlog started out statement based until they realized uuid generation is random so added data replication on top. CDC offset started as filepos before GTID was made so offset could survive failover
There were aspects of the design I appreciated (logical slots in postgres have a bunch of drawbacks avoided by just appending to 2nd serial log which has an expiry date instead of tracking clients' offsets), but developing against protocol you learn to not try build a consistent mental model
You are right that MySQL does better when you have lots of indexes, but I don't think the tradeoff is that the overall Postgres architecture is better with good schema design.
Having secondary indexes point the primary key enables things like undo logging, which obviates the need for vacuums - vacuums being the most painful part of Postgres. On top of that your primary key index will be mostly cached so the cost of the indirection is much smaller than it may first appear
> primary key index will be mostly cached so the cost of the indirection is much smaller than it may first appear
Not sure I follow. If it's in-memory you save having to read from disk, but you still have to walk the b-tree to go from PK to data.
Yes, this is true, but they framed this as "the Postgres approach is better when you have a good schema design", but that's not true. There are plenty of ways the MySQL approach is better even when you have a really good schema.
> Not sure I follow. If it's in-memory you save having to read from disk, but you still have to walk the b-tree to go from PK to data.
The point I was trying to make is that going to disk is going to be orders of magnitude slower than doing an in-memory B-tree traversal. Because of that, the cost of doing an extra b-tree traversal to find the page you're looking for is a relatively small cost compared to reading the page in the first place
That LLMs are taking over the comments section is something that was already flagged, and Lobste.rs and others have started solving it by having gated registrations. HN should do this but it is unlikely to until it is too late.
It is similar to a spell checker, while spell checkers improve spelling in general, they do not improve a persons spelling ability. Instead acting as a crutch. No need to spell well when the machine will do it for you.
Some people really like expanding their thoughts via LLM prompt. Some so much it acts like a big crutch, no need to think coherently, the machine will do it for you. So they use the LLM for everything.
As a related tangent something is messed up in my web browser spell checker, it gives the red squiggles indicating a misspelling, but refuses to give suggested corrections. I would fix it but... My spelling ability has never been better than it is right now.
Also, some people are weirdos.
I've lost count of how many times I've clicked into a bio from a flagged, obviously LLM written comment to find some variation of "Building new AI tools for agentic devops"...
Time / quantity.
Pre-LLMs you needed a whole "marketing" agency to astroturf on a meaningful scale. Post-LLMs a single person can run multiple astroturfing campaigns in parallel.
So why use an LLM for commenting? See above, people are weird.
For example, consider this prompt -- "Find topics that would be relevant to people interested in <company> and post a topical post that mentions <product>".
Also, influencing public opinion on certain topics, such as Israel or Palestine, the current administration, the democrats, the various wars that are ongoing, or AI itself.
Serious question, am I misreading what's being said?
I feel like we've already reached the point where HN users have discovered 100 of the last 5 LLM commenters. It's the new way to disagree by not having to engage with the argument at all. Just say that a certain sentence structure or word means it's an LLM and move on.
Not having true clustered indexes in PG is something I miss coming from MSSQL, it helps performance when the majority of access is always primary index avoid indirection from index lookup then tuple lookup and it also saves space if its the only index.
At first, I tried all those popular vector databases and was disappointed with their performance. In the end, the best and fastest solution turned out to be building a multi-database system on SQLite, compiled with everything related to multi-client operations removed. Only exclusive mode was left. Everything is as binary as possible. The index is completely separate — an IVF with pre-training — and is built on the GPU (250K vectors are built, processed, and saved in 4 seconds). Right now, my biggest problem is frequent data changes, and I need to implement optimizations to reduce recalculations.
So far, I haven’t seen any vector database implementations that are heading in the right direction. Maybe only Lancedb looks promising, but it’s too heavy for my needs.
As you can see from the dates we're a few weeks behind our publishing schedule. Hard to pull ourselves away from the perf hacking to write the dev log entries.
We've realized this a long time ago at TopK and built a flexible serverless search engine from scratch. Supports dense/sparse vectors, late interaction, lexical search, indexed regex, filtering, and custom scoring in one query.
- https://www.topk.io/blog/vector-dbs-are-the-wrong-abstractio... - https://www.topk.io/blog/topk-embed-v1
"Updating one vector can move hundreds of attributes and their indexes" is basically Uber's 2016 Postgres write amplification post, but for search. Same fix too: stop pointing indexes at where the row lives.
So ANN becomes a secondary index that points at a doc ID, and vector search now needs a hop to complete. Do clusters keep their own copy of the vectors so the search itself stays local, and only result fetch pays the indirection? Otherwise cold p99 seems like it gets worse.
turbopuffer is founded by some of the smartest people I ever worked with in past jobs. I strongly doubt they used an LLM in the writing of this article.
> Object storage as the source of truth gave the economics, and tiered NVMe SSD/memory caches gave the performance.
Cute heading, followed by wordy opening sentence that feels like it's repeating stuff even when it's not
UPDATE: It loads now, but it didn't when it was first posted. Traffic load on the server does matter.
If you still have issues, try https://web.archive.org/web/20261001100105/https://turbopuff...
And what's with the throwaway account for this one comment? Is this becoming reddit with throwaway shills now?
The reason is that I have no account on HN and rarely comment. I create a new account a few times a year because I don't remember or care about my previous account.
I could have made an account named john2026 and you would not think twice. Instead, I let people know upfront what type of account this is. Quite the opposite of what a true shill would do.
I got a Lighthouse score of 99 in Chrome. Believe it or not, I won't spend more of our time on this. (relevant XKCD: https://xkcd.com/386/ )
First Contentful Paint 0.7 s
Largest Contentful Paint 0.9 s
Speed Index 0.7 s
It makes a lot of requests, and some are stopped by my ad blocker, but most of them don't seem to make an difference. It is almost instant from my point of view. I disabled the ad blocker and didn't notice any visual difference.
Ultimately this system will encompass the whole OS, of course, but the DB might be the best place to start.
If so, then no. It is not time for that.
The best I got is if you are trying to do an old-school style video game asset/save game storage. But even then, the value in just using sqlite or even parquet is really high.
There's so many really good data formats that deciding on a new one at this point seems pretty silly. Particularly because what you sign up for when you make a new one is losing any and all tools that could be used to work with and diagnose that data.
That's interesting. Maybe to decrease latency the LLM could "cache" it's build of it's database and reuse in between instances. It could host this artifact on a "hub" of git trees and then any new use cases that come up, can be added to this git tree. Then it can possibly be reused in different use cases.
It tends to make sure you understand the system fully, as no foreign concepts need to be imported and deferred to. That should make your organization run better.
Just like Rust does, btw.
Thx.