I tried TQ for vector search and my findings is not good, it is not worth it if you cannot use GPU, however I got same quality of search as 32f using 8bit quant
I wrote ann ext for sqlite, using tq, I do save a lot on space but 32f is still faster despite everything I have tried
So i assumed it would get crushed by OPQ (which requires training)
teamchong 20 hours ago [-]
you’re right that 32f is faster on raw query time, quantization adds extra step. main benefit on download size since gzip won’t help much, which matters most in browser contexts
glohbalrob 1 days ago [-]
Very cool. I added the new multi embedding 2 model to my site the other week from google
I guess need to dig into this and see if it’s faster and has more use cases! Thanks for publishing your work
hhthrowaway1230 1 days ago [-]
Awesome! Also love the gaussian splat demo, cool use case!
refulgentis 22 hours ago [-]
Sloppiest slop I've seen in a couple weeks:
- fork of a fork of a quantization technique
- Only contribution is...compiling JS to WASM by default?
- suspicious burst of ~nothing comments from new accounts
- 6 comments 7 hours in, 4 flagged/dead, other 2 also spammy, confused and making category errors at best, at worst, more spam.
- Demo shows it's worse: 800 ms instead of 2.6 ms for text embedding search
- "but it saves space" - yes! 1.2 MB in RAM instead of 7.2 MB to turn search into 1s on a MacBook Pro M4 Max, instead of sub-frame duration.
- It's not even wrong to do this with the output embeddings, there's way more obvious ways to save space that don’t affect retrieval time this much
himmelsee2018 1 days ago [-]
[flagged]
bingbong06 1 days ago [-]
[flagged]
newbrowseruser 1 days ago [-]
[dead]
aritzdf 1 days ago [-]
[flagged]
Rendered at 20:10:38 GMT+0000 (Coordinated Universal Time) with Vercel.
I wrote ann ext for sqlite, using tq, I do save a lot on space but 32f is still faster despite everything I have tried
code here https://github.com/netdur/munind/tree/main/src/tq
I guess need to dig into this and see if it’s faster and has more use cases! Thanks for publishing your work
- fork of a fork of a quantization technique
- Only contribution is...compiling JS to WASM by default?
- suspicious burst of ~nothing comments from new accounts
- 6 comments 7 hours in, 4 flagged/dead, other 2 also spammy, confused and making category errors at best, at worst, more spam.
- Demo shows it's worse: 800 ms instead of 2.6 ms for text embedding search
- "but it saves space" - yes! 1.2 MB in RAM instead of 7.2 MB to turn search into 1s on a MacBook Pro M4 Max, instead of sub-frame duration.
- It's not even wrong to do this with the output embeddings, there's way more obvious ways to save space that don’t affect retrieval time this much