-
@ David
2025-06-12 08:10:21
you could also segment the data set and run multiple queries on it, at a lower cost
i mean, these things need a lot of power to run, most people think it's magic but it's way more expensive than a human expert
human expert needs equal to a couple pints of diesel per day and has a memory system that is holographic so it can be constantly updated
a full text index only costs about 1/8th extra data in the database to implement.
what would probably be the optimal solution is to use a general purpose model to sift through full text search results and highlight the results it considers to be most relevant (as in, semantics, grammar, order)