Limit of number of sentences
#78
by
bantonybow
- opened
Hi, is there a limit to the number of sentences one can compare the source to?
Hello!
There isn't. You get embeddings directly from the model, and then you can save up as many as you want to simultaneously compare with cosine similarity. The only constraint is your memory, but you can get around that by e.g. writing to disk or using some sort of vector database. But you can keep tons of vectors in memory at once.
Or perhaps you mean the Inference API. In that case, there's probably eventually a limit due to memory issues. The API expects that everything fits in memory at once.
- Tom Aarsen