Face Matching
MTCNN detect face
Extract features
Indexing
Explain different types of INDEXes of FAISS How does the memory and speed compromise happen here. Are there any other things that we can affect the performance of the index.
Let's go with FLAT index for first implementation. and HNSW implementation since these are fast for small dbs.
The memory requirement of flat INDEX grows linearly.
Searching
How search can be performed on FAISS server. And the different metrics that can be used for comparing
1
Security Aspects
Right now we are storing the meta data and the FAISS indices in the db. The db doesn't have any kind of protection since its just a sqlite implementation. This needs to be configured
Things that affect the performance
Faiss search can happen either by getting the k-nearest neighbours or it can happen by fetching all the vectors within a range
The faiss index.search requires k-nearest neighbours to be passed
while performing the search, I do understand that giving the required number of neighbours can speed up the process, since when the required number match is reached, the search can return early. However our use case seems to expect almost all records that fall on this scoring range as that gives us more control. This also seems to suggest that we should go with a grouping approach since we need to go through all records anyway as our goal is to filter by threshold.
Embedding dimension
Right now the InceptionResnet with vggface2 model returns a tensor of 512 dimensions. We should experiment with this to decide if there is a need for dimension reduction. Since nobody wants the Curse of dimensionality.