Try our conversational search powered by Generative AI!

Lower the weight of common terms



We're creating a FAQ page, it contains a content area where we drop 'QuestionAndAnswer' blocks, that just have two properties: Question (string) and Answer (XhtmlString)

Also, the FAQ page contains an input so users can do a free text search.

The problem is that we'd like to lower the weight of common terms such as "what is" or "how does" while performing the search, i.e for "What is technology" search we'd like to get all the question blocks that contains "technology" first, instead, now seems we're getting some other questions first because of the matching of the 'what is' part. 

My first approach was to add a IList<string> Keywords property to the QuestionAndAnswer block, and then giving it more weight that the others

search = search.For(query.FreeTextSearch).InField(x => ((IQuestionAndAnswerBlock)x).Keywords, 2)
.InField(x => ((IQuestionAndAnswerBlock)x).Question, 0.01)                    
.InField(x => ((IQuestionAndAnswerBlock)x).Answer, 0.001)

 I was wondering if there's a better approach for this, maybe there's a way to get this done without having to add the keywords property to the blocks? 

Find version:

Edited, Feb 12, 2021 19:48

They should naturally have a lower weight as they are more common (should be) across documents and index in general.

Check out Term Frequency and Inversed Document Frequency

I believe you could add a boost on invidiual terms as well but I am not sure this is the way you want to go.

what is^2
Feb 18, 2021 10:59
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.