Lucene Search Not Compatible with DXP - Why?

postjoe

Vote:

DXP, I am told, does not support Lucene for search, I am looking for more details on the "why". I am looking for technical/developer documentation.

#260063

Aug 04, 2021 14:15

Scott Reed

Vote:

When you purchase the DXP platform you get Search & Navigation (Find) as part of it which uses Elasticsearch with a far better .NET wrapper and far better admin/editor management feature set. Lucene also can be tricky with scaling instances due to the way it writes the lucene data so S&N/Elasticsearch is far better option.

With search and navigation you get boost, set synnoynms, related queries and auto complete all out of the box. It also has tracking and if using commerce support direct built in personalization for commerce searching.

#260068

Edited, Aug 04, 2021 15:21

postjoe - Aug 04, 2021 16:02

Thanks for the comment. I am looking for why it is not supported. In your response, you say "tricky with scaling instances." Can you elaborate? Still hoping someone has some technical insight into why it is not supported.

Scott Reed - Aug 04, 2021 16:13

When the lucence search creates it's index it create the index as files on the file system usually under the AppData folder of the files that are indexed. When I used to use this mechanism on load balanced servers it would cause issues if we let each web server create its only version with sometimes different search results so we used to make the CMS server be the primary indexer and then the web instances opened up to call the index service so there was one. I guess this could be configured maybe use blob storage if that would work.

Basic lucence isnt scalable whereas Elastic also based on lucence is and can scale out to meet load so as part of the DXP when they created it they created search and navigation (find) to support the enterprise. Therefore as it's designed for DXP and included in the package there's really no need to support the old, inferior system.

postjoe - Aug 04, 2021 17:12

I am told there may be some issues with DXP and Lucene due to Windows services that are not available in the DXP?

David Knipe - Aug 04, 2021 17:26

Correct there are no Windows services in DXP, it uses Azure App Service under the hood so no Windows Services obviously :)

Is there a specific problem that you want to solve using Lucene in DXP?

postjoe - Aug 04, 2021 17:28

Lucene is not supported in DXP. I am trying to get a complete answer on why that is.

David Knipe - Aug 05, 2021 9:01

It comes down to scalability, development and PaaS vs SaaS considerations. Lucene search was used by the now legacy Episerver Search, It's not been in active development for a long time and ran as part of the PaaS implementation. As Scott has mentioned Search and Nav runs on Elastic, you get it as part of the DXP Service, is scalable and runs as a SaaS service removing complexity and the need for you to run and support services yourself. So it's an architectural choice where for search SaaS services are favoured over PaaS.

Paul McGann (Netcel)

Vote:

@postjoe, I think the reason why it is not supported is due to the fact that Lucence is a filebased search solution. Taking this into account you can then see why this is not a good option when working with the cloud. The lucence index would need to be stored upon the web app, which is not a good practice when working with cloud solutions.

Scott and David have already mentioned that DXP comes with 'Search and Navigation' which has many benefits. Having worked with Lucence in the past and now mostly focused on search and navigation I would never go back. The ease of using find is one of the selling points for using the product. You will not loose functionality by switching from Lucence to search and navigation and will instead gain more control over your ability to perform searching.

To give you a better answer as to 'why'? We would need more information around 'why not' switch to Search and Navigation?

#260348

Aug 09, 2021 14:57

postjoe - Aug 09, 2021 15:11

My question was abstract in the sense I just wanted to understand the why. But the IRL scenario was a migration from on-prem to the DXP where re-building search with Opti Search & Nav was not in scope of the migration to the cloud, but it was planned in the roadmap.

Paul McGann (Netcel) - Aug 09, 2021 15:52

While I am not saying that it isn't possible to host the index in DXP, I would say that the amount of effort that you need to make it work will most likely out way the effort involved in switching to search and navigation. You could spend 1 month getting it to work vs maybe only a 1-2 weeks build the search and navigation. Generally all that would change would be your business logic of retrieving results. There is a library on GitHub that allow you to index more fields in lucence here https://github.com/mychien1995/EpiServer-Dynamic-Lucene-Extensions so that assumes you can do it. As we all know just because you can, doesn't mean you should.

Scott Reed - Aug 10, 2021 8:14

Paul as David mentioned the Lucene index implementation is dependant on Windows Services so that puts it out of the picture for the DXP.

Postjoe for your IRL scenario when you're looking at a migration to DXP moving to Search and Navigation is a must have I'm afraid (unless you want to use a different cloud search provider, but why would you with the index included). I migrated a fairly big global site last year and unless you're doing REALLY COMPLEX things it's not actually that difficult. Most of the things for S&N are set up out of the box and the .NET wrapper makes it really easy to do. I would raise to your product manager or whomever is in charge of the DXP migration that the S&N features HAVE to be in scope for the migration.

Paul McGann (Netcel) - Aug 10, 2021 8:25

Thanks Scott for correcting me.

Quan Mai

Vote:

Technically? Not really. But as many mentioned, DXP subscription bundled with Find (Search and Navigation) which is simply superior, and you can migrate very easy to. We of course want to upsell our products and provide the best experiences/services to our customers, so there is no point of bringing Search to DXP.

#260405

Aug 10, 2021 13:04