Configuring SOLR for MoreLikeThis functionality

As we transition to the new Hyland Community, some post links may be temporarily disrupted as we finalize redirects. We apologize for the inconvenience and are working to resolve this as quickly as possible. Details.

cszamudio

Hi,

I've been trying to generate "More Like This" results from searches using SOLR and have not been successful in getting any results.
I assume I need to reference the TEXT field similar to the Lucene search expression.

I'm testing this using a URL like the following, using the default mlt parameters:

https://localhost:8493/solr/alfresco/afts?q=patent&mlt=true&mlt.count=10&mlt.fl=TEXT&rows=10

My repository is full of related patents, so I know there should be related entries.

I get the appropriate hits back from the search in the JSON response, but nothing in the moreLikeThis portion of the JSON response, e.g.,

"LEAF-4934":{"numFound":0,"start":0,"docs":[]} for each hit.

I can't find any reference to setting up a MoreLikeThisHandler for SOLR in Alfresco, but the SOLR documentation seems to indicate this needs to be set up. I've noticed that the Alfresco solr configuration of the mlt component specified in the SearchHandler so perhaps setting up a new Handler is not necessary.

<requestHandler name="/afts" class="solr.SearchHandler" >
    <lst name="defaults">
     <str name="defType">afts</str>
    </lst>
    <arr name="components">
      <str>setLocale</str>
      <str>query</str>
      <str>facet</str>
      <str>mlt</str>
      <str>highlight</str>
      <str>stats</str>
      <str>debug</str>
      <str>clearLocale</str>
    </arr>
  </requestHandler>
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Has anyone had success generating MoreLikeThis results from a search?

Thanks,
Carlos S. Zamudio

afaust

Hello,

I have recently worked to get the MoreLikeThis feature running in Alfresco for one of our commercial modules. In short, you can forget about using this feature without adding some additional code into Alfresco SOLR <b>and</b> modifying Alfresco base code. Due to the way that Alfresco builds its index, there is no basis for the MoreLikeThisComponent to work with in an out-of-the-box install.

1) Alfresco does not include term vectors in their index fields for most of the really important elements, such as content, properties and types/aspects. It also does not store the actual value (which would blow up the size of the index) of a field. The MoreLikeThisComponent needs either of these two to build a similarity query for a document / search result.

2) Alfresco stores multiple index documents for the same piece of content. There is a main document (LEAF-*) with core information about the type/aspect, metadata and content, as well as multiple secondary documents (AUX-*) for PATH information. Without a custom MoreLikeThisComponent that merges data from both types of documents into a single query, you will not be able to include PATH in the calculation of MoreLikeThis results.

3) The standard MoreLikeThisComponent of SOLR does not respect any filter queries. Alfresco bases its tenant / permission check on filter queries. Without a custom MoreLikeThisComponent that includes these filter queries into the evaluation of MoreLikeThis results, you may end up exposing the existence of sensitive documents to an end user that should not see it.


Short: It is do-able to get Alfresco SOLR to include MoreLikeThis results, but it requires very low-level modifications to Alfresco SOLR. It is definitely not possible to enable this feature with configuration alone.

Regards
Axel