Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://issues.apache.org/jira/browse/SOLR-18466
Description
The
search()function in streaming expressions currently ignores_route_for shard selection and queries every active slice of the target collection. Users with known routing keys therefore incur unnecessary request and response overhead, which can be significant on large clusters. This change adds support for_route_param in the streaming expression search function, reusingdocCol.getRouter().getSearchSlices()which is used for shard selection in normal distributed search.Solution
Two files in
solr/solrj-streaming:the
CloudSolrStreamclass:the existing method:
is kept for existing calls from
TopicStream,TextLogitStream,FeaturesSelectionStream, andStatementImpl(all out of scope for this change); these are unaffected and continue to receive all active slicesA new overloaded helper method is added:
When
solrParamscontains a nonempty_route_, each collection's slices are resolved viadocCol.getRouter().getSearchSlices(routeKeys, params, docCol)and when_route_is absent (or empty), all active slices are returned exactly as before.the
TupleStreamclass:CloudSolrStream.getSlices(collection, cloudSolrClient, true, solrParams)is called after the request params are assembled and passed to the route awaregetSlices.Example usages:
One note on backward compatibility:
Previously,
_route_was ignored during shard selection for streaming expressionsearch(). With this change, nonempty route values are passed todocCol.getRouter().getSearchSlices(routeKeys, params, docCol). For collections using the implicit router, an invalid_route_value now causes the request to fail. Previously, the same request searched all active shards despite the invalid_route_. This should be documented in the release/ upgrade notes.Tests
The following four tests cover single and multiple route keys, composite id routing, missing and empty route values, and invalid implicit router shard names.
Checklist
Please review the following and check all that apply:
mainbranch../gradlew check.