-
Notifications
You must be signed in to change notification settings - Fork 864
SOLR-18457: Migrate V2 /update endpoints to using JAX-RS #4177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
epugh
wants to merge
41
commits into
apache:main
Choose a base branch
from
epugh:copilot/migrate-update-api-to-jax-rs
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
41 commits
Select commit
Hold shift + click to select a range
7b437c7
Initial plan
Copilot 89fdf9d
Migrate UpdateAPI from @EndPoint to JAX-RS annotations
Copilot 863fbf1
Add NodeHealthAPITest2 integration test for V2 node health API
Copilot f5e49ea
Fix TestDistributedTracing to use proper SolrJ add() instead of broke…
Copilot 3b79fb8
Support /c/ path alias in JAX-RS routing for all V2 endpoints
Copilot 014880c
Fix NPE in UpdateAPI: call pre/postDecorateResponse when delegating t…
Copilot 3cc0acc
code warning cleanups
epugh e40d417
Lint
epugh 92d70a8
We didn't need this.
epugh 74bdf15
Properly route reqeusts by returning an empty getApis
epugh 6a675bf
current v2 doesnt support multiple commands
epugh 4d2cbc0
Relocate test to be in the same package as what we are testing.
epugh 7337062
Add indexing-with-v2-apis.adoc documenting V2 update API endpoints
Copilot b932cb2
Tweak formatting
epugh 055f200
Update solr/solr-ref-guide/modules/indexing-guide/pages/indexing-with…
epugh 296f33a
relocate via intellij went werid
epugh 090749a
Merge remote-tracking branch 'upstream/main' into copilot/migrate-upd…
epugh 793eef0
/bin really should be /javabin. Open door to other binary protocols.
epugh 9d87a15
Fixed this code, thanks copilot. Not committing the new tests that p…
epugh 3dab0ef
add changelog
epugh 3ed21dd
Restore the versions=true support that v1 equivalent has, and so did …
epugh 6455c2e
add versions to the docs for v2... trying not to get too verbose.
epugh 6b32a74
Our future is STC, not STCJ4
epugh cfb1d52
Now update parameters show up in annoations.
epugh d11876f
restore the /update end point being the root. will make v1 to v2 doc…
epugh 162511f
Rework test to focus on compat between V1 and V2
epugh d904bef
Strong enough typing enabled to use generate V2 api in Solr Admin
epugh ec090e1
Due to limitation in v2 architecture for content types, by swapping t…
epugh 7523441
fix the layout issue that has plagued this page where results showed …
epugh 8723661
pretty print the sample raw json command by just formatting it here.
epugh 9b00cce
No longer need v1 update
epugh 8971144
string formatting is hard.
epugh 37d6f22
better changelog
epugh 0f23c24
Collapse seperate v2 into v1, and add javabin and version response info
epugh c783adb
Reorder and refactor the content.
epugh ed6bc2c
Updates to the transforming page for v2, and a test to ensure it works
epugh 3a96567
Updated screenshot for v2 default api.
epugh 0337730
Revamp partial doc updates to use real techproducts data and schema.
epugh 8fc4bef
small tweak that reads better
epugh 1de44e5
tighten test
epugh acca0f3
Deal with fall out of adding nesting to techproducts
epugh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
7 changes: 7 additions & 0 deletions
7
changelog/unreleased/SOLR-18457-migrate-update-api-to-jax-rs.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| title: Migrate v2 update endpoints to JAX-RS and rename the Javabin endpoint to /update/javabin. Migrate Admin UI to using v2 endpoints, fixing UI issues in the document upload screen. Document screen only works with /upload endpoint, no longer can specify a custom endpoint. | ||
| type: changed | ||
| authors: | ||
| - name: Eric Pugh | ||
| links: | ||
| - name: SOLR-18457 | ||
| url: https://issues.apache.org/jira/browse/SOLR-18457 |
211 changes: 211 additions & 0 deletions
211
solr/api/src/java/org/apache/solr/client/api/endpoint/UpdateApi.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,211 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| package org.apache.solr.client.api.endpoint; | ||
|
|
||
| import static org.apache.solr.client.api.util.Constants.GENERIC_ENTITY_PROPERTY; | ||
| import static org.apache.solr.client.api.util.Constants.INDEX_PATH_PREFIX; | ||
|
|
||
| import io.swagger.v3.oas.annotations.Operation; | ||
| import io.swagger.v3.oas.annotations.Parameter; | ||
| import io.swagger.v3.oas.annotations.extensions.Extension; | ||
| import io.swagger.v3.oas.annotations.extensions.ExtensionProperty; | ||
| import io.swagger.v3.oas.annotations.parameters.RequestBody; | ||
| import jakarta.ws.rs.Consumes; | ||
| import jakarta.ws.rs.POST; | ||
| import jakarta.ws.rs.Path; | ||
| import jakarta.ws.rs.QueryParam; | ||
| import java.io.InputStream; | ||
| import org.apache.solr.client.api.model.UpdateResponse; | ||
| import org.apache.solr.client.api.util.StoreApiParameters; | ||
|
|
||
| /** V2 API definitions for indexing documents via the update handler. */ | ||
| @Path(INDEX_PATH_PREFIX + "/update") | ||
| public interface UpdateApi { | ||
|
|
||
| @POST | ||
| @Consumes({ | ||
| "application/json", | ||
| "text/json", | ||
| "application/xml", | ||
| "text/xml", | ||
| "application/csv", | ||
| "text/csv", | ||
| "application/javabin", | ||
| "application/cbor" | ||
| }) | ||
| @StoreApiParameters | ||
| @Operation( | ||
| summary = "Send updates using any supported content type", | ||
| tags = {"update"}) | ||
| UpdateResponse update( | ||
| @Parameter(description = "Commit the update immediately") @QueryParam("commit") | ||
| Boolean commit, | ||
| @Parameter(description = "Commit the update within this many milliseconds") | ||
| @QueryParam("commitWithin") | ||
| Integer commitWithin, | ||
| @Parameter(description = "Overwrite documents with the same unique key") | ||
| @QueryParam("overwrite") | ||
| Boolean overwrite, | ||
| @Parameter(description = "Perform a soft commit") @QueryParam("softCommit") | ||
| Boolean softCommit, | ||
| @Parameter(description = "Include assigned document versions in the response") | ||
| @QueryParam("versions") | ||
| Boolean versions, | ||
| @Parameter(required = true) | ||
| @RequestBody( | ||
| required = true, | ||
| description = "Update content in the format selected by the Content-Type header.", | ||
| extensions = | ||
| @Extension( | ||
| properties = { | ||
| @ExtensionProperty(name = GENERIC_ENTITY_PROPERTY, value = "true") | ||
| })) | ||
| InputStream requestBody) | ||
| throws Exception; | ||
|
|
||
| @POST | ||
| @Path("/json") | ||
| @Consumes({"application/json", "text/json"}) | ||
| @StoreApiParameters | ||
| @Operation( | ||
| summary = "Index documents in JSON format", | ||
| tags = {"update"}) | ||
| UpdateResponse updateJson( | ||
| @Parameter(description = "Commit the update immediately") @QueryParam("commit") | ||
| Boolean commit, | ||
| @Parameter(description = "Commit the update within this many milliseconds") | ||
| @QueryParam("commitWithin") | ||
| Integer commitWithin, | ||
| @Parameter(description = "Overwrite documents with the same unique key") | ||
| @QueryParam("overwrite") | ||
| Boolean overwrite, | ||
| @Parameter(description = "Perform a soft commit") @QueryParam("softCommit") | ||
| Boolean softCommit, | ||
| @Parameter(description = "Include assigned document versions in the response") | ||
| @QueryParam("versions") | ||
| Boolean versions, | ||
| @Parameter(required = true) | ||
| @RequestBody( | ||
| required = true, | ||
| description = "JSON update content.", | ||
| extensions = | ||
| @Extension( | ||
| properties = { | ||
| @ExtensionProperty(name = GENERIC_ENTITY_PROPERTY, value = "true") | ||
| })) | ||
| InputStream requestBody) | ||
| throws Exception; | ||
|
|
||
| @POST | ||
| @Path("/xml") | ||
| @Consumes({"application/xml", "text/xml"}) | ||
| @StoreApiParameters | ||
| @Operation( | ||
| summary = "Index documents in XML format", | ||
| tags = {"update"}) | ||
| UpdateResponse updateXml( | ||
| @Parameter(description = "Commit the update immediately") @QueryParam("commit") | ||
| Boolean commit, | ||
| @Parameter(description = "Commit the update within this many milliseconds") | ||
| @QueryParam("commitWithin") | ||
| Integer commitWithin, | ||
| @Parameter(description = "Overwrite documents with the same unique key") | ||
| @QueryParam("overwrite") | ||
| Boolean overwrite, | ||
| @Parameter(description = "Perform a soft commit") @QueryParam("softCommit") | ||
| Boolean softCommit, | ||
| @Parameter(description = "Include assigned document versions in the response") | ||
| @QueryParam("versions") | ||
| Boolean versions, | ||
| @Parameter(required = true) | ||
| @RequestBody( | ||
| required = true, | ||
| description = "XML update content.", | ||
| extensions = | ||
| @Extension( | ||
| properties = { | ||
| @ExtensionProperty(name = GENERIC_ENTITY_PROPERTY, value = "true") | ||
| })) | ||
| InputStream requestBody) | ||
| throws Exception; | ||
|
|
||
| @POST | ||
| @Path("/csv") | ||
| @Consumes({"application/csv", "text/csv"}) | ||
| @StoreApiParameters | ||
| @Operation( | ||
| summary = "Index documents in CSV format", | ||
| tags = {"update"}) | ||
| UpdateResponse updateCsv( | ||
| @Parameter(description = "Commit the update immediately") @QueryParam("commit") | ||
| Boolean commit, | ||
| @Parameter(description = "Commit the update within this many milliseconds") | ||
| @QueryParam("commitWithin") | ||
| Integer commitWithin, | ||
| @Parameter(description = "Overwrite documents with the same unique key") | ||
| @QueryParam("overwrite") | ||
| Boolean overwrite, | ||
| @Parameter(description = "Perform a soft commit") @QueryParam("softCommit") | ||
| Boolean softCommit, | ||
| @Parameter(description = "Include assigned document versions in the response") | ||
| @QueryParam("versions") | ||
| Boolean versions, | ||
| @Parameter(required = true) | ||
| @RequestBody( | ||
| required = true, | ||
| description = "CSV update content.", | ||
| extensions = | ||
| @Extension( | ||
| properties = { | ||
| @ExtensionProperty(name = GENERIC_ENTITY_PROPERTY, value = "true") | ||
| })) | ||
| InputStream requestBody) | ||
| throws Exception; | ||
|
|
||
| @POST | ||
| @Path("/javabin") | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. i debated leaving this on /update/bin to mimic more closing v1... |
||
| @Consumes("application/javabin") | ||
| @StoreApiParameters | ||
| @Operation( | ||
| summary = "Index documents in Javabin format", | ||
| tags = {"update"}) | ||
| UpdateResponse updateJavabin( | ||
| @Parameter(description = "Commit the update immediately") @QueryParam("commit") | ||
| Boolean commit, | ||
| @Parameter(description = "Commit the update within this many milliseconds") | ||
| @QueryParam("commitWithin") | ||
| Integer commitWithin, | ||
| @Parameter(description = "Overwrite documents with the same unique key") | ||
| @QueryParam("overwrite") | ||
| Boolean overwrite, | ||
| @Parameter(description = "Perform a soft commit") @QueryParam("softCommit") | ||
| Boolean softCommit, | ||
| @Parameter(description = "Include assigned document versions in the response") | ||
| @QueryParam("versions") | ||
| Boolean versions, | ||
| @Parameter(required = true) | ||
| @RequestBody( | ||
| required = true, | ||
| description = "Javabin update content.", | ||
| extensions = | ||
| @Extension( | ||
| properties = { | ||
| @ExtensionProperty(name = GENERIC_ENTITY_PROPERTY, value = "true") | ||
| })) | ||
| InputStream requestBody) | ||
| throws Exception; | ||
| } | ||
37 changes: 37 additions & 0 deletions
37
solr/api/src/java/org/apache/solr/client/api/model/UpdateResponse.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| package org.apache.solr.client.api.model; | ||
|
|
||
| import com.fasterxml.jackson.annotation.JsonProperty; | ||
| import io.swagger.v3.oas.annotations.media.Schema; | ||
| import java.util.List; | ||
|
|
||
| /** Version information returned by an update request with {@code versions=true}. */ | ||
| public class UpdateResponse extends SolrJerseyResponse { | ||
|
|
||
| @JsonProperty("adds") | ||
| @Schema(description = "Documents added and the versions assigned to them.") | ||
| public List<VersionedDocument> adds; | ||
|
|
||
| @JsonProperty("deletes") | ||
| @Schema(description = "Documents deleted and the versions assigned to their delete operations.") | ||
| public List<VersionedDocument> deletes; | ||
|
|
||
| @JsonProperty("deleteByQuery") | ||
| @Schema(description = "Delete-by-query operations and the versions assigned to them.") | ||
| public List<VersionedQuery> deleteByQuery; | ||
| } |
32 changes: 32 additions & 0 deletions
32
solr/api/src/java/org/apache/solr/client/api/model/VersionedDocument.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| package org.apache.solr.client.api.model; | ||
|
|
||
| import com.fasterxml.jackson.annotation.JsonProperty; | ||
| import io.swagger.v3.oas.annotations.media.Schema; | ||
|
|
||
| /** A document identifier and the version assigned to an update operation. */ | ||
| public class VersionedDocument { | ||
|
|
||
| @JsonProperty("id") | ||
| @Schema(description = "The document's unique identifier.") | ||
| public String id; | ||
|
|
||
| @JsonProperty("version") | ||
| @Schema(description = "The version assigned to the operation.") | ||
| public long version; | ||
| } |
32 changes: 32 additions & 0 deletions
32
solr/api/src/java/org/apache/solr/client/api/model/VersionedQuery.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| package org.apache.solr.client.api.model; | ||
|
|
||
| import com.fasterxml.jackson.annotation.JsonProperty; | ||
| import io.swagger.v3.oas.annotations.media.Schema; | ||
|
|
||
| /** A delete-by-query expression and the version assigned to the operation. */ | ||
| public class VersionedQuery { | ||
|
|
||
| @JsonProperty("query") | ||
| @Schema(description = "The delete-by-query expression.") | ||
| public String query; | ||
|
|
||
| @JsonProperty("version") | ||
| @Schema(description = "The version assigned to the operation.") | ||
| public long version; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a big enhancement over the existing non JAX-RS appaoche, which required you to call /update/csv for cSV etc. Now we support content-types and you can access /update.