Skip to content

Commit 2fcbbf4

Browse files
authored
[Connector-V2] [ElasticSearch] Fix ElasticSearch Connector V2 Bug (#2817)
1 parent e568bcf commit 2fcbbf4

29 files changed

Lines changed: 27 additions & 17 deletions

File tree

docs/en/connector-v2/sink/Elasticsearch.md

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -19,15 +19,15 @@ Engine Supported
1919

2020
## Options
2121

22-
| name | type | required | default value |
23-
|-------------------| ------ | -------- |---------------|
24-
| hosts | array | yes | - |
25-
| index | string | yes | - |
26-
| index_type | string | no | |
27-
| username | string | no | |
28-
| password | string | no | |
29-
| max_retry_size | int | no | 3 |
30-
| max_batch_size | int | no | 10 |
22+
| name | type | required | default value |
23+
|----------------|--------|----------|---------------|
24+
| hosts | array | yes | - |
25+
| index | string | yes | - |
26+
| index_type | string | no | |
27+
| username | string | no | |
28+
| password | string | no | |
29+
| max_retry_size | int | no | 3 |
30+
| max_batch_size | int | no | 10 |
3131

3232

3333

@@ -36,6 +36,7 @@ Engine Supported
3636

3737
### index [string]
3838
`Elasticsearch` `index` name.Index support contains variables of field name,such as `seatunnel_${age}`,and the field must appear at seatunnel row.
39+
If not, we will treat it as a normal index.
3940

4041
### index_type [string]
4142
`Elasticsearch` index type, it is recommended not to specify in elasticsearch 6 and above

seatunnel-connectors-v2/connector-common/src/main/java/org/apache/seatunnel/connectors/seatunnel/common/source/AbstractSingleSplitSource.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ public abstract class AbstractSingleSplitSource<T> implements SeaTunnelSource<T,
2929

3030
@Override
3131
public final AbstractSingleSplitReader<T> createReader(SourceReader.Context readerContext) throws Exception {
32-
checkArgument(readerContext.getIndexOfSubtask() == 0, "A single split source allows only one single reader to be created.");
32+
checkArgument(readerContext.getIndexOfSubtask() == 0, "A single split source allows only one single reader to be created. Please make sure source parallelism = 1");
3333
return createReader(new SingleSplitReaderContext(readerContext));
3434
}
3535

seatunnel-connectors-v2/connector-elasticsearch/pom.xml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@
3232
<properties>
3333
<elasticsearch-rest-client.version>7.5.1</elasticsearch-rest-client.version>
3434
<jackson.databind.version>2.12.6</jackson.databind.version>
35+
<jackson-datatype-jsr310.version>2.13.3</jackson-datatype-jsr310.version>
3536
</properties>
3637

3738
<dependencies>
@@ -45,6 +46,11 @@
4546
<artifactId>jackson-databind</artifactId>
4647
<version>${jackson.databind.version}</version>
4748
</dependency>
49+
<dependency>
50+
<groupId>com.fasterxml.jackson.datatype</groupId>
51+
<artifactId>jackson-datatype-jsr310</artifactId>
52+
<version>${jackson-datatype-jsr310.version}</version>
53+
</dependency>
4854
</dependencies>
4955

5056
</project>

seatunnel-connectors-v2/connector-elasticsearch/src/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/client/EsRestClient.java renamed to seatunnel-connectors-v2/connector-elasticsearch/src/main/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/client/EsRestClient.java

File renamed without changes.

seatunnel-connectors-v2/connector-elasticsearch/src/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/config/SinkConfig.java renamed to seatunnel-connectors-v2/connector-elasticsearch/src/main/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/config/SinkConfig.java

File renamed without changes.

seatunnel-connectors-v2/connector-elasticsearch/src/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/constant/BulkConfig.java renamed to seatunnel-connectors-v2/connector-elasticsearch/src/main/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/constant/BulkConfig.java

File renamed without changes.

seatunnel-connectors-v2/connector-elasticsearch/src/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/constant/ElasticsearchVersion.java renamed to seatunnel-connectors-v2/connector-elasticsearch/src/main/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/constant/ElasticsearchVersion.java

File renamed without changes.

seatunnel-connectors-v2/connector-elasticsearch/src/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/dto/BulkResponse.java renamed to seatunnel-connectors-v2/connector-elasticsearch/src/main/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/dto/BulkResponse.java

File renamed without changes.

seatunnel-connectors-v2/connector-elasticsearch/src/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/dto/IndexInfo.java renamed to seatunnel-connectors-v2/connector-elasticsearch/src/main/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/dto/IndexInfo.java

File renamed without changes.

seatunnel-connectors-v2/connector-elasticsearch/src/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/exception/BulkElasticsearchException.java renamed to seatunnel-connectors-v2/connector-elasticsearch/src/main/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/exception/BulkElasticsearchException.java

File renamed without changes.

0 commit comments

Comments
 (0)