0%

基于Docker部署ElasticSearch以及分词插件

Posted on

背景

本文讲解如何基于Docker部署ElasticSearch以及对应ik分词插件。

安装ElasticSearch

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.14.2

# 要启动单节点 Elasticsearch 集群进行开发或测试，请指定单节点发现以绕过启动检查：

# -d 以守护进程模式后台运行容器
docker run -d \
  --name elasticsearch \
  -p 9200:9200 -p 9300:9300 \
  -e "discovery.type=single-node" \
  -e xpack.security.enabled=false \
  -e "ES_JAVA_OPTS=-Xms512m -Xmx512m" \
  -v /data/opt/es_data/data:/usr/share/elasticsearch/data \
  -v /data/opt/es_data/logs:/usr/share/elasticsearch/logs  \
  -v /data/opt/es_data/plugins:/usr/share/elasticsearch/plugins \
  docker.elastic.co/elasticsearch/elasticsearch:8.14.2

启动遇到的问题

1
docker logs -f elasticsearch

启动异常：AccessDeniedException: /usr/share/elasticsearch/data/nodes

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
java.lang.IllegalStateException: failed to obtain node locks, tried [/usr/share/elasticsearch/data]; maybe these locations are not writable or multiple nodes were started on the same data path?at org.elasticsearch.server@8.14.2/org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:293)
  at org.elasticsearch.server@8.14.2/org.elasticsearch.node.NodeConstruction.validateSettings(NodeConstruction.java:509)
  at org.elasticsearch.server@8.14.2/org.elasticsearch.node.NodeConstruction.prepareConstruction(NodeConstruction.java:256)
  at org.elasticsearch.server@8.14.2/org.elasticsearch.node.Node.<init>(Node.java:192)
  at org.elasticsearch.server@8.14.2/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:240)
  at org.elasticsearch.server@8.14.2/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:240)
  at org.elasticsearch.server@8.14.2/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:75)\nCaused by: java.io.IOException: failed to obtain lock on /usr/share/elasticsearch/data
  at org.elasticsearch.server@8.14.2/org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:238)
  at org.elasticsearch.server@8.14.2/org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:206)
  at org.elasticsearch.server@8.14.2/org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:285)
... 6 more\nCaused by: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/data/node.lock
  at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
  at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
  at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
  at java.base/sun.nio.fs.UnixPath.toRealPath(UnixPath.java:886)
  at org.apache.lucene.core@9.10.0/org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:94)
  at org.apache.lucene.core@9.10.0/org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:43)
  at org.apache.lucene.core@9.10.0/org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:44)
  at org.elasticsearch.server@8.14.2/org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:231)
... 8 more
Suppressed: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/node.lock
    at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
    at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
    at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
    at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:261)
    at java.base/java.nio.file.Files.newByteChannel(Files.java:379)
    at java.base/java.nio.file.Files.createFile(Files.java:657)
    at org.apache.lucene.core@9.10.0/org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:84)
    ... 11 more\n

发现确实权限不足导致

磁盘主机目录权限

1
2
# 挂载主机目录时需赋予读写权限
chmod -R 777  /data/opt/es_data

启动验证

1
2
3
4
curl -s  'http://localhost:9200'

# 提交请求 _cat/nodes 查看节点是否启动运行
curl -X GET "localhost:9200/_cat/nodes?v=true&pretty"

启动验证

下载插件

插件下载可在docker内部通过elasticsearch-plugin 在线安装，也可离线下载，复制到plugin目录下。

在线安装

1
2
3
docker exec -it elasticsearch /bin/bash

bin/elasticsearch-plugin install https://get.infini.cloud/elasticsearch/analysis-ik/8.4.1

Es插件版本兼容

​​问题原因​​

错误提示 ​​Plugin [analysis-ik] was built for Elasticsearch version 8.4.1 but version 8.14.2 is running​​ 表示 IK 分词器插件版本与当前 Elasticsearch 版本不兼容。

主要原因为​版本差异过大​或更新滞后​：IK 插件版本（8.4.1）远低于 Elasticsearch 版本（8.14.2），导致版本校验失败 。

解决方式

可以采用离线安装，修改 IK 插件配置文件适配当前版本​​，适用于​已有接近版本的 IK 插件（如 8.10.4），需快速适配 8.14.2。

离线安装

在线安装经常会因为网络问题导致插件下载失败，所以可以采取离线安装。

​下载插件​​：从 IK 官方仓库或其他仓库（https://release.infinilabs.com/） 下载与 Elasticsearch 版本最接近的插件（如 8.10.4）。

版本匹配

如果插件版本和Elastic-Search版本一致，则可复制到plugin目录下或者使用elasticsearch-plugin进行安装。

1
2
3
docker cp elasticsearch-analysis-ik-8.14.2.zip elasticsearch:/usr/share/elasticsearch/plugins/
docker exec -it elasticsearch /bin/bash
elasticsearch-plugin install file:///usr/share/elasticsearch/plugins/elasticsearch-analysis-ik-8.14.2.zip

版本不匹配

如果版本不匹配，则找​已有接近版本的 IK 插件下载并调整配置：

​​解压并修改配置​​：
1
2
unzip elasticsearch-analysis-ik-8.10.4.zip -d elasticsearch-analysis-ik-8.14.2
cd elasticsearch-analysis-ik-8.14.2
编辑 plugin-descriptor.properties 文件：
1
2
version=8.14.2
elasticsearch.version=8.14.2

​​关键点​​：确保两处版本号与 Elasticsearch 一致

重新打包并安装:
1
2
3
4
zip -r elasticsearch-analysis-ik-8.14.2.zip *
docker cp elasticsearch-analysis-ik-8.14.2.zip elasticsearch:/usr/share/elasticsearch/plugins/
docker exec -it elasticsearch /bin/bash
elasticsearch-plugin install file:///usr/share/elasticsearch/plugins/elasticsearch-analysis-ik-8.14.2.zip

重启Docker：

1
2
3
4
5
6
7
8
9
docker restart  elasticsearch
# 验证IK分词效果

# 测试 ik_max_word（最细粒度分词）
curl -X POST "http://localhost:9200/_analyze" -H 'Content-Type: application/json' -d '{
  "analyzer": "ik_max_word",
  "text": "中华人民共和国成立"
}'

创建索引

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
curl -X PUT "localhost:9200/my_index?pretty" -H 'Content-Type: application/json' -d'
{
    "mappings" : {
        "properties" : {
          "entity_name" : {
            "type" : "text",
            "analyzer": "ik_max_word",
            "search_analyzer": "ik_max_word"
          },
          "aliaes" : {
            "type" : "text",
            "analyzer": "ik_max_word",
            "search_analyzer": "ik_max_word"
          },
          "col_name" : {
            "type" : "keyword",
            "index" : false
          },
          "col_comment" : {
            "type" : "keyword",
            "index" : false
          }
        }
      }
  }'