Elasticsearch学习笔记

elasticsearch vagrant

概念介绍

• Elasticsearch: Elastic Stack 的核心
  是一个分布式的 RESTful 风格的搜索和数据分析引擎，能够解决不断涌现出的各种用例。作为 Elastic Stack 的核心，它集中存储您的数据，帮助您发现意料之中以及意料之外的情况。
• X-Pack：一个程序包，带来丰富的可能性。
  单就其自身而言，Elastic Stack 就是一款值得考虑的强大工具。X-Pack 将诸多强大功能集合到一个单独的程序包中，更将它带上了一个新的层次。
• Logstash: 集中、转换和存储数据
  是开源的服务器端数据处理管道，能够同时 从多个来源采集数据、转换数据，然后将数据发送到您最喜欢的 “存储库” 中。（我们的存储库当然是 Elasticsearch。）
• Kibana: 这是您走进 Elastic Stack 的窗口
  Kibana 让您能够可视化 Elasticsearch 中的数据并操作 Elastic Stack，因此您可以在这里解开任何疑问：例如，为何会在凌晨 2:00 被传呼，雨水会对季度数据造成怎样的影响。

安装前的检查

1. 安装 Elasticsearch 之前，你需要先安装一个较新的版本的 Java，最好的选择是，你可以从 www.java.com 获得官方提供的最新版本的 Java。

2. 安装JDK

   sudo yum install java-1.8.0-openjdk.x86_64

3. 测试

   [vagrant@localhost vagrant_data]$ java -version
   openjdk version "1.8.0_161"
   OpenJDK Runtime Environment (build 1.8.0_161-b14)
   OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode)

安装Elasticsearch

1. 执行以下命令

   curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.8.tar.gz
   tar -xvf elasticsearch-5.6.8.tar.gz
   cd elasticsearch-5.6.8/bin
   ./elasticsearch

2. 测试是否安装成功

   [vagrant@localhost elasticsearch-5.6.8]$ curl 'http://localhost:9200/?pretty'
   {
     "name" : "Lx20sHw",
     "cluster_name" : "elasticsearch",
     "cluster_uuid" : "gnYSlRb9TUqpVnBscm1-GQ",
     "version" : {
       "number" : "5.6.8",
       "build_hash" : "688ecce",
       "build_date" : "2018-02-16T16:46:30.010Z",
       "build_snapshot" : false,
       "lucene_version" : "6.6.1"
     },
     "tagline" : "You Know, for Search"
   }
   

安装报错处理

1. Vagrant内存不足报错

   [vagrant@localhost elasticsearch-5.6.8]$ ./bin/elasticsearch
       OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
       OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000085330000, 2060255232, 0) failed; error='Cannot allocate memory' (errno=12)
       #
       # There is insufficient memory for the Java Runtime Environment to continue.
       # Native memory allocation (mmap) failed to map 2060255232 bytes for committing reserved memory.
       # An error report file with more information is saved as:
       # /vagrant_data/elasticsearch-5.6.8/hs_err_pid22178.log
   [vagrant@localhost elasticsearch-5.6.8]$ free -m
                 total        used        free      shared  buff/cache   available
   Mem:            488         101          88           3         297         340
   Swap:          1535           1        1534

   解决方法：

   • 打开Vagrantfile

   • 添加如下信息：

     config.vm.provider "virtualbox" do |v|
     v.memory = 2048
     v.cpus = 2
     end
     

   • 重启Vagrant vagrant reload

2. ElasticSearch 启动报错

   ERROR: [2] bootstrap checks failed
   [1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
   [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
   

   解决方法：

   sudo vim /etc/security/limits.conf
   添加如下信息:
   [当前用户名] hard nofile 65536
   [当前用户名] soft nofile 65536
   保存退出
   *******************************************************************
   sysctl -w vm.max_map_count=655360
   sysctl -a | grep "vm.max_map_count"

docker-compose 安装

version: "3"
services:
    php_pc:
        image: php:7.2-cli
        container_name: pcsoft_php
        ports :
            - "9000:9000"
        volumes:
            - ./phpcli:/var/www/html/
    db_pc:
        image: mysql:5.7
        container_name: pcsoft_db
        volumes:
            - ./dbdata:/var/lib/mysql/
        ports:
            - "3306:3306"
        environment:
            MYSQL_USER: root
            MYSQL_PASSWORD: root
            MYSQL_ROOT_PASSWORD: root
    elasticsearch_soft:
        image: registry.cn-hangzhou.aliyuncs.com/amor/elastic:6.2.3
        container_name: es_pc_soft
        environment:
            - cluster.name=docker-cluster
            - bootstrap.memory_lock=true
            - xpack.security.enabled=false
            - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
        ulimits:
            memlock:
                soft: -1
                hard: -1
        volumes:
            - ./docker_es/esdata_soft:/usr/share/elasticsearch/data
        ports:
            - 9200:9200
    elasticsearch_game:
        image: registry.cn-hangzhou.aliyuncs.com/amor/elastic:6.2.3
        container_name: es_pc_game
        environment:
            - cluster.name=docker-cluster
            - bootstrap.memory_lock=true
            - xpack.security.enabled=false
            - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
            - "discovery.zen.ping.unicast.hosts=es_pc_soft"
        ulimits:
            memlock:
                soft: -1
                hard: -1
        volumes:
            - ./docker_es/esdata_game:/usr/share/elasticsearch/data
    kibana:
        image: registry.cn-hangzhou.aliyuncs.com/amor/kibana:6.2.3
        container_name: es_pc_kibana
        environment:
            SERVER_NAME: kibana.pc_amor.com
            ELASTICSEARCH_URL: http://elasticsearch_soft:9200
            XPACK_SECURITY_ENABLED: "false"

docker-compose 安装报错处理

• vm.max_map_count 报错

grep vm.max_map_count /etc/sysctl.conf
# 如果找不到，则在该文件中添加 vm.max_map_count=262144 然后执行 sysctl -p
# 临时改变轻执行 
sysctl -w vm.max_map_count=262144

• 不能以root权限运行问题

请检查Elasticsearch挂载的目录是否是root用户创建的

• Kibana链接问题

直接通过docker-compose server name链接即可

• kibana无法登陆问题

# 添加以下选项：
XPACK_SECURITY_ENABLED: "false"