[关闭]
@RebornHuan 2017-05-16T00:54:06.000000Z 字数 8576 阅读 5060

基于kubernetes平台的Zeppelin自助式分析工具落地

编译工程

编译对应的版本为：Zeppelin：0.7.1，CDH：5.7.1

编译环境依赖

Name Value
Git (Any Version)
Maven 3.1.x or higher
JDK 1.8

安装编译所需软件,具体步骤为:
* 安装JDK，安装版本是JDK-1.8.0_60
* 安装NPM，命令： yum install npm
* 安装FontConfig，命令：yum install fontconfig
* 安装Maven

  1. wget http://www.eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
  2. sudo tar -zxf apache-maven-3.3.9-bin.tar.gz -C /usr/local/
  3. sudo ln -s /usr/local/apache-maven-3.3.9/bin/mvn /usr/local/bin/mvn

Notes:
- 使用命令 node --version 来确认 node 已经安装
- 使用命令 mvn -version 来确认 maven 的版本是否在 3.1.x 以上

编译源码

使用maven来编译 Zeppelin：0.7.1 源码：

  1. cd zeppelin
  2. mvn clean package -Pbuild-distr -Pspark-1.6 -Dspark.version=1.6.0 -Dhadoop.version=2.6.0-cdh5.7.1 -Phadoop-2.6 -Pvendor-repo -DskipTests

具体可以参考Zeppelin官网文档

部署Zeppelin

基本配置

  1.  useradd zeppelin
  2.  passwd XXXXXX  # XXXXX为zeppelin用户的密码
  3.  su - zeppelin  # 切换为zeppelin用户
  4.  mkdir zeppelin  #~/zeppelin作为zeppelin解压tar包的根目录
  1.  tar zxf zeppelin-0.7.1.tar.gz -C ~/zeppelin/
  1.  cd ~/zeppelin/zeppelin-0.7.1/conf
  2.  cp zeppelin-env.sh{.template,}
  3.  cp zeppelin-site.xml{.template,}
  4.  cp shiro.ini{.template,}
  1. export JAVA_HOME=/usr/java/latest
  2. export ZEPPELIN_MEM="-Xmx4096m -XX:MaxPermSize-2048m"
  3. export HADOOP_HOME=/opt/cloudera/parcels/CDH
  4. export HADOOP_CONF_DIR=/etc/hadoop/conf
  5. export HBASE_HOME="/opt/cloudera/parcels/CDH/lib/hbase"
  6. export HBASE_CONF_DIR=/etc/hbase/conf
  7. export SPARK_HOME=/opt/cloudera/parcels/CDH/lib/spark
  8. export SPARK_SUBMIT_OPTIONS="--driver-memory 2G --executor-memory 6G"
  9. export ANACONDA_HOME=/opt/cloudera/parcels/Anaconda
  10. export PATH=$ANACONDA_HOME/bin:$HADOOP_HOME/bin:$JAVA_HOME/bin:$PATH
  1. $ hadoop fs -mkdir /user/zeppelin
  2. $ hadoop fs -chmod 777 /user/zeppelin
  1. ### A sample for configuring LDAP Directory Realm
  2. ldapRealm = org.apache.zeppelin.realm.LdapGroupRealm
  3. ## search base for ldap groups (only relevant for LdapGroupRealm):
  4. ldapRealm.contextFactory.environment[ldap.searchBase] = dc=idc,dc=wanda-group,dc=net
  5. ldapRealm.contextFactory.url = ldap://10.214.128.11:389
  6. ldapRealm.userDnTemplate = uid={0},ou=People,dc=idc,dc=wanda-group,dc=net
  7. ldapRealm.contextFactory.authenticationMechanism = SIMPLE
  1. bin/zeppelin-daemon.sh start

Interpreter配置

权限验证

Docker镜像制作

Dockfile

  1. FROM   docker.io/centos:centos7.2.1511
  2. MAINTAINER yuanbowen1 <yuanbowen1@wanda.cn>
  4. ADD latest.tar.gz /usr/java/
  5. ADD zeppelin-0.7.1.tar.gz /home
  6. ADD start.sh /start.sh
  7. COPY krb5.conf /etc/krb5.conf
  8. COPY ldap.conf /etc/openldap/ldap.conf
  10. ENV JAVA_HOME /usr/java/latest
  11. ENV HBASE_HOME /opt/cloudera/parcels/CDH/lib/hbase
  12. ENV HBASE_CONF_DIR /etc/hbase/conf
  13. ENV SPARK_HOME /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/spark
  14. ENV SPARK_SUBMIT_OPTIONS "--driver-memory 2G --executor-memory 6G"
  15. ENV HIVE_HOME /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hive
  16. ENV IMPALA_HOME /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/impala
  17. ENV ZOOKEEPER_HOME /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/zookeeper
  18. ENV HADOOP_HOME /opt/cloudera/parcels/CDH
  19. ENV HADOOP_CONF_DIR /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hadoop/etc/hadoop
  20. ENV ANACONDA_HOME /opt/cloudera/parcels/Anaconda
  21. ENV PATH $ANACONDA_HOME/bin:$HBASE_HOME/bin:$SPARK_HOME/bin:$HIVE_HOME/bin:$IMPALA_HOME/bin:$ZOOKEEPER_HOME/bin:$HADOOP_HOME/bin:$JAVA_HOME/bin:$PATH
  24. RUN yum -y install net-tools \
  25.     && yum -y install openssh-server \
  26.     && yum -y install openssh-clients \
  27.     && yum -y  install krb5-workstation krb5-libs krb5-auth-dialog 1.3 \
  28.     && yum -y install nss-pam-ldapd \
  29.     && yum -y install authconfig \
  30.     && yum -y install initscripts \
  31.     && echo "root:123456" | chpasswd \
  32.     && systemctl enable sshd \
  33.         && chmod 755 /start.sh
  35. ENTRYPOINT ["/start.sh"]

里面包含了单机部署zeppelin的基本配置

  1. #!/bin/sh
  2. cp -Lr /opt/cloudera/parcels/CDH/lib/hive/lib/* /home/zeppelin-0.7.1/interpreter/jdbc/
  3. cp -Lr /opt/cloudera/parcels/CDH/lib/hadoop/hadoop*.jar /home/zeppelin-0.7.1/interpreter/jdbc/
  4. cp -Lr /opt/cloudera/parcels/CDH/lib/hbase/* /home/zeppelin-0.7.1/interpreter/hbase/
  5. cp -r /opt/hosts /etc/hosts
  7. ln -s /opt/etc/hadoop /etc/hadoop
  8. ln -s /opt/etc/hbase /etc/hbase
  9. ln -s /opt/etc/spark /etc/spark
  10. rm -rf /home/zeppelin-0.7.1/notebook
  11. rm -rf /home/zeppelin-0.7.1/conf
  12. if [ ! -d "/opt/zeppelin-${ZEPPELIN_USER}" ]; then
  13.   cp -r /opt/zeppelin-current /opt/zeppelin-${ZEPPELIN_USER}
  14. fi
  15. ln -s /opt/zeppelin-${ZEPPELIN_USER}/notebook /home/zeppelin-0.7.1/notebook
  16. ln -s /opt/zeppelin-${ZEPPELIN_USER}/conf /home/zeppelin-0.7.1/conf
  17. ln -s /opt/zeppelin-${ZEPPELIN_USER}/key /etc/keytab
  19. sed  -i "s/yuanbowen1/${ZEPPELIN_USER}/g"  /home/zeppelin-0.7.1/conf/interpreter.json
  20. export ZEPPELIN_LOG_DIR=/opt/zeppelin-logs/zeppelin-${ZEPPELIN_USER}
  21. /home/zeppelin-0.7.1/bin/zeppelin.sh

镜像制作

进入到Dockerfile所在目录，执行

  1. docker build  -t  zeppelin0.7.1:centos .
  1. docker save zeppelin0.7.1:centos  > /home/yuanbowen1/zeppelin.tar
  1. # scp 
  2. scp -r  /home/yuanbowen1/zeppelin.tar  root@10.214.128.35:/home/
  3. ssh  root@10.214.128.35 "docker load -i /home/zeppelin.tar ; docker rmi 10.199.192.16/zeppelin/zeppelin0.7.1:backup ; docker tag 10.199.192.16/zeppelin/zeppelin0.7.1:centos7.2 10.199.192.16/zeppelin/zeppelin0.7.1:backup ; docker tag zeppelin0.7.1_huan:centos 10.199.192.16/zeppelin/zeppelin0.7.1:centos7.2 ; docker push 10.199.192.16/zeppelin/zeppelin0.7.1:centos7.2"

使用Jenkins生成Docker镜像到Habor

gitlab维护内部工程

jenkins配置

  1. clean package -Pbuild-distr -Pspark-1.6 -Dspark.version=1.6.0 -Dhadoop.version=2.6.0-cdh5.7.1 -Phadoop-2.6 -DskipTests
  1. cd jenkins_distribution
  2. # 备份老的包文件
  3. rm -rf /home/yuanbowen1/new/zeppelin-0.7.1.tar.gz.bak
  4. mv /home/yuanbowen1/new/zeppelin-0.7.1.tar.gz /home/yuanbowen1/new/zeppelin-0.7.1.tar.gz.bak
  5. # 拷贝到 执行目录
  6. cp -r zeppelin-0.7.1.tar.gz /home/yuanbowen1/new
  7. #进入 Dockerfile 所在目录
  8. cd  /home/yuanbowen1/new
  9. # docker build
  10. docker rmi  zeppelin0.7.1_huan:centos
  11. ret=`docker build  -t  zeppelin0.7.1_huan:centos .`
  12. echo $ret
  13. # docker save
  14. docker save zeppelin0.7.1_huan:centos  > /home/yuanbowen1/zeppelin.tar
  15. # scp 
  16. scp -r  /home/yuanbowen1/zeppelin.tar  root@10.214.128.35:/home/
  17. ssh  root@10.214.128.35 "docker load -i /home/zeppelin.tar ; docker rmi 10.199.192.16/zeppelin/zeppelin0.7.1:backup ; docker tag 10.199.192.16/zeppelin/zeppelin0.7.1:centos7.2 10.199.192.16/zeppelin/zeppelin0.7.1:backup ; docker tag zeppelin0.7.1_huan:centos 10.199.192.16/zeppelin/zeppelin0.7.1:centos7.2 ; docker push 10.199.192.16/zeppelin/zeppelin0.7.1:centos7.2"

Kubernetes集群部署Zeppelin

本机配置DNS

访问Kubernetes集群需要专门DNS解析域名 *.kubernetes.wanda.cn. Mac下配置private DNS请参考:
image_1bg5nhc6018hom85v1r1nqbjq9m.png-223.3kB

生成yaml文件

k8s部署

添加新批注
在作者公开此批注前，只有你和作者可见。
回复批注