在此处输入标题

第十三章、大数据协作框架之Hue

1.功能

整合大多数常用的大数据框架在web页面上使用

2.配置安装

2.1 准备MySQL

2.1.1 卸载冲突依赖包

# rpm -e MySQL-server-5.6.24-1.el6.x86_64 MySQL-client-5.6.24-1.el6.x86_64

2.1.2 删除5.6.24-1.el6.x86_64的MySQL存放数据物理目录/var/lib/mysql

2.1.3 安装 mysql-sever

# yum -y install mysql-sever

2.1.4 设置管理员及密码

MySQL5.5以下

# mysqladmin -uroot password '123'

MySQL5.5及以上设置管理员密码参照作业十三

2.1.5 登录MySQL并授权管理员通过使用主机登录,授权所有主机用

# mysql -uroot -p123
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.1.73 Source distribution
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> grant all on *.* to root@'hadoop-senior.ibeifeng.com' identified by '123';
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

授权所有主机用 %

mysql> grant all on *.* to root@'%' identified by '123' ;
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

2.2 Hue安装

2.2.1 解压 tar -zxf hue-3.7.0-cdh5.3.6.tar.gz -C /opt/modules/

2.2.2 安装依赖包

依据官网安装依赖包

 http://archive.cloudera.com/cdh5/cdh/5/hue-3.7.0-cdh5.3.6/manual.html#_install_hue

# yum -y install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libtidy  libxml2-devel libxslt-devel openldap-devel python-devel sqlite-devel openssl-devel mysql-devel gmp-devel 

2.2.3编译Hue

$ cd /opt/modules/hue-3.7.0-cdh5.3.6/
$ make apps

...
编译完成

Running migrations for useradmin:
 - Migrating forwards to 0003_remove_metastore_readonly_huepermission.
 > useradmin:0001_permissions_and_profiles
 - Migration 'useradmin:0001_permissions_and_profiles' is marked for no-dry-run.
 > useradmin:0002_add_ldap_support
 - Migration 'useradmin:0002_add_ldap_support' is marked for no-dry-run.
 > useradmin:0003_remove_metastore_readonly_huepermission
 - Migration 'useradmin:0003_remove_metastore_readonly_huepermission' is marked for no-dry-run.
 - Loading initial data for useradmin.
Installed 0 object(s) from 0 fixture(s)
make[1]: Leaving directory `/opt/modules/hue-3.7.0-cdh5.3.6/apps'

查看编译

$ cd apps/
$ ll
total 84
drwxr-xr-x 4 beifeng beifeng 4096 Jun  7 16:19 about
drwxr-xr-x 6 beifeng beifeng 4096 Jun  7 16:19 beeswax
drwxr-xr-x 3 beifeng beifeng 4096 Jun  7 16:19 filebrowser
drwxr-xr-x 7 beifeng beifeng 4096 Jun  7 16:19 hbase
drwxr-xr-x 3 beifeng beifeng 4096 Jun  7 16:19 help
drwxr-xr-x 4 beifeng beifeng 4096 Jun  7 16:19 impala
drwxr-xr-x 4 beifeng beifeng 4096 Jun  7 16:19 jobbrowser
drwxr-xr-x 4 beifeng beifeng 4096 Jun  7 16:19 jobsub
drwxrwxr-x 2 beifeng beifeng 4096 Jun  7 16:20 logs
-rw-r--r-- 1 beifeng beifeng 3369 Jul 29  2015 Makefile
drwxr-xr-x 4 beifeng beifeng 4096 Jun  7 16:19 metastore
drwxr-xr-x 5 beifeng beifeng 4096 Jun  7 16:19 oozie
drwxr-xr-x 5 beifeng beifeng 4096 Jun  7 16:19 pig
drwxr-xr-x 3 beifeng beifeng 4096 Jun  7 16:19 proxy
drwxr-xr-x 4 beifeng beifeng 4096 Jun  7 16:20 rdbms
drwxr-xr-x 5 beifeng beifeng 4096 Jun  7 16:19 search
drwxr-xr-x 4 beifeng beifeng 4096 Jun  7 16:20 security
drwxr-xr-x 4 beifeng beifeng 4096 Jun  7 16:20 spark
drwxr-xr-x 4 beifeng beifeng 4096 Jun  7 16:19 sqoop
drwxr-xr-x 4 beifeng beifeng 4096 Jun  7 16:19 useradmin
drwxr-xr-x 4 beifeng beifeng 4096 Jun  7 16:20 zookeeper

2.2.4 修改配置文件

2.2.4.1 依照 Hue官方文档配置 Hadoop hdfs-site.xml 和 core-site.xml ,修改后重启Hadoop服务

http://archive.cloudera.com/cdh5/cdh/5/hue-3.7.0-cdh5.3.6/manual.html#_hadoop_configuration
2.2.1. Configure WebHdfs
You need to enable WebHdfs or run an HttpFS server. To turn on WebHDFS, add this to your hdfs-site.xml and restart your HDFS cluster. Depending on your setup, your hdfs-site.xml might be in /etc/hadoop/conf.
    <property>
      <name>dfs.webhdfs.enabled</name>
      <value>true</value>
    </property>
You also need to add this to core-site.html.
    <property>
      <name>hadoop.proxyuser.hue.hosts</name>
      <value>*</value>
    </property>
    <property>
      <name>hadoop.proxyuser.hue.groups</name>
      <value>*</value>
    </property>

2.2.4.2修改配置文件 hue-3.7.0-cdh5.3.6/desktop/conf/hue.ini 各协调框架都在此文件集成

2.2.4.2.1 参照官方配置秘钥 以下为官方文档

http://archive.cloudera.com/cdh5/cdh/5/hue-3.7.0-cdh5.3.6/manual.html#_web_server_configuration
3.1.2. Specifying the Secret Key
For security, you should also specify the secret key that is used for secure hashing in the session store. Enter a long series of random characters (30 to 60 characters is recommended).
secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
[Note]  
If you don’t specify a secret key, your session cookies will not be secure. Hue will run but it will also display error messages telling you to set the secret key.

使用官方默认秘钥，添加至 hue.ini 注意秘钥长度 30 - 60 .

[desktop]
  # Set this to a random string, the longer the better.
  # This is used for secure hashing in the session store.
  secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o

2.2.4.2.2 配置主机地址

  # Webserver listens on this address and port
  http_host=hadoop-senior.ibeifeng.com
  http_port=8888
  # Time zone name
  time_zone=Asia/Shanghai

2.2.4.2在hue安装目录下执行 $ build/env/bin/supervisor 后，可以在 web 登录hue

创建管理员 admin 密码 admin

依照 Hue官方文档配置 Hadoop hdfs-site.xml 和 core-site.xml

http://archive.cloudera.com/cdh5/cdh/5/hue-3.7.0-cdh5.3.6/manual.html#_hadoop_configuration
2.2.1. Configure WebHdfs
You need to enable WebHdfs or run an HttpFS server. To turn on WebHDFS, add this to your hdfs-site.xml and restart your HDFS cluster. Depending on your setup, your hdfs-site.xml might be in /etc/hadoop/conf.
    <property>
      <name>dfs.webhdfs.enabled</name>
      <value>true</value>
    </property>
You also need to add this to core-site.html.
    <property>
      <name>hadoop.proxyuser.hue.hosts</name>
      <value>*</value>
    </property>
    <property>
      <name>hadoop.proxyuser.hue.groups</name>
      <value>*</value>
    </property>