Hue

When executing “hive sql” in Hue, it prompts Fetching results ran into the following error(s): Couldn’t find log associated with operation handle: OperationHandler

First, confirm whether the “/tmp/hadoop/operation_logs/” directory on the master2 node exists and whether the permissions have been deleted or modified; This address is the log address of the hive client. The above operation will cause hiveserver2 to fail to write the log. It can be solved by executing “service hive-server2 restart” on master2 to restart the hive-server2 service.

What if I forget my Hue password?

The default Hue password is hadoop/hadoop or hue/hue, and a password is generally required to be set for the first login.

The Hue page does not support password resetting by default. You can log in to the database where Hue is located on the master1 node and modify the encrypted string in the database.

Here is an example to change the password of the hadoop user to “hadoop”:

a) Log in to the master1 node of the cluster

b) Execute in shell

  mysql

c) Execute

  update hue.auth_user set password="pbkdf2_sha256$12000$FfWacKgK93h2$J4uP5UU5aR/JXvNydvPxFlquyrbWWF2FmtXkrBCOUUA=" where username='hadoop';

(The password in the above sql is the encrypted string of “hadoop”)

How to connect Hue to Spark; Hue homepage displays “The app won’t work without a running Livy Spark Server”, how to deal with it?

Hue3.8.1 has an independent Spark module, which is integrated into the notebook module in Hue3.10.0

Running Spark tasks on Hue depends on livyserver, which is configured by default but needs to be manually started

Start-up method:

cp /home/hadoop/hue/dependentpackages/livyserver /etc/init.d/
chmod u+x /etc/init.d/livyserver
echo "LivyServer##livyserver" >> /etc/default/process
sed -i "s/$/#LivyServer#livy/g" /etc/default/services

After submitting the task on the Hue page, multiple LivyServer processes on the master node do not exit, what to do?

When LivyServer is in the mode of using a spark cluster, it will not auto-terminate after submitting a task in a new window. It can only be manually killed at present. You can put the script below into /etc/cron.hourly and grant execution permissions to automatically clean up the spark submission process started one hour ago.

#!/bin/bash
ppid=`ps axu | grep livy.server.port | grep -v "grep" | awk '{ print  $2}'`
pids=`ps -elf | grep -v "grep" | grep $ppid | awk '{print \$4}'`
NOW=`date +"%s"`
echo now $NOW
for i in $pids
do
if [ $ppid -ne $i ]
then
        echo $i
  JIFFIES=`cat /proc/${i}/stat | cut -d" " -f22`
  UPTIME=`grep btime /proc/stat | cut -d" " -f2`
  START_SEC=$(( $UPTIME + $JIFFIES / $(getconf CLK_TCK)))
  LAST=$(( $NOW - $START_SEC))
  if [ $LAST -gt 3600 ]
  then
        echo $i last $LAST
        kill -9 $i
  fi
fi
done

HBase Spark