adhocshare Blog

logo-hive 0

Can’t connect Excel to Hive using ODBC driver on MAC

So you done everything right and can’t connect Excel to Hive using ODBC driver on your macOS? Let’s see what is going on. Are you running El Capitan on Sierra? Well I was running Sierra and tried connecting before while...

logo-hive 0

Connecting SQuirrel SQL to Hive

Pre-requisites In order to connect SQuirrel SQL client we need the following prerequisites, Client – http://squirrel-sql.sourceforge.net/ Hive connection JARs (found in lib directories) Hive JDBC JAR – hive-jdbc-1.2.1-standalone.jar Hadoop common JAR (for ) – hadoop-common-2.7.2.jar Running HiveServer2 instance For connections use the following...

logo-hive 0

Creating Hive tables on compressed files

Stuck with creating Hive tables on compressed files? Well the documentation on apache.org suggests that Hive natively supports compressed file – https://cwiki.apache.org/confluence/display/Hive/CompressedStorage Lets try that out. Store a snappy compressed file on HDFS. … thinking, I do not have such file… Wait!...

logo-hive 0

Query escaped JSON string in Hive

There are times when we want to parse a string that is actually a JSON. Usually that could be done with built in functions of Hive such as get_json_object(). Though get_json_object cannot parse JSON Array from my experience. These array...

logo-linux 0

Setting up tmux without root access

At times we want access to tmux and we are just stuck due to admin restrictions or root access. There are three important parts to setting up tmux, It is dependent on libevent Compiling and installing on non-system folders require customized...

logo-hive 0

Using JSON SerDe in Hive

Using JsonSerDe in Hive Download JSON Serde – https://github.com/rcongiu/Hive-JSON-Serde Compile command for hive 1.2.1 – “mvn -Pcdh5 -Dcdh5.hive.version=1.2.1 clean package” . change hive version per the environment Copy json-serde/target/json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jar (or similar) to hive/lib Restart hive Sample JSON with test HiveQLs...

logo-bash 0

SSH Keep connection alive

Add below lines to file ~/.ssh/config Host * ServerAliveCountMax 3 ServerAliveInterval 10 TCPKeepAlive yes The first line specifies that this applies to all hosts The second and third specify that SSH2 keepalive messages should be sent every 10 seconds during...

logo-apple 0

Change DNS server on Mac from command line

Fetch DNS servers list networksetup -getdnsservers Wi-Fi Set DNS servers list Format: networksetup -setdnsservers (Network Service) (DNS IP list) networksetup -setdnsservers Wi-Fi 208.67.222.222 208.67.220.220 8.8.8.8 8.8.4.4   References: http://osxdaily.com/2015/06/02/change-dns-command-line-mac-os-x/   Related posts: MAC terminal shortcut keys How-to: Install Hue on a Mac umask in...

logo-apple 0

SSH Auto completion on OSX

Something that is very much desired while working on shell. I love to have this feature of $ssh ser[TAB] Using brew is my preferred way, just type in – $brew install bash-completion Do not forget to add the text in...

logo-hadoop 0

HDFS disk consumption – Find what is taking hdfs space

Source: https://community.hortonworks.com/articles/16846/how-to-identify-what-is-consuming-space-in-hdfs.html Script #!/usr/bin/env bash max_depth=5 largest_root_dirs=$(hdfs dfs -du -s ‘/*’ | sort -nr | perl -ane ‘print “$F[1] “‘) printf “%15s %s\n” “bytes” “directory” for ld in $largest_root_dirs; do printf “%15.0f %s\n” $(hdfs dfs -du -s $ld| cut -d’ ‘...