4

I have used pipeline to read specific record from impala shell. Here is what I got

[cloudera@localhost ~]$ echo "select * from abc where key > 'a-26052014015400' limit 1;" | impala-shell
Starting Impala Shell without Kerberos authentication
Connected to localhost.localdomain:21000
Server version: impalad version cdh5-1.3.0 RELEASE (build 40e1b62cf0b97f666d084d9509bf9639c575068c)
Welcome to the Impala shell. Press TAB twice to see a list of available commands.

Copyright (c) 2012 Cloudera, Inc. All rights reserved.

(Shell build version: Impala Shell vcdh5-1.3.0 (40e1b62) built on Tue Mar 25 13:46:44 PDT 2014)
Query: select * from abc where key > 'a-26052014015400' limit 1
[localhost.localdomain:21000] > +------------------------+------+----------------+-------+
| key                    | hpid | uts            | value |
+------------------------+------+----------------+-------+
| a-26052014015700 | HS2  | 26052014015450 | 50    |
+------------------------+------+----------------+-------+
Returned 1 row(s) in 2.42s
Goodbye

What I really want is a-26052014015700 | HS2 | 26052014015450 | 50 this record in awk programming. I have tried with pipelined awk command

`echo "select * from abc where key > 'a-26052014015400' limit 1;" | impala-shell| awk -F'=' '{print $2}' | awk -F '>' '{print $1}`

but didn't get the expected output. Any better and efficient method to extract the record?

4
  • 2
    Won't grep be a better tool if you just want to extract the pattern?
    – unxnut
    Jul 14 '14 at 14:34
  • @unxnut thanks for your comment and am not stick to awk and pipe command,just want to get this record only.
    – Aashu
    Jul 14 '14 at 15:02
  • In that case, your best bet is to create a regular expression to describe some pattern in your record (key value, perhaps) and use grep to filter that.
    – unxnut
    Jul 14 '14 at 15:48
  • 1
    Why are you using = and > as your field separator? I don't see them in the table output.
    – Barmar
    Jul 14 '14 at 16:25
4

try to pipe it to grep:

$  grep -E "| a-[0-9]* | HS2  | [0-9]* | [0-9]* |" 

to get rid of the first | and the last |:

$  grep -Eo "  a-[0-9]* \| HS2 \| [0-9]* \| [0-9]* " 

"-E" to access the extended regular expression syntax

"-o" is used to only output the matching segment of the line, rather than the full contents of the line.

2
  • You need some * or + quantifiers in your regexp, don't you?
    – Barmar
    Jul 14 '14 at 16:24
  • 1
    @Networker thanks for your answer I have used it with slide change grep -Eo " a-[0-9]* \| HS2 \| [0-9]* \| [0-9]* " because in contains 2012 as well which is not required.
    – Aashu
    Jul 16 '14 at 6:04
3

If you know your output will always be in a format of X lines of header and Y lines of footer, you can use head and tail to get only the part you need,

    echo query | impala-shell | tail -n +X | head -n -Y

    # -- in your case above --
    echo "select * from abc where key > 'a-26052014015400' limit 1;" | impala-shell \
                | tail -n +13 | head -n -3 

    # returns
    | a-26052014015700 | HS2  | 26052014015450 | 50    |

tail

    -n, --lines=K
            output the last K lines, instead of the last 10; 
            or use -n +K to output lines starting with the Kth

head

    -n, --lines=[-]K
            print the first K lines instead of the first 10; 
            with the leading '-', print all but the last K lines of each file
1
  • 1
    You don’t even need to know how many lines come after the one(s) you’re looking for. To capture the 14th line, all you need is | tail -n +13 | head -n 1. (On some systems, the archaic, undocumented … | head -1 syntax might work.) To get the 14th-16th lines, use | tail -n +13 | head -n 3.
    – Scott
    Jul 14 '14 at 23:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.