4

I have used pipeline to read specific record from impala shell. Here is what I got

[cloudera@localhost ~]$ echo "select * from abc where key > 'a-26052014015400' limit 1;" | impala-shell
Starting Impala Shell without Kerberos authentication
Connected to localhost.localdomain:21000
Server version: impalad version cdh5-1.3.0 RELEASE (build 40e1b62cf0b97f666d084d9509bf9639c575068c)
Welcome to the Impala shell. Press TAB twice to see a list of available commands.

Copyright (c) 2012 Cloudera, Inc. All rights reserved.

(Shell build version: Impala Shell vcdh5-1.3.0 (40e1b62) built on Tue Mar 25 13:46:44 PDT 2014)
Query: select * from abc where key > 'a-26052014015400' limit 1
[localhost.localdomain:21000] > +------------------------+------+----------------+-------+
| key                    | hpid | uts            | value |
+------------------------+------+----------------+-------+
| a-26052014015700 | HS2  | 26052014015450 | 50    |
+------------------------+------+----------------+-------+
Returned 1 row(s) in 2.42s
Goodbye

What I really want is a-26052014015700 | HS2 | 26052014015450 | 50 this record in awk programming. I have tried with pipelined awk command

`echo "select * from abc where key > 'a-26052014015400' limit 1;" | impala-shell| awk -F'=' '{print $2}' | awk -F '>' '{print $1}`

but didn't get the expected output. Any better and efficient method to extract the record?

4
  • 2
    Won't grep be a better tool if you just want to extract the pattern?
    – unxnut
    Commented Jul 14, 2014 at 14:34
  • @unxnut thanks for your comment and am not stick to awk and pipe command,just want to get this record only.
    – Aashu
    Commented Jul 14, 2014 at 15:02
  • In that case, your best bet is to create a regular expression to describe some pattern in your record (key value, perhaps) and use grep to filter that.
    – unxnut
    Commented Jul 14, 2014 at 15:48
  • 1
    Why are you using = and > as your field separator? I don't see them in the table output.
    – Barmar
    Commented Jul 14, 2014 at 16:25

2 Answers 2

4

try to pipe it to grep:

$  grep -E "| a-[0-9]* | HS2  | [0-9]* | [0-9]* |" 

to get rid of the first | and the last |:

$  grep -Eo "  a-[0-9]* \| HS2 \| [0-9]* \| [0-9]* " 

"-E" to access the extended regular expression syntax

"-o" is used to only output the matching segment of the line, rather than the full contents of the line.

2
  • You need some * or + quantifiers in your regexp, don't you?
    – Barmar
    Commented Jul 14, 2014 at 16:24
  • 1
    @Networker thanks for your answer I have used it with slide change grep -Eo " a-[0-9]* \| HS2 \| [0-9]* \| [0-9]* " because in contains 2012 as well which is not required.
    – Aashu
    Commented Jul 16, 2014 at 6:04
3

If you know your output will always be in a format of X lines of header and Y lines of footer, you can use head and tail to get only the part you need,

    echo query | impala-shell | tail -n +X | head -n -Y

    # -- in your case above --
    echo "select * from abc where key > 'a-26052014015400' limit 1;" | impala-shell \
                | tail -n +13 | head -n -3 

    # returns
    | a-26052014015700 | HS2  | 26052014015450 | 50    |

tail

    -n, --lines=K
            output the last K lines, instead of the last 10; 
            or use -n +K to output lines starting with the Kth

head

    -n, --lines=[-]K
            print the first K lines instead of the first 10; 
            with the leading '-', print all but the last K lines of each file
1
  • 1
    You don’t even need to know how many lines come after the one(s) you’re looking for. To capture the 14th line, all you need is | tail -n +13 | head -n 1. (On some systems, the archaic, undocumented … | head -1 syntax might work.) To get the 14th-16th lines, use | tail -n +13 | head -n 3. Commented Jul 14, 2014 at 23:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .