Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am using AIX 6.1 ksh shell

I want to use one liner to do something like this

cat A_FILE | skip-first-3-bytes-of-the-file  

I want to skip the first 3 bytes of the first line, is there a way to do this?

share|improve this question

migrated from stackoverflow.com Oct 24 '12 at 21:26

This question came from our site for professional and enthusiast programmers.

up vote 7 down vote accepted

Old school — you could use dd:

dd if=A_FILE bs=1c skip=3

The input file is A_FILE, the block size is 1 character (byte), skip the first 3 'blocks' (bytes). There are other ways too:

sed '1s/^...//' A_FILE

This works if there are 3 or more characters on the first line.

tail -c +4 A_FILE

And you could use Perl, Python and so on too.

share|improve this answer
    
Thanks for your help. Both the sed and the tail commands work in AIX 6.1. For the dd command, it should be dd if=A_FILE bs=1 skip=3 in AIX 6.1 – Alvin SIU Oct 25 '12 at 13:55
    
You may want to use standard input as such cat A_FILE | tail -c +4 with gnu. – MUY Belgium Nov 8 '13 at 7:57

Instead of using cat you can use tail as such:

tail -c +4 FILE

This will print out the entire file except for the first 3 bytes. Consult man tail for more information.

share|improve this answer
    
Don't know about AIX, but on Solaris you must use /usr/xpg4/bin/tail, at least on my machine. Good tip nonetheless! – BellevueBob Oct 24 '12 at 19:34
    
@BobDuell It's hard to post something that is compatible with every OS. – squiguy Oct 24 '12 at 20:10
    
Yes, it works in AIX 6.1 – Alvin SIU Oct 25 '12 at 13:54
    
@AlvinSIU Good to know. Glad I could help. – squiguy Oct 25 '12 at 15:43

I needed to recently do something similar. I was helping with a field support issue and needed to let a technician see real time plots as they were making changes. The data is in a binary log that grows throughout the day. I have software that can parse and plot the data from logs, but it is currently not real time. What I did was capture the size of the log before I started processing the data, then went into a loop that would process the data and each pass create a new file with the bytes of the file that had not yet been processed.

#!/usr/bin/env bash

# I named this little script hackjob.sh
# The purpose of this is to process an input file and load the results into
# a database. The file is constantly being update, so this runs in a loop
# and every pass it creates a new temp file with bytes that have not yet been
# processed.  It runs about 15 seconds behind real time so it's
# pseudo real time.  This will eventually be replaced by a real time
# queue based version, but this does work and surprisingly well actually.

set -x

# Current data in YYYYMMDD fomat
DATE=`date +%Y%m%d`

INPUT_PATH=/path/to/my/data
IFILE1=${INPUT_PATH}/${DATE}_my_input_file.dat

OUTPUT_PATH=/tmp
OFILE1=${OUTPUT_PATH}/${DATE}_my_input_file.dat

# Capture the size of the original file
SIZE1=`ls -l ${IFILE1} | awk '{print $5}'`

# Copy the original file to /tmp
cp ${IFILE1} ${OFILE1}

while :
do
    sleep 5

    # process_my_data.py ${OFILE1}
    rm ${OFILE1}
    # Copy IFILE1 to OFILE1 minus skipping the amount of data already processed
    dd skip=${SIZE1} bs=1 if=${IFILE1} of=${OFILE1}
    # Update the size of the input file
    SIZE1=`ls -l ${IFILE1} | awk '{print $5}'`

    echo

    DATE=`date +%Y%m%d`

done
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.