11

Having byte offset for a file.

Is there a tool that gives line number for this byte?

  • Byte count starting with zero, as in: first byte is 0 not 1.
  • Line number starting with 1.
  • File can have both plain text, "binary" blobs, multibyte characters etc. But the section I am interested in: End of file, has only ASCII.

Example, file:

001
002
003  <<-- first zero on this line is byte 8
004

Having byte offset 8 that would give me line 3.

Guess I could use something like this to find line number:

 a. tail -c+(offset + 1) file | wc -l, here +1 as tail counts from 1.
 b. wc -l file
 c. Then tail -n+num where num is a - b + 1

But ... is there a, fairly common, tool that can give me num directly?


Edit, err: or the more obvious:

head -c+offset file | wc -l
  • 2
    Binary files don't have lines. – Kusalananda Jan 31 '17 at 18:21
  • @Kusalananda: Lines in this context is data separated by 0x0a bytes. – user367890 Jan 31 '17 at 18:29
  • 3
    Probably not what you're asking, but Vim has a function for it. It counts offsets from 1, so: :echo byte2line(offset+1). – Satō Katsura Jan 31 '17 at 18:46
  • @SatoKatsura: Yes, and thanks. Tried with vim first. But even with vim -b and vim + set binary + open file it got corrupted. (Ah. Suddenly I recall which plugin messes it up). But, anyway, as I use this in batches and in combination with a range of scripts Vim was early abandoned. But +1 anyways. – user367890 Jan 31 '17 at 23:22
  • @user367890 A binary file can have 0xa anywhere. The concept of lines in a binary file is meaningless. – user207421 Feb 1 '17 at 1:26
14

In your example,

001
002
003
004

byte number 8 is the second newline, not the 0 on the next line.

The following will give you the number of full lines after $b bytes:

$ dd if=data.in bs=1 count="$b" | wc -l

It will report 2 with b set to 8 and it will report 1 with b set to 7.

The dd utility, the way it's used here, will read from the file data.in, and will read $b blocks of size 1 byte.

As "icarus" rightly points out in the comments below, using bs=1 is inefficient. It's more efficient, in this particular case, to swap bs and count:

$ dd if=data.in bs="$b" count=1 | wc -l

This will have the same effect as the first dd command, but will read only one block of $b bytes.

The wc utility counts newlines, and a "line" in Unix is always terminated by a newline. So the above command will still say 2 if you set b to anything lower than 12 (the following newline). The result you are looking for is therefore whatever number the above pipeline reports, plus 1.

This will obviously also count the random newlines in the binary blob part of your file that precedes the ASCII text. If you knew where the ASCII bit starts, you could add skip="$offset" to the dd command, where $offset is the number of bytes to skip into the file.

  • @don_crissti head: unknown option -- c – Kusalananda Jan 31 '17 at 18:23
  • @Kusalananda You're using BSD head, options there are different – Sergiy Kolodyazhnyy Jan 31 '17 at 18:24
  • @Serg :-) I'm well aware of that. We don't know what tho OP uses, so I'm sticking to POSIX. – Kusalananda Jan 31 '17 at 18:26
  • 1
    As I mentioned in Q: byte count starts with 0, not 1, therefore 8==0 ... – user367890 Jan 31 '17 at 18:31
  • @user367890 In that case, use $(( b - 1 )). – Kusalananda Jan 31 '17 at 18:32
4

Currently there is no dedicated tool like that, although it can be done fairly easily in python:

#!/usr/bin/env python3
import sys
import os

offset = int(sys.argv[2])
newline = 1
with open(sys.argv[1]) as fd:
    fd.seek(offset)
    while True:
        try:
            byte = fd.read(1)
            if byte == '\n': newline+=1
            #print(byte)
            offset = offset - 1
            fd.seek(offset)
        except ValueError:
            break
print(newline)

Usage is simple:

line4byte.py <FILE> <BYTE>

Test run:

$ cat input.txt
001
002
003
004
$ chmod +x ./line4byte.py                                                     
$ ./line4byte.py input.txt 8                                                  
3

This is a very quick and simple script. It doesn't check if the file is empty or not, so it works only on non-empty files.

4

Track the bytes seen and emit the current line number should the given offset be within the sum:

perl -E '$off=shift;while(<>){$sum+=length;if($sum>=$off){say $.;exit}}' 8 file

Or at length:

#!/usr/bin/env perl
use strict;
use warnings;
die "Usage: $0 offset file|-\n" if @ARGV != 2;
my $offset = shift;
shift if $ARGV[0] eq '-';
my $sum;
while (readline) {
    $sum += length;
    if ($sum >= $offset) {
        print "$.\n";
        exit;
    }
}
exit 1;
3
$perl -0nE 'say substr($_,0,8)=~ y/\n//'  ex
2
  • perl -0nE exp slurps the input into $_ and executes exp
  • substr(string,0,8) selects the first 8 bytes
  • y/\n// removes the \n and returns its number

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.