Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Is there any way to read entire file into memory in bash, without using external command like cat?

I know the syntax to get output from a file, i.e <(/bin/ls), but now I need to read the file into entire memory, i.e $A would hold the contents (and there could be null, EOF like stuff)

share|improve this question
2  
    
@l0b0 … unless the shell is zsh. –  Gilles Dec 4 '12 at 23:19
    
@Gilles Interesting; I hope that makes it to Bash someday (unless it would break too much). Maybe they should amend the text to "Bash strings." –  l0b0 Dec 5 '12 at 6:49
    
@l0b0 Other shells (Bourne, ksh, ash, csh, tcsh) have the same restriction. They aren't designed to process binary data. Shell variables originated from environment variables which can't contain null bytes either. Unix favors text, and shells are geared towards handling text. –  Gilles Dec 5 '12 at 11:02
add comment

2 Answers

One way like this:

$ A=$(<file)

Instead of a variable, a better option would be to read into an array which makes it easier to access different lines:

$ OIFS=$IFS
$ IFS=$'\n'
$ arr=($(<file))
$ IFS=$OIFS
share|improve this answer
    
echo $(</usr/bin/ls) > ls; diff ls /usr/bin/ls differs, was it a problem with echo? –  warl0ck Dec 4 '12 at 1:58
    
The echo command assumes that the input is 'printable' ASCII, not binary data. The ls program file contains binary data. Beyond that, the echo will likely append a trailing newline, which in itself makes it different from the actual contents. –  Arcege Dec 4 '12 at 2:14
1  
Watch out: arr=($(<file)) is an array of words in the file, not necessarily lines (assuming default value of $IFS). For an array of lines, use the mapfile command. –  glenn jackman Dec 4 '12 at 5:01
    
@glennjackman : Thanks glenn for pointing it out..updated the solution using IFS since mapfile is available only from bash4... –  Guru Dec 4 '12 at 5:12
    
@warl0ck: try it with echo "$(</usr/bin/ls)" > ls –  Gert van den Berg Dec 4 '12 at 14:00
show 2 more comments

Bash doesn't support null bytes in variables. foo=$(cat file) makes $foo contain the beginning of the file, up to the first null byte. The command substitution also strips all final newlines from the file. For example, if the file consists of the four bytes (a, newline, newline, newline), then foo contains just a, same as if the file only contained that one byte.

If you want to preserve final newlines, arrange for the command to output an extra non-newline character, then strip it off.

foo=$(cat file; echo a)
foo=${foo%a}

So far this all worked in all Bourne/POSIX-style shells (ash, ksh, bash, …). Bash ≥4.0 has a faster way of reading a file into a variable: the mapfile.

mapfile <file foo

There simply isn't a way to stuff a null byte into a bash variable. If you want this, you have several options:

  • Use zsh.
  • Use Perl.
  • Store the data in a temporary file (which seems useless here, as the data is already in a file).
  • Encode the input and decode it back, e.g. with base64 and base64 -d.
share|improve this answer
    
why does gist.github.com/cirosantilli/9737428 fail with: fail var: 61 0a GNU bash, version 4.2.25? –  Ciro Santilli Mar 24 at 10:28
1  
@CiroSantilli S is an array. "$S" is the first element of the array in ksh and bash. It contains the first line: a and a newline. ${S[1]} contains the second line: a newline. ${#S[@]} is the array length which is 2. –  Gilles Mar 24 at 13:44
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.