Tag Info

Hot answers tagged

76

In appearance, dd is a tool from an IBM operating system that's retained its foreign appearance (its parameter passing), which performs some very rarely-used functions (such as EBCDIC to ASCII conversions or endianness reversal… not a common need nowadays). I used to think that dd was faster for copying large blocks of data on the same disk (due to more ...


54

Although both commands allow you to view the content of a file, their original purposes are quite different. less extends the capabilities of more. The latter was created to view the content of a file one screenful at a time. less adds features such as backward movements and better memory management (no need to read the entire file before being able to see ...


25

Whether such output can be exploited depends on the terminal program, and what that terminal does depending on escape codes that are being sent. I am not aware of terminal programs having such exploitable features, and the only problem now would be if there is an unknown buffer overflow or something like that, that could be exploited. With some older ...


25

Most terminal emulators will send back some response, if they receive certain escape sequences (have a look at the xterm control sequences documentation). E.g., you can send \e[0c to a VT100-like emulator and it will send back the device attributes, something like \e[?1;2c (This is probably what Keith observed.) But these answers are not arbitrary strings. ...


21

The "definitive" answer is of course brought to you by The Useless Use of cat Award. The purpose of cat is to concatenate (or "catenate") files. If it's only one file, concatenating it with nothing at all is a waste of time, and costs you a process. Instantiating cat just so your code reads differently makes for just one more process and one more set ...


21

You cannot tell cat to use multiple standard out that way, the last redirection takes precedence so cat file1.txt >> file2.txt >> file1.txt is equivalent to >> file2.txt ; cat file1.txt >> file1.txt which obviously quickly fills the file system, given the fact the source file being the destination too grows indefinitely as long ...


19

It may be useful to explain how files work at the lowest level: A file is a stream of bytes, zero or more in length. A byte is 8 bits. Since there are 256 combinations of 8 bits, that means a byte is any number from 0 to 255. So every file is, at its lowest level, a big hunk of numbers ranging from 0 to 255. It is completely up to programs and users to ...


18

Try it with -f or --force: zcat -f -- * man zcat for details. Also: so that I can pipe the output to grep for example You have zgrep for that: zgrep -- PATTERN *


17

You can use a while loop with process substitution: while read -r line do echo "$line" done < <(jobs) To read a multiline variable, a simple way is: printf %s "$var" | while IFS= read -r line do echo "$line" done Also, please don't call your variable jobs because that is a shell command and may cause confusion.


16

You are looking for fold text.txt -w 80 -s -w tells the width of the text, where 80 is standard. -s tells to break at spaces, and not in words. That's the way it's called on debian/ubuntu there are other systems, which need "-c" instead of "-w".


16

I suggest the sed solution, but for the sake of completeness, awk 'NR >= 57890000 && NR <= 57890010' To cut out after the last line: awk 'NR < 57890000 { next } { print } NR == 57890010 { exit } Speed test: 100,000,000-line file generated by seq 100000000 > test.in Reading lines 50,000,000-50,000,010 Tests in no particular ...


16

Allocating the space for the output file first may improve the overall speed as the system won't have to update the allocation for every write. For instance, if on Linux: size=$({ find . -maxdepth 1 -type f -name 'input_file*' -printf '%s+'; echo 0;} | bc) fallocate -l "$size" out && find . -maxdepth 1 -type f -name 'input_file*' -print0 | sort ...


15

Nope, cat is surely the best way to do this. Why use python when there is a program already written in C for this purpose? However, you might however want to consider using xargs in case the command line length exceeds ARG_MAX and you need more than one cat. Using GNU tools, this is equivalent to what you already have: find . -maxdepth 1 -type f -name ...


14

You don't need to pipe a file thru grep, grep takes filename(s) as command line args. grep -v '^#' file1 file2 file3 will print all lines EXCEPT those that begin with a # char. you can change the comment char to whatever you wish. If you have more than one comment char (assuming its at the beginning of a line) egrep -v '^(;|#|//)' filelist


14

Yes, it is possible. But not all formats support it. ffmpeg FAQ: A few multimedia containers (MPEG-1, MPEG-2 PS, DV) allow to join video files by merely concatenating them. When converting to RAW formats you also have a high chance that the files can be concatenated. ffmpeg -i input1.avi -qscale:v 1 intermediate1.mpg ffmpeg -i input2.avi -qscale:v 1 ...


14

I usually use the column program for this, it's in a package called bsdmainutils on Debian: column -t foo Output: case elems meshing nlsys uniform 2350 0.076662 2.78 non-conformal 348 0.013332 0.55 scale 318 0.013333 0.44 smarter 504 0.016666 0.64 submodel 360 .009999 0.40 unstruct-quad 640 ...


14

You are right in that echo & company don't seem to handle binary that well. I suspect that the null characters break the stream all too early. You can convert picture information in some ASCII based format. For instance, this is with base64: $ pic=`base64 pic.jpeg` $ echo $pic | base64 --decode > pic2.jpeg $ diff pic* $ echo $? 0


13

Yes. It has to do with ^D really does: it just stops the current read(2) call. If the program does int rdbytes = read(fd, buffer, sizeof buffer); and you press ^D inbetween, read() returns with the currently read bytes in the buffer, returning their number. The same happens on line termination; the \n at the end is always delivered. So only a ^D at the ...


13

The dd command includes LOTS of options that cat is not able to accommodate. Perhaps in your usage cases cat is a workable substitute, but it is not a dd replacement. One example would be using dd to copy part of something but not the whole thing. Perhaps you want to rip out some of the bits from the middle of an iso image or the partition table from a hard ...


13

The first command here emulates the formatting you see in vim. It intelligently expands tabs to the equivalent number of spaces, based on a tab-STOP (ts) setting of every 4 columns. printf "ab\tcd\tde\n" |expand -t4 Output ab cd de To keep the tabs as tabs and have the tab STOP positions set to every 4th column, then you must change the way ...


13

There are multiple ways to do this. The simplest is probably this: cat <<EOF | sh touch somefile echo foo > somefile EOF Another, which is nicer syntax in my opinion: ( cat <<EOF touch somefile echo foo > somefile EOF ) | sh This works as well, but without the subshell: { cat <<EOF touch somefile echo foo > somefile EOF } | ...


12

AProgrammer's suggestion of using xargs is often best, but another option is to use redirection into a while loop, which allows additional commands to be made and variables to be set: while read -r dir; do mkdir $dir; done < myfile An example of a more complicated structure would be: now=`date +%Y%m%d.%H%M%S` while read -r dir; do ...


11

No one has yet mentioned that you can use dd to create sparse files, though truncate can also be used for the same purpose. dd if=/dev/zero of=sparse-file bs=1 count=1 seek=10GB This is almost instant and creates an arbitrary large file that can be used as a loopback file for instance: loop=`losetup --show -f sparse-file` mkfs.ext4 $loop mkdir myloop ...


11

An ISO file is a complete, formatted filesystem image. All cat or dd does is do a bit-for-bit copy of that filesystem image to your target media. There is no magic going on behind the scenes. The ISO filesystem preparation was done beforehand (often by a specialized tool). All cat does is write that collection of bytes out. It doesn't interpret the .iso at ...


11

Those trailing newlines are added by nano, not by cat. Use nano's -L parameter: -L (--nonewlines) Don't add newlines to the ends of files. Or ~/.nanorc's nonewlines command: set/unset nonewlines Don't add newlines to the ends of files.


11

To concatenate files you use cat file1 file2 file3 ... To get a list of quoted filenames sorted by time, newest first, you use ls -t Putting it all together, cat $(ls -t) > outputfile You might want to give some arguments to ls (eg, *.html). But if you have filenames with spaces in them, this will not work. My file.html will be assumed to be two ...


11

The less than and symbol (<) is opening the file up and attaching it to the standard input device handle of some application/program. But you haven't given the shell any application to attach the input to. Example These 2 examples do essentially the same thing but get their input in 2 slightly different manners. opens file $ cat blah.txt hi opens ...


10

To process the output of a command line by line (explanation): jobs | while IFS= read -r line; do process "$line" done If you have the data in a variable already: printf %s "$foo" | … printf %s "$foo" is almost identical to echo "$foo", but prints $foo literally, whereas echo "$foo" might interpret $foo as an option to the echo command if it begins ...


10

Run reset. From the man page: When invoked as reset, tset sets cooked and echo modes, turns off cbreak and raw modes, turns on newline translation and resets any unset special characters to their default values before doing the terminal initialization described above. This is useful after a program dies leaving a terminal in an abnormal state.


10

I personally prefer view for static content or tail -f for dynamic content. This does not answer your question, though. There is a saying "why use more if you have less" ;-) But there are cases where I prefer cat to less: I usually work with X11-windows. These windows have a scroll-buffer which can be set to some hundred lines. Doing a cat for - let's say ...



Only top voted, non community-wiki answers of a minimum length are eligible