Questions tagged [shell-script]

Questions about shell scripts, executable files that are interpreted by a shell (bash, zsh, etc.).

Filter by
Sorted by
Tagged with
6
votes
1answer
58k views

convert CSV to XLS file on linux [closed]

The following Perl script can convert CSV file to XLS file the problem is that I need to install on customer Linux machine many Perl modules in order to run this Perl script , and actually I cant ...
25
votes
5answers
40k views

Delete range of lines above pattern with sed (or awk)

I have the following code that will remove lines with the pattern banana and 2 lines after it: sed '/banana/I,+2 d' file So far, so good! But I need it to remove 2 lines before banana, but I can't ...
25
votes
5answers
20k views

Bash in Linux v.s Mac OS

I am buying an iMac. I've been working on Linux for a long time and I do a bunch of bash scripting. I was wondering if all the bash commands and features are available on Mac OS ? Anyone has any ...
41
votes
10answers
113k views

Check if folder is a mounted remote filesystem

What is the best way (reliable, portable, etc.) to check if a given folder is on a mounted remote (nfs) filesystem within a shell script? I am looking for a command that would look like: chk-remote-...
33
votes
2answers
43k views

Joining bash arguments into single string with spaces

I'm trying to join all of the arguments to a Bash function into one single string with spaces separating each argument. I also need to have the string include single quotes around the whole string. ...
25
votes
3answers
29k views

How to “send” variable to sub-shell?

I run the following script: VAR="Test" sh -c 'echo "Hello $VAR"' But I get : # ./test.sh Hello How can I "send" the variable VAR of my script to the shell created with sh -c?
24
votes
2answers
40k views

Fastest way to concatenate files

I've got 10k+ files totaling over 20GB that I need to concatenate into one file. Is there a faster way than cat input_file* >> out ? The preferred way would be a bash command, Python is ...
23
votes
3answers
21k views

Diff, show modified line from right file only

I have file a and b and I would like to output lines of b that changed since it was cloned from a. Just the modified lines, no surrounding context, no diff offset marks. How can I do that using shell ...
22
votes
9answers
30k views

Preserve directory structure when moving files using find

I have created the following script that move old days files as defined from source directory to destination directory. It is working perfectly. #!/bin/bash echo "Enter Your Source Directory" read ...
18
votes
8answers
78k views

Calling multiple bash scripts and running them in parallel, not in sequence

Suppose that I have three (or more) bash scripts: script1.sh, script2.sh, and script3.sh. I would like to call all three of these scripts and run them in parallel. One way to do this is to just ...
17
votes
3answers
95k views

Adding a Column of values in a tab delimited file

How can I add a Column of values in a file which has a certain number of rows. I have a input file like this: Input file: SPATA17 1 217947738 LYPLAL1 1 219383905 FAM47E 4 77192838 SHROOM3 4 ...
21
votes
2answers
106k views

Cron job to delete files older than 3 days

I need to remove files older than 3 days with a cron job in 3 different directories. (these 3 directories are children of a parent directory /a/b/c/1 & /a/b/c/2 & /a/b/c/3) Can this be done ...
17
votes
2answers
19k views

How to get the name of the user who executed a bash script as sudo?

I want to create a bash script that must be executed with sudo but should take into account the name of the non-sudo user who executed it. So if user bob runs sudo ./myscript.sh I would like myscript....
33
votes
1answer
2k views

How does !! work in bash?

Very useful when you forget a sudo at the beginning of your command, !! acts like an alias of the previous command. Example : $ mv /very/long/path/for/a/protected/sensible/file/caution.h . (...) ...
17
votes
6answers
81k views

How to check if a filesystem is mounted with a script

I am new at scripting ... I can do very basic stuff, but now I need a hand. I have a local filesystem that only will be mounted when I need to do a backup. I'm starting with this. #!/bin/bash ...
14
votes
6answers
5k views

Using “reserved” codes for exit status of shell scripts

I recently came across this list of Exit Codes With Special Meanings from the Advanced Bash-Scripting Guide. They refer to these codes as being reserved and recommend that: According to the above ...
13
votes
8answers
60k views

How to check if any IP address is present in a file using shell scripting?

I have a script that generates some output. I want to check that output for any IP address like 159.143.23.12 134.12.178.131 124.143.12.132 if (IPs are found in <file>) then // bunch of ...
37
votes
2answers
5k views

What does “${-#*i}” != “$-” mean?

In /etc/profile I see this: for i in /etc/profile.d/*.sh ; do if [ -r "$i" ]; then if [ "${-#*i}" != "$-" ]; then . "$i" else . "$i" >/dev/null 2>&1 ...
27
votes
4answers
3k views

Are if else statement equivalent to logical and && or || and where should I prefer one over the other?

I'm learning about decision making structures and I came across these codes: if [ -f ./myfile ] then cat ./myfile else cat /home/user/myfile fi [ -f ./myfile ] && cat ./myfile || ...
25
votes
3answers
24k views

How to sort the string which combined with string + numeric using bash script?

This is the data what I want to sort. But sort treats the numeric to string, the data it no sorted as I expected. /home/files/profile1 /home/files/profile10 /home/files/profile11 /home/files/...
19
votes
2answers
9k views

How do I test to see if an application exists in $PATH?

I'm trying to write all of my sh startup/env scripts to work with as much DRY and as much: "works on every *nix I clone it to", as possible. This means making sure that if I try to run code that's not ...
19
votes
2answers
1k views

Is there a POSIX way of setting zeroth argument of a target application?

In bash you can use exec -a and in zsh you can alternatively also set ARGV0 to execute a program with a certain zeroth argument but is there also a POSIX way of doing so? As suggested in this one ...
14
votes
1answer
64k views

proper way to iterate through contents in a directory [duplicate]

I need to iterate through every file inside a directory. One common way I saw was using the for loop that begins with for file in *; do. However, I realized that it does not include hidden files (...
12
votes
2answers
6k views

find and remove duplicates in a directory

I have a directory with multiple img files and some of them are identical but they all have different names. I need to remove duplicates but with no external tools only with a bash script. I'm a ...
10
votes
2answers
43k views

Create variables and assign values via loop (bash)

Is there a way to create bash variables and assign them values via a loop? Something along the lines of: #!/bin/bash c=0 for file in $( ls ); do var"$c"="$file"; let c=$c+1; done EDIT: ...
9
votes
1answer
8k views

bashscript to detect right arrow key being pressed

Why does this always detect as true, even if the keycode is not right-arrow-key? stty_state=`stty -g` stty raw; stty -echo keycode=`dd bs=1 count=1 2>/dev/null` stty "$stty_state" echo $keycode ...
7
votes
2answers
7k views

Create sub-directories and organize files by date

I have some directories of files copied from my security camera that I would like to organize into sub-directories by file date. So for example; -rwxrwxrwx 0 root root 4935241 Jul 19 2012 DSCN1406....
6
votes
4answers
8k views

Clone directory tree structure and copy files to the corresponding directories modified after a specific date

I have a folder with more than 30 sub directories and I want to get the list of the files which was modified after a specified date(say sep 8 which is the real case) and to be copied with the same ...
24
votes
3answers
80k views

Run script in a screen

I want to run a bash script in a detached screen. The script calls a program a few times, each of which takes too long to wait. My first thought was to simply open a screen and then call the script, ...
22
votes
4answers
21k views

sed with multiple expression for in-place arguement

I am trying to replace multiple words in file by using sed -i #expression1 #expression2 file Something 123 item1 Something 456 item2 Something 768 item3 Something 353 item4 Output (Desired) ...
21
votes
10answers
7k views

Shell: is it possible to delay a command without using `sleep`?

Are there any substitutes, alternatives or bash tricks for delaying commands without using sleep? For example, performing the below command without actually using sleep: $ sleep 10 && echo "...
18
votes
1answer
65k views

Syntax Error: unexpected end of file — Bash script [closed]

I am trying to create an spritz app. Everything was working fine, but since yesterday I keep getting this error: ./spritz: line 176: syntax error: unexpected end of file I have checked the script ...
12
votes
4answers
1k views

Use case / practical example for Bash' builtin exec

Consider this from the documentation of Bash' builtin exec: exec replaces the shell without creating a new process Please provide a use case / practical example. I don’t understand how this makes ...
9
votes
3answers
26k views

Bash script; optimization of processing speed

I was wondering if there are general guidelines for optimizing Bash scripts. For example, it is more convenient to write loops than lines of commands, but is it also faster to process for the system? ...
9
votes
5answers
3k views

Joining two files with unique identifier

I have two files with approximately 12900 and 4400 entries respectively, that I want to join. The files contain location information for all landbased weather observing stations around the globe. The ...
9
votes
3answers
22k views

BASH and Carriage Return Behavior

I have one quick question. Is it normal that bash (i am using 4.4.11) is not displaying lines/text that is separated / end with plain \r ? I was a bit surprised to see this behavior: $ a=$(printf "...
9
votes
6answers
6k views

Compare files that are in directory 1 but not directory 2?

I'm having trouble with a bash script I want to make I know ls will list files that are in a directory but I want it to list directories that are in directory1 but NOT in directory2, and then list ...
8
votes
4answers
1k views

Split an input for different command and combine the result

I know how to combine the result of different command paste -t',' <(commanda) <(commandb) I know pipe same input to different command cat myfile | tee >(commanda) >(commandb) Now how ...
7
votes
2answers
2k views

Difference of using () and $() to execute a series of commands

I am currently trying to make a script that creates bytes that will be piped as input to netcat. Here is the idea of the script: (perl -e "print \"$BYTES\x00\"; cat file; perl -e "print \"More ...
7
votes
3answers
13k views

Do I need to encapsulate awk variables in quotes in order to sanitize them?

As per an answer on stackoverflow, it's my understanding that encapsulating bash variables in double-quotes is a fairly safe way of sanitizing user input. What about awk variables? For example, if ...
26
votes
6answers
10k views

Convert underscore to PascalCase, ie UpperCamelCase

If I have a string that looks like this: "this_is_the_string" Inside a bash script, I would like to convert it to PascalCase, ie UpperCamelCase to look like this: "ThisIsTheString" I found that ...
22
votes
2answers
1k views

Any command in my terminal that exits with non-zero code closes my terminal window

At first this was a bit funny, like playing "Bash Roulette" ...but now it's getting old lol Any command in my terminal that exits with non-zero code closes my terminal window I was told that perhaps ...
17
votes
2answers
10k views

zsh: excluding files from a pattern

Say I have the following files: |-- bar `-- foo |-- type_A_1 |-- type_A_2 |-- type_B_1 |-- type_B_2 |-- type_B_xx |-- type_B_xx `-- something_else I thought the following ...
17
votes
3answers
12k views

Is there a way to convert a zip to a tar without extracting it to the filesystem?

Is there a way to convert a zip archive to a tar archive without extracting to a temporary directory first? (and without writing my own implementation of tar or unzip)
16
votes
4answers
17k views

How can I use a variable as a case condition?

I am trying to use a variable consisting of different strings separated with a | as a case statement test. For example: string="\"foo\"|\"bar\"" read choice case $choice in $string) echo "...
16
votes
8answers
15k views

cron Vs. sleep - which is the better one in terms of efficient cpu/memory utilization?

The Case: I need to run some commands/script at certain intervals of time and for this I have two options: set up a cron-job implement a loop with sleep in the script itself. Question: Which is ...
14
votes
4answers
24k views

find -exec in bash script with variable expansion

I'm trying to run a command similar to the one below in a bash script. It should search through all subfolders of $sourcedir and copy all files of a certain type to the root level of $targetdir. #!/...
14
votes
3answers
68k views

spawn - command not found!

I am using Mac OS X 10.9.4, following is my script to copy files from local machine to different host #!/bin/bash #!/usr/bin/expect echo "I will fail if you give junk values!!" echo " " echo "Enter ...
12
votes
3answers
4k views

How would I use GNU Parallel for this while loop?

So I have a while loop: cat live_hosts | while read host; do \ sortstuff.sh -a "$host" > sortedstuff-"$host"; done But this can take a long time. How would I use GNU Parallel for this while ...
10
votes
5answers
14k views

Delete spaces, hyphens, and underscores in filenames?

What is a good command to delete spaces, hyphens, and underscores from all files in a directory, or selected files? I use the following command with Thunar Custom Actions to slugify filenames: for ...