String manipulation: extracting a part of a string, text replacement, formatting to a given width, etc.

learn more… | top users | synonyms

55
votes
11answers
116k views

Delete the last character of a string using string manipulation in shell script

I would like to delete the last character of a string, I tried this little script : #! /bin/sh t="lkj" t=${t:-2} echo $t but it prints "lkj", what I am doing wrong? Thank you in advance!
48
votes
6answers
85k views

How can I test if a variable is empty or contains only spaces?

The following bash syntax verifies if param isn't empty: [[ ! -z $param ]] For example: param="" [[ ! -z $param ]] && echo "I am not zero" No output and its fine. But when param ...
27
votes
8answers
8k views

How to generate a random string?

I would like to generate a random string (e.g. passwords, user names, etc.). It should be possible to specify the needed length (e.g. 13 chars). What tools can I use? (For security and privacy ...
23
votes
4answers
74k views

Splitting string by the first occurrence of a delimiter

I have a string in the next format id;some text here with possible ; inside and want to split it to 2 strings by first occurrence of the ;. So, it should be: id and some text here with possible ; ...
18
votes
7answers
48k views

bash - replace space with new line

How can I replace spaces with new lines on an input like: /path/to/file /path/to/file2 /path/to/file3 /path/to/file4 /path/to/file5 etc... To obtain the following: /path/to/file /path/to/file2 ...
18
votes
2answers
2k views

Is there something like JavaScript's “split()” in the shell?

It's very easy to use split() in JavaScript to break a string into an array. What about shell script? Say I want to do this: $ script.sh var1_var2_var3 When the user give such string ...
17
votes
4answers
106k views

how to concatenate strings in bash?

I need to concatenate 2 strings in bash, so that: string1=hello string2=world mystring=string1+string2 echo mystring should produce helloworld
17
votes
7answers
11k views

How to check if $PWD is a subdirectory of a given path

E.g. check if $PWD is a subdirectory of /home. In other words I'm searching for a bash string operation to check if one string starts with another.
16
votes
2answers
29k views

grep on a variable

Lets say I have a variable line="This is where we select from a table." now I want to grep how many times does select occur in the sentence. grep -ci "select" $line I tried that, but it did not ...
15
votes
1answer
42k views

bash - how to uppercase the command line argument?

I searched SO and found that to uppercase a string following would work str="Some string" echo ${str^^} But I tried to do a similar thing on a command-line argument, which gave me the following ...
13
votes
6answers
723 views

/bin/dash: check whether $1 is a number

What would be the best way to check whether $1 is an integer in /bin/dash ? In bash, I could do: [[ $1 =~ ^([0-9]+)$ ]] But that does not seem to be POSIX compliant and dash does not support that
12
votes
3answers
7k views

Add thousands separator in a number

In python re.sub(r"(?<=.)(?=(?:...)+$)", ",", stroke ) To split a number by triplets, e.g.: echo 123456789 | python -c 'import sys;import re; print re.sub(r"(?<=.)(?=(?:...)+$)", ",", ...
12
votes
1answer
1k views

When can I edit strings in an executable binary?

I have an executable binary; let's call it a.out. I can see the binary contains strings $ strings a.out ... /usr/share/foo .... I need to change the string /usr/share/foo to /usr/share/bar. Can I ...
11
votes
3answers
2k views

Get all possible combinations of a word in lower/uppercase letters

I want to write a bash script to print all possible lower and upper case permutations of a certain word, e.g. harley: harley harleY harlEy harLey ... HARLey HARLEy HARLEY My naive solution is to ...
11
votes
7answers
35k views

How to get the first word of a string?

When I 'echo *' I get the following output: file1 file2 file3 ... What I want is to pick out the first word. How can I proceed?
10
votes
3answers
666 views

How do you remove dot character from string without calling sed or awk again?

I have a file called hostlist.txt that contains text like this: host1.mydomain.com host2.mydomain.com anotherhost www.mydomain.com login.mydomain.com somehost host3.mydomain.com I have the ...
10
votes
5answers
4k views

Pattern matching on path names in bash

I want to act on a list of subdirectories in a directory. Consider: for x in x86-headers/*/C/populate.sh; do echo $x; done This gives x86-headers/elf/C/populate.sh x86-headers/gl/C/populate.sh ...
9
votes
3answers
16k views

Case-insensitive substring search in a shell script [closed]

How can I write bash code that will do a case-insensitive substring match of command output?
9
votes
7answers
4k views

Bash - Continuous String Manipulation

#!/bin/bash FILE="$(basename "$1")" FILE="${FILE/%.jpeg/.jpg}" Is there anyway to glue these two lines together into a one-liner?
9
votes
4answers
2k views

How do I find the overlap of two strings in bash? [closed]

I have two strings. For the sake of the example they are set like this: string1="test toast" string2="test test" What I want is to find the overlap starting at the beginning of the strings. With ...
9
votes
5answers
1k views

Number of characters in a shell command's output

I am writing a script which needs to calculate the number of characters in a command's output in a single step. For example, using the command readlink -f /etc/fstab should return 10 because the ...
9
votes
3answers
10k views

How to defer variable expansion

I was wanting to initialize some strings at the top of my script with variables that have no yet been set, such as: str1='I went to ${PLACE} and saw ${EVENT}' str2='If you do ${ACTION} you will ...
8
votes
2answers
19k views

Extracting a string, according to a pattern, in a bash script

In bash, suppose that I have a string strname: strname="ph7go04325r" I would like to extract the characters between the first "3" character and the last "r" character in strname, saving the result ...
8
votes
4answers
1k views

How to assert that a string has a newline character and, if so, remove it

I have a string that is the result of some operation I have no control over. When I print this variable using echo, I get: echo $myvar hello However, when I do if [ $myvar = "hello" ]; then ...
8
votes
3answers
16k views

How to discover a new line using a for loop?

In various places around the web I've found: \015 \012 \x0a - hex \n \r all as synonyms for various newlines / carriage returns... But in this small script I cannot recognise when I come across a ...
7
votes
7answers
23k views

How to find a position of a character using grep?

I need to identify the postion of a character in string using grep command. Example, the string is RAMSITALSKHMAN|1223333. grep -n '[^a-zA-Z0-9\$\~\%\#\^]' How do I find the position of | in the ...
7
votes
3answers
2k views

How to remove a specific character in a string but only if there are no numbers in that line in Linux

I've been stumped with this seemingly simple-to-fix problem.. well, for a while. Here's example output from the file I need to edit: $cat file George Washington Geneva Convention 123,281,029 USD ...
7
votes
2answers
5k views

grep surrounding characters of a match

I'm looking to do a find and replace within a giant database dump, and it's not doing what I think should happen. I'd like to grep for my target string in the file, and then see the surrounding 8 ...
7
votes
2answers
5k views

JSON parsing on the shell

How can I parse JSON output on the shell? For example, Amazon Web Services provides a CLI to retrieve the status of your instances: $ aws ec2 describe-instances <my_instance_id> But the ...
7
votes
6answers
16k views

bash find lines starting with string

I have a bunch of files and I want to find which one contains sequential lines starting with a certain string. For example for the following file : Aaaaaaaaaaaa Baaaaaaaaaaa Cxxxxxxxxx Cyyyyyyyyy ...
7
votes
3answers
4k views

Replace multiple strings in a single pass

I'm looking for a way to replace placeholder strings in a template file with concrete values, with common Unix tools (bash, sed, awk, maybe perl). It is important that the replacement is done in a ...
6
votes
5answers
948 views

Replace string after last dot in bash

I have the following pattern in a string (an IP address): 123.444.888.235 I want to replace the last number after the dot with 0, so it becomes: 123.444.888.0 How could I do it in bash or ...
6
votes
5answers
631 views

How to abbreviate /path/to/file to /p/t/file

I'm looking for an elegant one-liner (eg, awk) that will shorten a string of a Unix path using the first character of each parent/intermediate level, but the full basename. Easier to show by ...
6
votes
2answers
2k views

Piping bash string manipulation

I've read some other piping bash string manipulation questions but they seem to be specialized applications. Essentially, is there a way to do the below simpler? instead of $ string='hello world'; ...
6
votes
3answers
903 views

Change case of n-th letter in a string

I want to change the case of the n-th letter of a string in BASH (or any other *nix tools, e.g. sed, awk, tr, etc). I know that you can change the case a whole string using: ${str,,} # to lowercase ...
6
votes
4answers
3k views

Split single string into character array using ONLY bash

I want to split 'hello' into h e l l o in an array using only bash, I could do it in sed with sed 's/./& /g' but I want to know how to split a string into an array in Bash when I do not know what ...
5
votes
4answers
17k views

How to capture error message from executed command?

I was tasked to create an automated server hardening script and one thing that they need is a report of all the output of each command executed. I want to store the error message inside a string and ...
5
votes
6answers
9k views

How to reverse a string made of digit in bash?

What is the best way to turn out digits in number? E.g. $ echo 123 | hook 321 $ echo 12358 | hook 85321
5
votes
2answers
3k views

Utility to get binary representation from string?

Is there a simple utility like printf that'll give me a "binary representation", i.e. a string like "01010010", from an input string? It would need to support multi-byte encodings.
5
votes
3answers
1k views

Get last part of string after hyphen

Is there a simple command line to extract the last part of a string separated by hyphens? E.g., I want to extract 123 from foo-bar-123.
5
votes
1answer
269 views

Test -n gives unexpected result

I understand test -n <expression> to return false if the expression evaluates to a string length of greater than 0. Why then does the following happen? Macbook:~ echo ${#undeclared_variable} 0 ...
5
votes
6answers
21k views

How to split a string into an array in bash

I have a problem with the output of a program. I need to launch a command in bash and take its output (a string) and split it to add new lines in certain places. The string looks like this: ...
5
votes
3answers
7k views

Most common encoding for strings in C++ in Linux (and Unix?)

For creating a C++ program that is source code level portable between Windows and Linux and handles internationalization well, there are IMHO three main encodings to consider: The encoding of the ...
5
votes
6answers
7k views

How to match exact string using `sed`? But not the part of it.?

I have a input file FILE1.TXT as below. 11 id1 12 13 AGE = 20 14 NAME = NAME1 15 16 id2 17 18 AGE = 30 19 NAME = NAME2 . . . 110 idXYZ 111 112 AGE = AGEXYZ 113 NAME = ...
5
votes
2answers
3k views

Splitting bash command line argument

Is this the best way to split up a colon separated bash command line argument? #!/bin/bash hostlist=`echo $1| awk '{split($0,Ip,":")} END{for (var in Ip) print Ip[var];}'` for host in $hostlist do ...
5
votes
3answers
2k views

Is there a tr for translating into longer strings?

tr is great for translating a set of characters to another set of characters. Sometimes, I want to translate a set of characters to a set of strings, e.g. all < and > to &lt; and &gt;. ...
5
votes
3answers
290 views

Go from a String to an Array of “words” in Bash

I need to go from a string to an array where each entry is each word on that string. For example, starting with: VotePedro="Vote for Pedro" I need the array: Vote For Pedro Which I should then ...
5
votes
3answers
203 views

how do I set quotes around a variable so that the programs sees them as quote marks

I am trying to get quotes around a variable to make is just like I typed it in the terminal to get this script to work. it shows " quotes around the varibale " but still does not see it as quotes, as ...
5
votes
2answers
881 views

ls for cut -f string not working

I have a problem with feeding cut output to ls. This doesn't work: rep="S1_1000,0000-00-00c,0000-00-00d,0000-00-00e" ls tab_yeast/{`echo $rep | cut -f2- -d','`}*.tab.gz ls: cannot access ...
5
votes
1answer
316 views

Escaping strings in associative arrays (bash)

I am writing a script which executes a number of programs with arguments. To simplify it as much as possible, consider the following example: programs=( "ls" "echo" ) declare -A parameters ...