String manipulation: extracting a part of a string, text replacement, formatting to a given width, etc.

learn more… | top users | synonyms

58
votes
11answers
124k views

Delete the last character of a string using string manipulation in shell script

I would like to delete the last character of a string, I tried this little script : #! /bin/sh t="lkj" t=${t:-2} echo $t but it prints "lkj", what I am doing wrong? Thank you in advance!
52
votes
6answers
93k views

How can I test if a variable is empty or contains only spaces?

The following bash syntax verifies if param isn't empty: [[ ! -z $param ]] For example: param="" [[ ! -z $param ]] && echo "I am not zero" No output and its fine. But when param ...
28
votes
8answers
9k views

How to generate a random string?

I would like to generate a random string (e.g. passwords, user names, etc.). It should be possible to specify the needed length (e.g. 13 chars). What tools can I use? (For security and privacy ...
23
votes
4answers
77k views

Splitting string by the first occurrence of a delimiter

I have a string in the next format id;some text here with possible ; inside and want to split it to 2 strings by first occurrence of the ;. So, it should be: id and some text here with possible ; ...
18
votes
4answers
112k views

how to concatenate strings in bash?

I need to concatenate 2 strings in bash, so that: string1=hello string2=world mystring=string1+string2 echo mystring should produce helloworld
18
votes
7answers
50k views

bash - replace space with new line

How can I replace spaces with new lines on an input like: /path/to/file /path/to/file2 /path/to/file3 /path/to/file4 /path/to/file5 etc... To obtain the following: /path/to/file /path/to/file2 /...
18
votes
1answer
44k views

bash - how to uppercase the command line argument?

I searched SO and found that to uppercase a string following would work str="Some string" echo ${str^^} But I tried to do a similar thing on a command-line argument, which gave me the following ...
18
votes
2answers
2k views

Is there something like JavaScript's “split()” in the shell?

It's very easy to use split() in JavaScript to break a string into an array. What about shell script? Say I want to do this: $ script.sh var1_var2_var3 When the user give such string ...
17
votes
7answers
11k views

How to check if $PWD is a subdirectory of a given path

E.g. check if $PWD is a subdirectory of /home. In other words I'm searching for a bash string operation to check if one string starts with another.
16
votes
2answers
32k views

grep on a variable

Lets say I have a variable line="This is where we select from a table." now I want to grep how many times does select occur in the sentence. grep -ci "select" $line I tried that, but it did not ...
13
votes
6answers
747 views

/bin/dash: check whether $1 is a number

What would be the best way to check whether $1 is an integer in /bin/dash ? In bash, I could do: [[ $1 =~ ^([0-9]+)$ ]] But that does not seem to be POSIX compliant and dash does not support that
12
votes
3answers
7k views

Add thousands separator in a number

In python re.sub(r"(?<=.)(?=(?:...)+$)", ",", stroke ) To split a number by triplets, e.g.: echo 123456789 | python -c 'import sys;import re; print re.sub(r"(?<=.)(?=(?:...)+$)", ",", ...
12
votes
1answer
1k views

When can I edit strings in an executable binary?

I have an executable binary; let's call it a.out. I can see the binary contains strings $ strings a.out ... /usr/share/foo .... I need to change the string /usr/share/foo to /usr/share/bar. Can I ...
11
votes
3answers
754 views

How do you remove dot character from string without calling sed or awk again?

I have a file called hostlist.txt that contains text like this: host1.mydomain.com host2.mydomain.com anotherhost www.mydomain.com login.mydomain.com somehost host3.mydomain.com I have the ...
11
votes
3answers
2k views

Get all possible combinations of a word in lower/uppercase letters

I want to write a bash script to print all possible lower and upper case permutations of a certain word, e.g. harley: harley harleY harlEy harLey ... HARLey HARLEy HARLEY My naive solution is to ...
11
votes
7answers
37k views

How to get the first word of a string?

When I 'echo *' I get the following output: file1 file2 file3 ... What I want is to pick out the first word. How can I proceed?
10
votes
3answers
16k views

Case-insensitive substring search in a shell script [closed]

How can I write bash code that will do a case-insensitive substring match of command output?
10
votes
5answers
4k views

Pattern matching on path names in bash

I want to act on a list of subdirectories in a directory. Consider: for x in x86-headers/*/C/populate.sh; do echo $x; done This gives x86-headers/elf/C/populate.sh x86-headers/gl/C/populate.sh x86-...
9
votes
7answers
4k views

Bash - Continuous String Manipulation

#!/bin/bash FILE="$(basename "$1")" FILE="${FILE/%.jpeg/.jpg}" Is there anyway to glue these two lines together into a one-liner?
9
votes
4answers
2k views

How do I find the overlap of two strings in bash? [closed]

I have two strings. For the sake of the example they are set like this: string1="test toast" string2="test test" What I want is to find the overlap starting at the beginning of the strings. With ...
9
votes
5answers
2k views

Number of characters in a shell command's output

I am writing a script which needs to calculate the number of characters in a command's output in a single step. For example, using the command readlink -f /etc/fstab should return 10 because the ...
9
votes
3answers
11k views

How to defer variable expansion

I was wanting to initialize some strings at the top of my script with variables that have no yet been set, such as: str1='I went to ${PLACE} and saw ${EVENT}' str2='If you do ${ACTION} you will ${...
8
votes
2answers
20k views

Extracting a string, according to a pattern, in a bash script

In bash, suppose that I have a string strname: strname="ph7go04325r" I would like to extract the characters between the first "3" character and the last "r" character in strname, saving the result ...
8
votes
4answers
1k views

How to assert that a string has a newline character and, if so, remove it

I have a string that is the result of some operation I have no control over. When I print this variable using echo, I get: echo $myvar hello However, when I do if [ $myvar = "hello" ]; then ...
8
votes
3answers
16k views

How to discover a new line using a for loop?

In various places around the web I've found: \015 \012 \x0a - hex \n \r all as synonyms for various newlines / carriage returns... But in this small script I cannot recognise when I come across a ...
7
votes
7answers
24k views

How to find a position of a character using grep?

I need to identify the postion of a character in string using grep command. Example, the string is RAMSITALSKHMAN|1223333. grep -n '[^a-zA-Z0-9\$\~\%\#\^]' How do I find the position of | in the ...
7
votes
3answers
2k views

How to remove a specific character in a string but only if there are no numbers in that line in Linux

I've been stumped with this seemingly simple-to-fix problem.. well, for a while. Here's example output from the file I need to edit: $cat file George Washington Geneva Convention 123,281,029 USD 342,...
7
votes
2answers
5k views

grep surrounding characters of a match

I'm looking to do a find and replace within a giant database dump, and it's not doing what I think should happen. I'd like to grep for my target string in the file, and then see the surrounding 8 ...
7
votes
2answers
5k views

JSON parsing on the shell

How can I parse JSON output on the shell? For example, Amazon Web Services provides a CLI to retrieve the status of your instances: $ aws ec2 describe-instances <my_instance_id> But the ...
7
votes
6answers
17k views

bash find lines starting with string

I have a bunch of files and I want to find which one contains sequential lines starting with a certain string. For example for the following file : Aaaaaaaaaaaa Baaaaaaaaaaa Cxxxxxxxxx Cyyyyyyyyy ...
7
votes
3answers
4k views

Replace multiple strings in a single pass

I'm looking for a way to replace placeholder strings in a template file with concrete values, with common Unix tools (bash, sed, awk, maybe perl). It is important that the replacement is done in a ...
6
votes
5answers
1k views

Replace string after last dot in bash

I have the following pattern in a string (an IP address): 123.444.888.235 I want to replace the last number after the dot with 0, so it becomes: 123.444.888.0 How could I do it in bash or ...
6
votes
5answers
636 views

How to abbreviate /path/to/file to /p/t/file

I'm looking for an elegant one-liner (eg, awk) that will shorten a string of a Unix path using the first character of each parent/intermediate level, but the full basename. Easier to show by examples:...
6
votes
2answers
2k views

Piping bash string manipulation

I've read some other piping bash string manipulation questions but they seem to be specialized applications. Essentially, is there a way to do the below simpler? instead of $ string='hello world'; ...
6
votes
4answers
4k views

Split single string into character array using ONLY bash

I want to split 'hello' into h e l l o in an array using only bash, I could do it in sed with sed 's/./& /g' but I want to know how to split a string into an array in Bash when I do not know what ...
6
votes
3answers
930 views

Change case of n-th letter in a string

I want to change the case of the n-th letter of a string in BASH (or any other *nix tools, e.g. sed, awk, tr, etc). I know that you can change the case a whole string using: ${str,,} # to lowercase $...
5
votes
4answers
18k views

How to capture error message from executed command?

I was tasked to create an automated server hardening script and one thing that they need is a report of all the output of each command executed. I want to store the error message inside a string and ...
5
votes
6answers
10k views

How to reverse a string made of digit in bash?

What is the best way to turn out digits in number? E.g. $ echo 123 | hook 321 $ echo 12358 | hook 85321
5
votes
2answers
3k views

Utility to get binary representation from string?

Is there a simple utility like printf that'll give me a "binary representation", i.e. a string like "01010010", from an input string? It would need to support multi-byte encodings.
5
votes
6answers
43k views

Extract date from a variable in a different format

Let me explain you the problem $ date +%c -d "$d" Tue 31 Dec 2013 01:13:06 PM CET $ date +'Today is %F' -d "$d" Today is 2013-12-31 This solution corresponds to current date. But I have one ...
5
votes
3answers
1k views

Get last part of string after hyphen

Is there a simple command line to extract the last part of a string separated by hyphens? E.g., I want to extract 123 from foo-bar-123.
5
votes
1answer
271 views

Test -n gives unexpected result

I understand test -n <expression> to return false if the expression evaluates to a string length of greater than 0. Why then does the following happen? Macbook:~ echo ${#undeclared_variable} 0 ...
5
votes
6answers
21k views

How to split a string into an array in bash

I have a problem with the output of a program. I need to launch a command in bash and take its output (a string) and split it to add new lines in certain places. The string looks like this: battery....
5
votes
3answers
7k views

Most common encoding for strings in C++ in Linux (and Unix?)

For creating a C++ program that is source code level portable between Windows and Linux and handles internationalization well, there are IMHO three main encodings to consider: The encoding of the C++...
5
votes
6answers
7k views

How to match exact string using `sed`? But not the part of it.?

I have a input file FILE1.TXT as below. 11 id1 12 13 AGE = 20 14 NAME = NAME1 15 16 id2 17 18 AGE = 30 19 NAME = NAME2 . . . 110 idXYZ 111 112 AGE = AGEXYZ 113 NAME = ...
5
votes
2answers
3k views

Splitting bash command line argument

Is this the best way to split up a colon separated bash command line argument? #!/bin/bash hostlist=`echo $1| awk '{split($0,Ip,":")} END{for (var in Ip) print Ip[var];}'` for host in $hostlist do ...
5
votes
3answers
2k views

Is there a tr for translating into longer strings?

tr is great for translating a set of characters to another set of characters. Sometimes, I want to translate a set of characters to a set of strings, e.g. all < and > to &lt; and &gt;. ...
5
votes
3answers
337 views

Go from a String to an Array of “words” in Bash

I need to go from a string to an array where each entry is each word on that string. For example, starting with: VotePedro="Vote for Pedro" I need the array: Vote For Pedro Which I should then ...
5
votes
3answers
218 views

how do I set quotes around a variable so that the programs sees them as quote marks

I am trying to get quotes around a variable to make is just like I typed it in the terminal to get this script to work. it shows " quotes around the varibale " but still does not see it as quotes, as ...
5
votes
2answers
893 views

ls for cut -f string not working

I have a problem with feeding cut output to ls. This doesn't work: rep="S1_1000,0000-00-00c,0000-00-00d,0000-00-00e" ls tab_yeast/{`echo $rep | cut -f2- -d','`}*.tab.gz ls: cannot access ...