Skip to main content

Questions tagged [text-processing]

Manipulation or examining of text by programs, scripts, etc.

Filter by
Sorted by
Tagged with
-3 votes
2 answers
75 views

Using awk, append a value in last field in csv file based upon value in specific field

input.csv - "family_guy","Brian","b47f0a80-d848-4d81-a45b-7ba930e6048b","son" "family_guy","Brian","b47f0a80-d848-4d81-a45b-...
Raman Kathpalia's user avatar
3 votes
7 answers
823 views

how to round numbers after comma everytime up

I have a file on a Linux machine, with one number per line: 3073771824 1513517589 4982173058 1539400944 3175320163 5247018359 14359014635 .... How can I multiply each by 1.03 (3%) and then ...
user447274's user avatar
1 vote
2 answers
58 views

Need to make multiple multiline replacements in file

I have a file with multiple "paragraphs" like this: <type>TRANS</type> <attributes/> <specification_method>rep_name</specification_method> <trans_object_id/&...
Todd Porter's user avatar
2 votes
5 answers
272 views

Grouping rows by categories avoiding repetition

I have a tab-separated file with two columns on a Linux machine. The first column contains names, the second column contains GO IDs (these are always of the format GO: followed by seven digits) ...
rseg's user avatar
  • 35
3 votes
4 answers
561 views

Cut and replace Nth character on every row

I have predictable piped input, I want to iterate over each row and change two characters. the character positions are 19 and 20 (on every row.) The 19th character is a comma, I want to cut that. The ...
XJMZX's user avatar
  • 75
5 votes
5 answers
754 views

how to format a text file in bash with dots from right

I have a file on a Linux system with various numbers, all on a single line (numbers_in_one_line.txt): 1941102556 1750145810 2604905 7000793682 5160065824 3000350768 6449300295 3046118928 12693055728 ...
user447274's user avatar
0 votes
3 answers
63 views

Pass output of command line to awk variable

I am trying to normalise a data file using the number of lines in a previous version of the data file. After reading these questions, I thought this could work: awk -v num=$(wc -l my_first_file.bed) '{...
Whitehot's user avatar
  • 145
1 vote
1 answer
84 views

Print csv columns according to user input

I have a .csv file with multiple lines with the following format: ¬Country¬,¬Year¬,¬Singer¬,¬Song Title¬ ¬Japan¬,¬1999¬,¬Utada Hikaru¬,¬First Love¬ ¬South Korea¬,¬1999¬,¬Lee Jung Hyun¬,¬Wa¬ ... I can ...
baiserekku's user avatar
-2 votes
1 answer
104 views

How to replace a parameter's value in json file with a value taken from a different text file

I have a a large number of JSON files on a Linux machine, where I need to change the value of a parameter with value taken from a different text file. E.g.: If the JSON file has values like: { "...
vishal's user avatar
  • 97
-4 votes
3 answers
133 views

Pattern match using awk

I have an input file as below Col1|Col2|Col3|Col4 abc123|654565|abc|apple abc123|654565|pir|orange abc123|654565|val|plum I want to select all the lines that have Col2=654565 and Col3=abc and print ...
gdba's user avatar
  • 1
-1 votes
1 answer
52 views

how to omit/ignore/remove lines starting with local/ using awk

Variable _RESULT contains an output of pacinfo --verbose local/jre. Using awk script (by Ed Morton - thanks to him!): awk '{ match($0,/:[[:space:]]*/) nextTag = substr($0,1,RSTART-1) ...
ludvick's user avatar
  • 25
0 votes
2 answers
44 views

From hexdump to ascii (xxd -r stuck writing zero bytes)

To diagnose secure boot issues, I'm looking at the sbat sections of all bootloaders on a machine: sudo sh -c 'for f in /boot/efi/EFI/*/*.efi; do echo "$f"; objdump -j .sbat -s "$f" ...
Tobu's user avatar
  • 6,630
1 vote
1 answer
87 views

How to modify a part of an output of pacinfo in bash?

Having an example output of pacinfo local/puddletag like this: Name: puddletag Base: puddletag Repository: local Version: 2.3.0-2 Description: An audio tag editor for ...
ludvick's user avatar
  • 25
0 votes
1 answer
81 views

How to search in a pdf or text file by a regex pattern and output locations of matches?

Is there a pdf viewer that allows me to search its text by a regex expression? In case that I haven't found one, I am thinking about extracting the text and layout from a pdf file by less my.pdf > ...
Tim's user avatar
  • 103k
0 votes
1 answer
42 views

Append the arguments to a script as lines in a file

How do you append $@ or $* , each as a new line to a file? I think I already searched for that before. Adding a new option -i --insert-newline with possible values 1, 2, 3 or Linux, Windows, Mac to ...
Laurent Lyaudet's user avatar
2 votes
3 answers
341 views

Extract Value from Kafka-Topics Command

Given the following output, how can I extract retention.ms's value of 3333? Ultimately I'd like to print out topic has retention of 3333 milliseconds from the output of the kafka-topics.sh ... command:...
Kevin Meredith's user avatar
-1 votes
2 answers
76 views

I need to filter out `grep`'s output to find a device's serial number, in order to store it into a variable

I'm working on a section of a script that will find the serial number of a Mac, store it into a variable and rename the device according to the naming standards of my workplace. So far, I have the ...
TheBigDog's user avatar
0 votes
1 answer
48 views

Add \t to jq resulted array extraction

I want to add \t before each element in the resulting jq array and this is for formatting purposes as the result will get added to a file. here is the json file: { "array": ["...
kyles's user avatar
  • 33
0 votes
5 answers
89 views

How can I replace the last occurrence of " before ] (text-manipulation)

I'm doing some text manipulation and I want to replace the last occurrence of " (with ", "four") before ] which could be in different lines. (this essentially adds a new item in ...
kyles's user avatar
  • 33
1 vote
5 answers
627 views

How to only print odd characters of a string in terminal?

Let's say I have a file with content "hheelllloo" and I want to clean it up and get "hello", how do I do this in bash? echo "hheelllloo" | <?>
d9ngle's user avatar
  • 357
-1 votes
6 answers
112 views

fastest way to repeat a list

I need do some speed test on my Linux commands. The input data is a list of words. For example: $ cat list elephant horse raven cat dog fish For a reasonable speed test I need to have a much larger ...
user167612's user avatar
  • 1,700
3 votes
5 answers
165 views

How to remove double quotes within the double-quoted field values in .dat file

I have a text file that has around 15 columns.  The fields are separated by comma.  One column that is description is double-quoted and also has some words which are double-quoted.  I need to retain ...
Mythri's user avatar
  • 43
1 vote
3 answers
297 views

I know I can create folders from a list.txt, but can I move files to those folders based on a mapping?

If right now, I only had a bunch of files dumped into a single directory, is there a way that I could move the files into a folder based on a mapping file. For example: A B 1 File Name Folder Name ...
Nicole's user avatar
  • 19
4 votes
2 answers
396 views

jq: Printing multiple values from multiple arrays at once

The default functionality of jq is to send each object from an array one at a time, though the join operator can merge those values. My problem is in trying to print all the values from multiple ...
T145's user avatar
  • 213
0 votes
1 answer
45 views

Replacing variable correctly in alias function

I'm trying to create an alias function to remove inputted line from file: function remove_line(){ line_to_remove="'s/^"$1"$//g'" sed -i $(line_to_remove) my_file } ...
ellat's user avatar
  • 37
5 votes
5 answers
330 views

Is it possible to sort the letters in a word in a line of text?

So I have a file full of test commands that I like to run against some of my functions to make sure they are handling all possible situations correctly. No point in having duplicate commands tho. ...
Ray Andrews's user avatar
  • 2,527
-2 votes
1 answer
174 views

Error in command su

I am creating a command that will change a certain line in the /etc/profile file from a script, however in certain versions of Linux the sudo command may not be activated so it would be necessary to ...
user avatar
3 votes
1 answer
377 views

How can I use $0 in Makefile?

I would like to include the following command (taken from here) in a Makefile. A simple version of my Makefile script containing the command is: copy_files: sed 's/^/\./' ./input_file | \ ...
ellat's user avatar
  • 37
7 votes
5 answers
1k views

How to create grep output in the order of the input file?

I have a scrambledA.txt with data in a list like so: efrrsu aehmmr aeeglnr alnors acflno aaahmy aceimru 1469en aelprsy cehrry and a corresponding wordlistC.txt that contains many many key-value-pairs:...
bigstone's user avatar
7 votes
4 answers
247 views

Display occurrences and count lines between them and after the last one

I have this file: table_01 (id, field01, field02, field03, field04) record_01 record_02 record_03 table_02 (id, field01, field02, field03) record_01 table_03 (id, field01, field02, field03, field04) ...
Sistemas Libsur's user avatar
0 votes
4 answers
110 views

Remove subdomains or existing domains

I have a list of domains, a sample is: account.google.com drive.google.com google.com bgoogle.com yahoo.co.uk stats.wikipedia.org media.wikipedia.org files.media.wikipedia.org bible.com I would like ...
ellat's user avatar
  • 37
1 vote
5 answers
101 views

Remove lines from csv file

I have a csv file that contains, file share path, username and permissions, like this: "FolderName";"ADGroupOrUser";"Permissions" \\path\1 - folder 01\1 - folder2; user....
Hugo's user avatar
  • 11
2 votes
7 answers
750 views

How to remove embedded newlines from CSV fields

I am struggling with processing a file. Normally I should have a line for each host definition. But sometimes, someone split some fields on different lines. Here is an example: "host1","...
Nicolas Greder's user avatar
2 votes
1 answer
61 views

Playing random videos on one's system; minor improvement to include DVDs

In my .bashrc I have a function which I use to play random videos (not written below, just fyi), and another one for all media files, e.g.: createmediafiles() { find ~+ -type f -iregex '.*\.\(mp3\|wav\...
1toneboy's user avatar
  • 441
2 votes
3 answers
264 views

List files only in rpm package

Launching rpm -ql <somerpm> lists files and directories contained in an rpm package. I am interested in only listing the files strictly speaking, not the directories. Is there any flag or ...
carce-bo's user avatar
  • 145
5 votes
6 answers
249 views

subset columns from the 1st file using column names in 2nd file

I have two text files: 1st file is a Tab delimited file which looks like this: chrom pos ref alt a1 a2 a3 a4 10 12345 C T aa bb cc dd 10 12345 C T aa bb cc dd 10 12345 C ...
user3138373's user avatar
  • 2,559
3 votes
5 answers
238 views

How to let sed un-comment one of multiple paragraphs separated by newlines?

Clarification: Using GNU sed. Status quo - ~/.screenrc, last lines: # name synthpop # screen -t script emacs -nw /home/$USER/bin/synthpop # name thrashmetal # screen -t script emacs -nw /home/$USER/...
futurewave's user avatar
0 votes
3 answers
121 views

Remove all lines between two similar patterns if a specific pattern in between is not matched

The title should be: Remove all lines between two similar patterns, including the line of the starting pattern. if another specific pattern in between is not matched. I made it shorter because its ...
pwrsheller's user avatar
-1 votes
2 answers
113 views

Remove all words that appear n+ times without deleting the lines or change the file order

I have a text file that consists of sections, each with two header lines and a body line consisting of space-separated words. An example looks as follows: Shares for DED-SHD-ED-1: [--- Listable ...
pwrsheller's user avatar
2 votes
2 answers
63 views

Listing a certain part of filenames and their contents

I have a Linux machine with BusyBox, which has a directory called /data/var/lib/connman.  That directory holds some directories that I'm not interested in.  But it contains many .config files that ...
Norbert's user avatar
  • 121
-1 votes
5 answers
96 views

how to print rows based on latest date in each month

I have a file in Linux with below content. I want to print the rows with maximum date value in each month based on date value in column col3.The data is already sorted in descending order by col3. ...
Josin Abraham's user avatar
1 vote
2 answers
81 views

Reformat text to exact width [duplicate]

On Ubuntu, how to reformat text to fit width (except final line), adding spaces where necessary? The closest I can get is with fmt --width=64, but this does not add the spaces between words. Input ...
FromTheStackAndBack's user avatar
3 votes
3 answers
255 views

Strip filesize from rclone file list

I use rclone and I want to have the file list without the size of the files mention. I can't find how to do this in rclone so I thought of stripping it with awk or something like that. My output looks ...
unixcandles's user avatar
0 votes
6 answers
79 views

Grep variable number before / after string from textline

I want to grep a number from a Textline. cat log.txt | grep "License term" 01/01/2024:00:30 License term is 123 days. I only want to seperate the "downcounting number" from that. ...
mkcdu's user avatar
  • 13
-1 votes
5 answers
123 views

How to check the contents of an entire file against the pattern \s*{\s*}\s*?

How to check the contents of an entire file for compliance with the pattern \s*{\s*}\s* using busybox? You need to check the entire file from beginning to end, and not individual lines, as grep does. ...
Maxim's user avatar
  • 119
1 vote
4 answers
177 views

Use sed or grep in bash script to check if file ends with specific string

I have a file called output.txt on my ubuntu server, how do I use sed or grep to check if the last two characters in the file are "nt" The text in the file is below, I expect my if statement ...
user23376904's user avatar
1 vote
2 answers
112 views

Use shell script to select a specific pdf with unique filename format and recent creation-date?

I am needing some help with a script. I'm currently trying to select PDF file that meets a specific requirement in the file name in order to move it to a different location. The filename format I'm ...
Brad Barnes's user avatar
5 votes
5 answers
686 views

Remove lines from a csv file based on column value

I have a csv file with 12 million lines in the following format: mcu_i,INIT,200,iFlash, 11593925, 88347,,0x00092684,r,0x4606b570, ok,, 32,single,op-c,0,, 0, 0, 0,...
raphui's user avatar
  • 84
0 votes
0 answers
54 views

Geany: Possible to automatically enable/disable line wrap dependent on file type?

In Geany text editor / IDE, is there a way to tell it to automatically enable line wrapping for certain file types, but not for others? For example, for .txt files, I would like it to always enable ...
End Antisemitic Hate's user avatar
2 votes
4 answers
151 views

SED command to delete a forward slash "/" between 2 tags

I am using Linux, and I want to use sed to delete a forward slash (/) between 2 tags. From this: <file>/text</file> <file>/text2</file <file>/text</file>.. to this &...
Allan Tori's user avatar

1
2 3 4 5
169