Questions tagged [numeric-data]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
8 votes
5 answers
1k views

How to round to 2 decimals in bash like MS Excel does?

I spent hours in searching how to round "floating numbers" in BASH but couldn't find any correct! solution :( If I put these numbers to Excel, then I will receive the correct results after ...
user3719454's user avatar
-1 votes
2 answers
355 views

Test for a number in bash [duplicate]

Is this the correct way to test for a number, with double [[]] enclosing :digit: and single quotes surrounding the regex ? if [[ "$var" =~ '^[[:digit:]]+$' ]]; then
Vera's user avatar
  • 1,223
0 votes
0 answers
26 views

numeric sort with unique option does not show 0! [duplicate]

I have a file with many redundant numbers in each row. Imagine something like the below: echo "10 9 5 6 4 cell 3 2 0 7 0 1" > test When I use sort -un test I get the following output: ...
javadr's user avatar
  • 131
0 votes
2 answers
409 views

Add commas in place of number with sed?

Trying to write a simple script to fetch ethereum price and format the response. Here is what I've tried: #!/bin/bash response=$(curl 'https://min-api.cryptocompare.com/data/price?fsym=ETH&tsyms=...
ash's user avatar
  • 1
2 votes
1 answer
326 views

How do I use sort on multiple columns with different data types

Here's my tab-delimited file t.tsv: $ cat t.tsv 2022/05/05 -258.03 2022/05/07 -18.10 2022/05/09 -10.74 2022/05/09 -132.60 2022/05/12 -18.56 2022/05/12 -20.20 2022/05/17 -11.00 2022/05/17 -112....
Chap's user avatar
  • 369
2 votes
0 answers
68 views

Scientific notations to specific scientific notation [duplicate]

I want to convert the value from 1.759114e-07 to 175.9114e-09. How can I script the conversion? I tried this but I'm not getting what I want (which is 175.9114e-09): $ echo 1.759114e-07 | awk '{printf(...
Ahamad Ahamad's user avatar
0 votes
1 answer
79 views

How to display only the content between opening and closing tags in XML file?

How to search the XML file using grep or similar for a particular tag but show only the content between opening and closing tags? Here is the exact tag I'd like to locate: <max-diskusage>...
Joey's user avatar
  • 199
3 votes
3 answers
1k views

How to sort each 20 lines in a 1000 line file and save only the sorted line with highest value in each interval to another file?

I have a file that has 1000 text lines. I want to sort the 4th column at each 20 lines interval and print the output to another file. Can anybody help me with sorting them with awk or sed? Here is an ...
Sanjukta's user avatar
0 votes
2 answers
506 views

Finding values of one file within range of another file and selecting the top value

I have two files A and B. File-A has 4 columns and 600,000 rows. File-B has 4 columns and 5000 rows. Example: File-A: ENSB1 1 12245 0.53 0.002 ENSB2 1 13400 0.27 0.0003 ENSB3 1 14780 0.13 0.00001 ...
Muhammad Sajid Tahir's user avatar
4 votes
5 answers
984 views

Converting HEX to DEC is out of range by using `mawk`

When the hex number is relative small, I can use echo 0xFF| mawk '{ printf "%d\n", $1}' to convert hex to dec. When then hex number is huge, mawk does not work any more, e.g. echo ...
John's user avatar
  • 298
2 votes
1 answer
2k views

What's the difference between "sort -n" and "sort -g"?

What is the difference between the two sort options -n and -g? It's a bit confusing to have too much detail but not enough adequate documentation.
user avatar
4 votes
3 answers
955 views

Random number output between two ranges linked together

This question is about generating random numbers between a range, which is fine, but it doesn't fit my case. I'll explain in SQL terms because it seems to me that's easier to understand, though the ...
schrodingerscatcuriosity's user avatar
0 votes
1 answer
77 views

Highlight every number that is smaller than all previous numbers in the last column

Input: network-snapshot-000000 time 6m 40s fid50k_full 34.9546 network-snapshot-000201 time 6m 52s fid50k_full 30.8073 network-snapshot-000403 time 6m 51s ...
Navin's user avatar
  • 333
1 vote
1 answer
275 views

Reformat number in specific column

I have semicolon delimited data, where third column is decimal number: A;B;1234.56 C;D;789.23 I need to make 2 changes to the format of the number: remove numbers after the decimal point add "...
Martin Vegter's user avatar
1 vote
1 answer
231 views

Sort the data to plot a circle

Here I have a data which I would to plot with line using Gnuplot. Using the code pl 'Sphere_ISOTEST_data.txt' w p I get below figure but, using pl 'Sphere_ISOTEST_data.txt' w l I get the following: ...
newstudent's user avatar
2 votes
2 answers
3k views

How assign numbers range into a simple bash variable

How assign numbers sequence into bash variable as this fail: $ n={0..9}; echo $n {0..9} $ n=\"{0..9}\"; echo $n "{0..9}" $ eval n={0..9}; echo $n $ eval n=\"{0..9}\"; ...
user avatar
0 votes
3 answers
205 views

Extract maximum values for each objects from a file [duplicate]

I need to extract some values from a file in bash, on my CentOS system. In myfile.txt I have a list of objects called Info_region in which each object is identified with a code (eg. BARD1_region_005 ...
Tommaso Palomba's user avatar
2 votes
3 answers
2k views

How to generate a comma separated list of random ints

I am trying to generate a comma separated unordered list of ints between 1 and 10, I have tried the following but it results in an ordered list: seq -s "," 10 | shuf
spudger's user avatar
  • 23
7 votes
2 answers
1k views

Can `awk` sum a column over a specified number of lines

I've reviewed the "Similar questions", and none seem to solve my problem: I have a large CSV input file; each line in the file is an x,y data point. Here are a few lines for illustration, ...
Seamus's user avatar
  • 2,949
0 votes
1 answer
132 views

shell script set the default numbers as double digit zero as prefix [duplicate]

In a shell script, I'm processing, some addition process will print an output. If it is a single-digit one, then it has to add zero as a prefix. Here is my current script: c_year=2020 for i in {01..11}...
TheDataGuy's user avatar
0 votes
2 answers
1k views

shell script set the default numbers as double digit zero as prefix

In a shell script, I'm processing, some addition process will print an output. If it is a single-digit one, then it has to add zero as a prefix. Here is my current script: c_year=2020 for i in {01..11}...
TheDataGuy's user avatar
0 votes
7 answers
155 views

Insert Zero Before and After a data point

I need a script that inserts a zero before and after each data point. I am trying to do this in Ubuntu. Example: 1 2 3 4 5 What I need: 0 1 0 0 2 0 0 3 0 0 4 0 0 5 0
BirkbyBoyz's user avatar
0 votes
2 answers
538 views

Rounding numbers of a CSV file, skipping the header

This question is a derivative of the https://askubuntu.com/questions/601149/is-there-a-command-to-round-decimal-numbers-in-txt-files, which was successfully solved by using: perl -i -pe 's/(\d*\.\d*)/...
Charlie's user avatar
  • 93
1 vote
3 answers
1k views

Extracting numbers between two string patterns

I have a file with following type of expression in every line "Age=22 years and Height=6 feet", I want to extract Age and height numbers only. I have tried grep -oP '(?<=Age=)[^years]+' $...
user3696623's user avatar
3 votes
3 answers
776 views

Calculating rates / 'derivative' with awk

awk newbie here. Suppose I have two columns of data, and I want to calculate the rate of increase, given by delta(y)/delta(x). How would I do this in an awk script? What I've learnt so far only deals ...
Suraaj K S's user avatar
2 votes
1 answer
939 views

Testing if a string is a number

That should be easy, just use [[ "$var" =~ '^[1-9][0-9]*$' ]]. But I don't get the behavior I expect excepted with zsh. I don't control the machine where the script will be run, so portability along ...
AProgrammer's user avatar
  • 2,328
4 votes
2 answers
538 views

Numerical sorting of every column in a file individually using awk

I'm trying to numerically sort every column individually in a very large file. I need the command to be fast, so I'm trying to do it in an awk command. Example Input: 1,4,2,7,4 9,2,1,1,1 3,9,9,2,2 5,...
Giles's user avatar
  • 897
4 votes
2 answers
2k views

Number formatting and rounding issue with awk

I want to get the exact number when I try to find the average of a column of values. For example, this is the column of input values: 1426044 1425486 1439480 1423677 1383676 1360088 1390745 1435123 ...
rplee's user avatar
  • 377
1 vote
2 answers
1k views

Sum of large numbers and print the result with all decimal points for the stated question when using awk arrays

I have a below input file which I need to split into multiple files based on the date in 3rd column. Basically all the same dated transactions should be splitted into particular dated file. Post ...
hunter's user avatar
  • 83
0 votes
4 answers
263 views

Filling empty cells and making data type the same in a tab separated file

I am dealing with a tab separated file with nearly 200MM rows on Linux. In one the columns which contains binary values, I noticed the data type is not consistent and there is a large number of ...
Alex Man's user avatar
  • 109
3 votes
3 answers
2k views

Delete duplicate lines in file without creating new file in ubuntu

I can't seem to find a command that lets me delete duplicates in my file without creating a new file and also preserving the order of the contents in my file. Would there be another command besides ...
Nathan's user avatar
  • 39
0 votes
2 answers
68 views

Find max values within Column 1 range and print column2

I have a representative dataset 35.5259 327 35.526 326 35.526 325 35.5261 324 35.5262 323 35.5263 322 35.5264 321 35.5265 320 35.5266 319 35.5268 318 # ...
newstudent's user avatar
0 votes
2 answers
98 views

How to replace a fix location in each line in a file

I have a file as below, how do I replace the $number (location 19-30) and (location 59-70) with its number in cents. update: All fields are required to stay at the same fixed location. input: ...
user408286's user avatar
2 votes
3 answers
863 views

How can I convert a float to ratio?

I would like to convert a float to a ratio or fraction. Do we have the option to convert a float 1.778 to ratio as 16:9 or 16/9 in bash, in a fashion similar to Python's fractions module (Fraction(1....
Siva's user avatar
  • 9,077
-1 votes
1 answer
573 views

Hexadecimal color codes grep regex in makefile

So I have a text file containing valid and invalid hexadecimal color codes. I want to be able to filter out the invalid codes and print just the valid ones. For a code to be valid it must have a hash ...
Casper's user avatar
  • 1
2 votes
2 answers
152 views

Comparing two columns(numeric) in two different text files

I have two text files with space-delimited fields (03 fields) in Linux environment: File1 : ram-service1.txt RAM used Program 23500 kb firewalld 14780 kb tuned 10140 kb polkitd 5370 ...
Lasith Anjula Galanga's user avatar
1 vote
6 answers
10k views

how to extract maximum and minimum value from column 1 and column 2

Dear all I have a big data file lets say file.dat, it contains two columns e.g file.dat (showing few rows) 0.0000 -23.4334 0.0289 -23.4760 0.0578 -23.5187 0.0867 -23.5616 0....
sai's user avatar
  • 77
1 vote
3 answers
883 views

Converting base-6 number to base-2 (binary) using commands or a program

I have a base-6 number generated by rolling a physical six-sided dice and want to convert it into base-2 / binary. Are there built-in commands or programs that can do that for me? If not, what is ...
bmvanity8723's user avatar
-1 votes
1 answer
2k views

regex (or other way) to find numbers followed by a space

I'm running the following awk '$2 ~ /^[0-9]\s+/ {print($2)}' file >> file1 Not sure how to correct this to only select lines where digits are followed by a space?
ZakS's user avatar
  • 305
-1 votes
2 answers
55 views

How I can change 01 and 25 in 001 and 025 without change 125 in 0125?

I have a file like this Thomas_4567_01 Luc_2344_125 Ann_5780_25 I would like an outpu like this Thomas_4567_001 Luc_2344_125 Ann_5780_025 How I can change 01 and 25 in 001 and 025 without change ...
Tommaso Palomba's user avatar
1 vote
5 answers
265 views

sed/awk: replace numbers in a line after last occurance of '.'

I have the following tcpdump stream: Current: 07:36:03.848461 IP 172.17.3.41.33101 > 172.17.3.43.17408: UDP, length 44 07:36:03.848463 IP 172.17.3.42.33101 > 172.17.3.43.17409: UDP, length 44 ...
umar14's user avatar
  • 23
3 votes
2 answers
103 views

Why does gawk (sometimes?) think 2.0e-318 > 2.0?

I am trying to find the maximum value of a column of data using gawk: gawk 'BEGIN{max=0} {if($1>0+max) max=$1} END {print max}' dataset.dat where dataset.dat looks like this: 2.0 2.0e-318 ...
HerpDerpington's user avatar
4 votes
2 answers
2k views

Sort numerical column

I'm trying to sort a file based on a particular position but that does not work, here is the data and output. ~/scratch$ cat id_researchers_2018_sample id - 884209 , researchers - 1 id - 896781 , ...
user1050619's user avatar
0 votes
5 answers
976 views

Max, Mean, Min for range within column of data

I have data structured like this :- X 43808873 A 14 X 43808874 G 7 X 43808875 T 15 X 43808876 A 13 X 43808877 T 4 X 43808878 C 13 X 43808879 T 11 X ...
kellogg76's user avatar
-1 votes
1 answer
117 views

sum the values of all occurrences parameters

we have the following file , in file we can see the same parameters names with their values more file.txt gg_ff_rr =56 gg_ff_rr =56 gg_ff_rr =52 jk_ll_tt=24 jk_ll_tt=24 jk_ll_tt=24 pl_pl_uu=10 ...
yael's user avatar
  • 13.1k
1 vote
1 answer
528 views

how to change color of text based on value?

I want to use cat to display a file content and changes the text color to red if the number is larger than 50. Example: cat file1 would give 1 33 56 the number 56 will be displayed in red ...
machinecat's user avatar
2 votes
2 answers
1k views

How to get max, min, and mean from values in column 4

I have data structured like this X 43808504 G 1 ^]. < X 43808505 C 3 . 4 X 43808506 T 8 . ? X 43808507 G 5 . C I want to get the max (8), min (1), and ...
kellogg76's user avatar
0 votes
1 answer
575 views

Calculating average in awk based on several column conditions in csv

I currently have the following csv format, it's much larger but I've taken a chunk of it for now. EV,"Houston","-7.0" AB,"Boston","19.0" OO,"Mystic","13.0" AB,"Boston","-12.0" EN,"New York City","9....
kayatoast's user avatar
4 votes
1 answer
713 views

Awk inconsistent behavior filtering scientific notation numbers

I recently found that for some very small values, filtering using AWK doesn't seem to behave correctly. As in the following file test_loc.txt: 10:10000018 10 0.4505 X:99997421 X 0.95508 X:99997626 ...
Aomdahl's user avatar
  • 41
1 vote
3 answers
327 views

Find the maximum value of Column3 and print the values of Column1 and 2 only

In Unix, I am trying to find a command that would find the maximum value in Column3 and print the corresponding values from Column2 and Column1 (but not from Column3) in a new file. Column1 ...
Michaela14's user avatar

1
2 3 4 5