Questions tagged [numeric-data]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
-1 votes
2 answers
50 views

Test for a number in bash [duplicate]

Is this the correct way to test for a number, with double [[]] enclosing :digit: and single quotes surrounding the regex ? if [[ "$var" =~ '^[[:digit:]]+$' ]]; then
  • 141
0 votes
0 answers
26 views

numeric sort with unique option does not show 0! [duplicate]

I have a file with many redundant numbers in each row. Imagine something like the below: echo "10 9 5 6 4 cell 3 2 0 7 0 1" > test When I use sort -un test I get the following output: ...
  • 131
0 votes
2 answers
222 views

Add commas in place of number with sed?

Trying to write a simple script to fetch ethereum price and format the response. Here is what I've tried: #!/bin/bash response=$(curl 'https://min-api.cryptocompare.com/data/price?fsym=ETH&tsyms=...
  • 1
1 vote
1 answer
103 views

How do I use sort on multiple columns with different data types

Here's my tab-delimited file t.tsv: $ cat t.tsv 2022/05/05 -258.03 2022/05/07 -18.10 2022/05/09 -10.74 2022/05/09 -132.60 2022/05/12 -18.56 2022/05/12 -20.20 2022/05/17 -11.00 2022/05/17 -112....
  • 359
2 votes
0 answers
67 views

Scientific notations to specific scientific notation [duplicate]

I want to convert the value from 1.759114e-07 to 175.9114e-09. How can I script the conversion? I tried this but I'm not getting what I want (which is 175.9114e-09): $ echo 1.759114e-07 | awk '{printf(...
0 votes
1 answer
25 views

How to display only the content between opening and closing tags in XML file?

How to search the XML file using grep or similar for a particular tag but show only the content between opening and closing tags? Here is the exact tag I'd like to locate: <max-diskusage>...
  • 189
3 votes
3 answers
986 views

How to sort each 20 lines in a 1000 line file and save only the sorted line with highest value in each interval to another file?

I have a file that has 1000 text lines. I want to sort the 4th column at each 20 lines interval and print the output to another file. Can anybody help me with sorting them with awk or sed? Here is an ...
0 votes
2 answers
299 views

Finding values of one file within range of another file and selecting the top value

I have two files A and B. File-A has 4 columns and 600,000 rows. File-B has 4 columns and 5000 rows. Example: File-A: ENSB1 1 12245 0.53 0.002 ENSB2 1 13400 0.27 0.0003 ENSB3 1 14780 0.13 0.00001 ...
4 votes
5 answers
682 views

Converting HEX to DEC is out of range by using `mawk`

When the hex number is relative small, I can use echo 0xFF| mawk '{ printf "%d\n", $1}' to convert hex to dec. When then hex number is huge, mawk does not work any more, e.g. echo ...
  • 258
0 votes
2 answers
2k views

Matching numbers with glob pattern

I am trying to match using a glob pattern. But it is failing when using myfun 12. dgt='^+([0123456789])$' [[ "$1" == $dgt ]] && echo "SUCCESS" || echo "FAILURE"
  • 141
2 votes
1 answer
637 views

What's the difference between "sort -n" and "sort -g"?

What is the difference between the two sort options -n and -g? It's a bit confusing to have too much detail but not enough adequate documentation.
user avatar
4 votes
3 answers
787 views

Random number output between two ranges linked together

This question is about generating random numbers between a range, which is fine, but it doesn't fit my case. I'll explain in SQL terms because it seems to me that's easier to understand, though the ...
0 votes
0 answers
26 views

Check if variable in numeric in a bash script [duplicate]

Is this the way to check a variable in numeric? case "$var" in ( +([[:digit:]]) ) cn="$var" ;; (*) cn=208 ;; esac
  • 189
0 votes
1 answer
68 views

Highlight every number that is smaller than all previous numbers in the last column

Input: network-snapshot-000000 time 6m 40s fid50k_full 34.9546 network-snapshot-000201 time 6m 52s fid50k_full 30.8073 network-snapshot-000403 time 6m 51s ...
  • 314
2 votes
1 answer
200 views

Reformat number in specific column

I have semicolon delimited data, where third column is decimal number: A;B;1234.56 C;D;789.23 I need to make 2 changes to the format of the number: remove numbers after the decimal point add "...
1 vote
1 answer
197 views

Sort the data to plot a circle

Here I have a data which I would to plot with line using Gnuplot. Using the code pl 'Sphere_ISOTEST_data.txt' w p I get below figure but, using pl 'Sphere_ISOTEST_data.txt' w l I get the following: ...
2 votes
2 answers
2k views

How assign numbers range into a simple bash variable

How assign numbers sequence into bash variable as this fail: $ n={0..9}; echo $n {0..9} $ n=\"{0..9}\"; echo $n "{0..9}" $ eval n={0..9}; echo $n $ eval n=\"{0..9}\"; ...
0 votes
3 answers
149 views

Extract maximum values for each objects from a file [duplicate]

I need to extract some values from a file in bash, on my CentOS system. In myfile.txt I have a list of objects called Info_region in which each object is identified with a code (eg. BARD1_region_005 ...
2 votes
3 answers
2k views

How to generate a comma separated list of random ints

I am trying to generate a comma separated unordered list of ints between 1 and 10, I have tried the following but it results in an ordered list: seq -s "," 10 | shuf
  • 23
7 votes
2 answers
625 views

Can `awk` sum a column over a specified number of lines

I've reviewed the "Similar questions", and none seem to solve my problem: I have a large CSV input file; each line in the file is an x,y data point. Here are a few lines for illustration, ...
  • 2,404
0 votes
1 answer
65 views

shell script set the default numbers as double digit zero as prefix [duplicate]

In a shell script, I'm processing, some addition process will print an output. If it is a single-digit one, then it has to add zero as a prefix. Here is my current script: c_year=2020 for i in {01..11}...
0 votes
2 answers
852 views

shell script set the default numbers as double digit zero as prefix

In a shell script, I'm processing, some addition process will print an output. If it is a single-digit one, then it has to add zero as a prefix. Here is my current script: c_year=2020 for i in {01..11}...
0 votes
7 answers
151 views

Insert Zero Before and After a data point

I need a script that inserts a zero before and after each data point. I am trying to do this in Ubuntu. Example: 1 2 3 4 5 What I need: 0 1 0 0 2 0 0 3 0 0 4 0 0 5 0
0 votes
2 answers
415 views

Rounding numbers of a CSV file, skipping the header

This question is a derivative of the https://askubuntu.com/questions/601149/is-there-a-command-to-round-decimal-numbers-in-txt-files, which was successfully solved by using: perl -i -pe 's/(\d*\.\d*)/...
  • 93
1 vote
3 answers
984 views

Extracting numbers between two string patterns

I have a file with following type of expression in every line "Age=22 years and Height=6 feet", I want to extract Age and height numbers only. I have tried grep -oP '(?<=Age=)[^years]+' $...
3 votes
3 answers
608 views

Calculating rates / 'derivative' with awk

awk newbie here. Suppose I have two columns of data, and I want to calculate the rate of increase, given by delta(y)/delta(x). How would I do this in an awk script? What I've learnt so far only deals ...
2 votes
1 answer
441 views

Testing if a string is a number

That should be easy, just use [[ "$var" =~ '^[1-9][0-9]*$' ]]. But I don't get the behavior I expect excepted with zsh. I don't control the machine where the script will be run, so portability along ...
  • 2,238
4 votes
2 answers
387 views

Numerical sorting of every column in a file individually using awk

I'm trying to numerically sort every column individually in a very large file. I need the command to be fast, so I'm trying to do it in an awk command. Example Input: 1,4,2,7,4 9,2,1,1,1 3,9,9,2,2 5,...
  • 877
4 votes
2 answers
2k views

Number formatting and rounding issue with awk

I want to get the exact number when I try to find the average of a column of values. For example, this is the column of input values: 1426044 1425486 1439480 1423677 1383676 1360088 1390745 1435123 ...
  • 369
1 vote
2 answers
979 views

Sum of large numbers and print the result with all decimal points for the stated question when using awk arrays

I have a below input file which I need to split into multiple files based on the date in 3rd column. Basically all the same dated transactions should be splitted into particular dated file. Post ...
  • 73
0 votes
4 answers
217 views

Filling empty cells and making data type the same in a tab separated file

I am dealing with a tab separated file with nearly 200MM rows on Linux. In one the columns which contains binary values, I noticed the data type is not consistent and there is a large number of ...
  • 109
3 votes
3 answers
950 views

Delete duplicate lines in file without creating new file in ubuntu

I can't seem to find a command that lets me delete duplicates in my file without creating a new file and also preserving the order of the contents in my file. Would there be another command besides ...
  • 39
0 votes
2 answers
65 views

Find max values within Column 1 range and print column2

I have a representative dataset 35.5259 327 35.526 326 35.526 325 35.5261 324 35.5262 323 35.5263 322 35.5264 321 35.5265 320 35.5266 319 35.5268 318 # ...
0 votes
2 answers
87 views

How to replace a fix location in each line in a file

I have a file as below, how do I replace the $number (location 19-30) and (location 59-70) with its number in cents. update: All fields are required to stay at the same fixed location. input: ...
2 votes
3 answers
704 views

How can I convert a float to ratio?

I would like to convert a float to a ratio or fraction. Do we have the option to convert a float 1.778 to ratio as 16:9 or 16/9 in bash, in a fashion similar to Python's fractions module (Fraction(1....
  • 8,847
-1 votes
1 answer
512 views

Hexadecimal color codes grep regex in makefile

So I have a text file containing valid and invalid hexadecimal color codes. I want to be able to filter out the invalid codes and print just the valid ones. For a code to be valid it must have a hash ...
  • 1
2 votes
2 answers
80 views

Comparing two columns(numeric) in two different text files

I have two text files with space-delimited fields (03 fields) in Linux environment: File1 : ram-service1.txt RAM used Program 23500 kb firewalld 14780 kb tuned 10140 kb polkitd 5370 ...
1 vote
6 answers
7k views

how to extract maximum and minimum value from column 1 and column 2

Dear all I have a big data file lets say file.dat, it contains two columns e.g file.dat (showing few rows) 0.0000 -23.4334 0.0289 -23.4760 0.0578 -23.5187 0.0867 -23.5616 0....
  • 77
1 vote
3 answers
801 views

Converting base-6 number to base-2 (binary) using commands or a program

I have a base-6 number generated by rolling a physical six-sided dice and want to convert it into base-2 / binary. Are there built-in commands or programs that can do that for me? If not, what is ...
-1 votes
1 answer
1k views

regex (or other way) to find numbers followed by a space

I'm running the following awk '$2 ~ /^[0-9]\s+/ {print($2)}' file >> file1 Not sure how to correct this to only select lines where digits are followed by a space?
  • 275
-1 votes
2 answers
55 views

How I can change 01 and 25 in 001 and 025 without change 125 in 0125?

I have a file like this Thomas_4567_01 Luc_2344_125 Ann_5780_25 I would like an outpu like this Thomas_4567_001 Luc_2344_125 Ann_5780_025 How I can change 01 and 25 in 001 and 025 without change ...
1 vote
5 answers
246 views

sed/awk: replace numbers in a line after last occurance of '.'

I have the following tcpdump stream: Current: 07:36:03.848461 IP 172.17.3.41.33101 > 172.17.3.43.17408: UDP, length 44 07:36:03.848463 IP 172.17.3.42.33101 > 172.17.3.43.17409: UDP, length 44 ...
  • 23
3 votes
2 answers
93 views

Why does gawk (sometimes?) think 2.0e-318 > 2.0?

I am trying to find the maximum value of a column of data using gawk: gawk 'BEGIN{max=0} {if($1>0+max) max=$1} END {print max}' dataset.dat where dataset.dat looks like this: 2.0 2.0e-318 ...
4 votes
2 answers
955 views

Sort numerical column

I'm trying to sort a file based on a particular position but that does not work, here is the data and output. ~/scratch$ cat id_researchers_2018_sample id - 884209 , researchers - 1 id - 896781 , ...
0 votes
5 answers
819 views

Max, Mean, Min for range within column of data

I have data structured like this :- X 43808873 A 14 X 43808874 G 7 X 43808875 T 15 X 43808876 A 13 X 43808877 T 4 X 43808878 C 13 X 43808879 T 11 X ...
-1 votes
1 answer
79 views

sum the values of all occurrences parameters

we have the following file , in file we can see the same parameters names with their values more file.txt gg_ff_rr =56 gg_ff_rr =56 gg_ff_rr =52 jk_ll_tt=24 jk_ll_tt=24 jk_ll_tt=24 pl_pl_uu=10 ...
  • 12.2k
1 vote
1 answer
411 views

how to change color of text based on value?

I want to use cat to display a file content and changes the text color to red if the number is larger than 50. Example: cat file1 would give 1 33 56 the number 56 will be displayed in red ...
2 votes
2 answers
595 views

How to get max, min, and mean from values in column 4

I have data structured like this X 43808504 G 1 ^]. < X 43808505 C 3 . 4 X 43808506 T 8 . ? X 43808507 G 5 . C I want to get the max (8), min (1), and ...
0 votes
1 answer
494 views

Calculating average in awk based on several column conditions in csv

I currently have the following csv format, it's much larger but I've taken a chunk of it for now. EV,"Houston","-7.0" AB,"Boston","19.0" OO,"Mystic","13.0" AB,"Boston","-12.0" EN,"New York City","9....
4 votes
1 answer
563 views

Awk inconsistent behavior filtering scientific notation numbers

I recently found that for some very small values, filtering using AWK doesn't seem to behave correctly. As in the following file test_loc.txt: 10:10000018 10 0.4505 X:99997421 X 0.95508 X:99997626 ...
  • 41

1
2 3 4 5