Questions tagged [numeric-data]
The numeric-data tag has no usage guidance.
222
questions
6
votes
2answers
201 views
Can `awk` sum a column over a specified number of lines
I've reviewed the "Similar questions", and none seem to solve my problem:
I have a large CSV input file; each line in the file is an x,y data point. Here are a few lines for illustration, ...
0
votes
1answer
19 views
shell script set the default numbers as double digit zero as prefix [duplicate]
In a shell script, I'm processing, some addition process will print an output. If it is a single-digit one, then it has to add zero as a prefix.
Here is my current script:
c_year=2020
for i in {01..11}...
0
votes
2answers
31 views
shell script set the default numbers as double digit zero as prefix
In a shell script, I'm processing, some addition process will print an output. If it is a single-digit one, then it has to add zero as a prefix.
Here is my current script:
c_year=2020
for i in {01..11}...
0
votes
4answers
80 views
Insert Zero Before and After a data point
I need a script that inserts a zero before and after each data point. I am trying to do this in Ubuntu.
Example:
1
2
3
4
5
What I need:
0
1
0
0
2
0
0
3
0
0
4
0
0
5
0
0
votes
1answer
20 views
Rounding numbers of a CSV file, skipping the header
This question is a derivative of the https://askubuntu.com/questions/601149/is-there-a-command-to-round-decimal-numbers-in-txt-files, which was successfully solved by using:
perl -i -pe 's/(\d*\.\d*)/...
1
vote
4answers
251 views
Extracting numbers between two string patterns
I have a file with following type of expression in every line "Age=22 years and Height=6 feet", I want to extract Age and height numbers only.
I have tried
grep -oP '(?<=Age=)[^years]+' $...
3
votes
3answers
211 views
Calculating rates / 'derivative' with awk
awk newbie here.
Suppose I have two columns of data, and I want to calculate the rate of increase, given by delta(y)/delta(x). How would I do this in an awk script? What I've learnt so far only deals ...
2
votes
1answer
81 views
Testing if a string is a number
That should be easy, just use [[ "$var" =~ '^[1-9][0-9]*$' ]]. But I don't get the behavior I expect excepted with zsh. I don't control the machine where the script will be run, so portability along ...
4
votes
2answers
133 views
Numerical sorting of every column in a file individually using awk
I'm trying to numerically sort every column individually in a very large file. I need the command to be fast, so I'm trying to do it in an awk command.
Example Input:
1,4,2,7,4
9,2,1,1,1
3,9,9,2,2
5,...
4
votes
3answers
479 views
Number formatting and rounding issue with awk
I want to get the exact number when I try to find the average of a column of values.
For example, this is the column of input values:
1426044
1425486
1439480
1423677
1383676
1360088
1390745
1435123
...
1
vote
2answers
300 views
Sum of large numbers and print the result with all decimal points for the stated question when using awk arrays
I have a below input file which I need to split into multiple files based on the date in 3rd column. Basically all the same dated transactions should be splitted into particular dated file. Post ...
0
votes
4answers
60 views
Filling empty cells and making data type the same in a tab separated file
I am dealing with a tab separated file with nearly 200MM rows on Linux. In one the columns which contains binary values, I noticed the data type is not consistent and there is a large number of ...
3
votes
3answers
77 views
Delete duplicate lines in file without creating new file in ubuntu
I can't seem to find a command that lets me delete duplicates in my file without creating a new file and also preserving the order of the contents in my file.
Would there be another command besides ...
0
votes
2answers
38 views
Find max values within Column 1 range and print column2
I have a representative dataset
35.5259 327
35.526 326
35.526 325
35.5261 324
35.5262 323
35.5263 322
35.5264 321
35.5265 320
35.5266 319
35.5268 318
# ...
0
votes
2answers
54 views
How to replace a fix location in each line in a file
I have a file as below, how do I replace the $number (location 19-30) and (location 59-70) with its number in cents.
update: All fields are required to stay at the same fixed location.
input:
...
3
votes
3answers
254 views
How can I convert a float to ratio?
I would like to convert a float to a ratio or fraction.
Do we have the option to convert a float 1.778 to ratio as 16:9 or 16/9 in bash, in a fashion similar to Python's fractions module (Fraction(1....
-1
votes
1answer
170 views
Hexadecimal color codes grep regex in makefile
So I have a text file containing valid and invalid hexadecimal color codes. I want to be able to filter out the invalid codes and print just the valid ones. For a code to be valid it must have a hash ...
2
votes
2answers
37 views
Comparing two columns(numeric) in two different text files
I have two text files with space-delimited fields (03 fields) in Linux environment:
File1 : ram-service1.txt
RAM used Program
23500 kb firewalld
14780 kb tuned
10140 kb polkitd
5370 ...
1
vote
6answers
1k views
how to extract maximum and minimum value from column 1 and column 2
Dear all I have a big data file lets say file.dat, it contains two columns
e.g file.dat (showing few rows)
0.0000 -23.4334
0.0289 -23.4760
0.0578 -23.5187
0.0867 -23.5616
0....
1
vote
3answers
74 views
Converting base-6 number to base-2 (binary) using commands or a program
I have a base-6 number generated by rolling a physical six-sided dice and want to convert it into base-2 / binary.
Are there built-in commands or programs that can do that for me?
If not, what is ...
0
votes
1answer
180 views
regex (or other way) to find numbers followed by a space
I'm running the following
awk '$2 ~ /^[0-9]\s+/ {print($2)}' file >> file1
Not sure how to correct this to only select lines where digits are followed by a space?
-1
votes
2answers
54 views
How I can change 01 and 25 in 001 and 025 without change 125 in 0125?
I have a file like this
Thomas_4567_01
Luc_2344_125
Ann_5780_25
I would like an outpu like this
Thomas_4567_001
Luc_2344_125
Ann_5780_025
How I can change 01 and 25 in 001 and 025 without change ...
1
vote
5answers
139 views
sed/awk: replace numbers in a line after last occurance of '.'
I have the following tcpdump stream:
Current:
07:36:03.848461 IP 172.17.3.41.33101 > 172.17.3.43.17408: UDP, length 44
07:36:03.848463 IP 172.17.3.42.33101 > 172.17.3.43.17409: UDP, length 44
...
4
votes
2answers
79 views
Why does gawk (sometimes?) think 2.0e-318 > 2.0?
I am trying to find the maximum value of a column of data using gawk:
gawk 'BEGIN{max=0} {if($1>0+max) max=$1} END {print max}' dataset.dat
where dataset.dat looks like this:
2.0
2.0e-318
...
3
votes
2answers
123 views
Sort numerical column
I'm trying to sort a file based on a particular position but that does not work, here is the data and output.
~/scratch$ cat id_researchers_2018_sample
id - 884209 , researchers - 1
id - 896781 , ...
0
votes
5answers
176 views
Max, Mean, Min for range within column of data
I have data structured like this :-
X 43808873 A 14
X 43808874 G 7
X 43808875 T 15
X 43808876 A 13
X 43808877 T 4
X 43808878 C 13
X 43808879 T 11
X ...
-1
votes
1answer
37 views
sum the values of all occurrences parameters
we have the following file ,
in file we can see the same parameters names with their values
more file.txt
gg_ff_rr =56
gg_ff_rr =56
gg_ff_rr =52
jk_ll_tt=24
jk_ll_tt=24
jk_ll_tt=24
pl_pl_uu=10
...
1
vote
1answer
136 views
how to change color of text based on value?
I want to use cat to display a file content and changes the text color to red if the number is larger than 50.
Example:
cat file1
would give
1 33 56
the number 56 will be displayed in red ...
2
votes
2answers
179 views
How to get max, min, and mean from values in column 4
I have data structured like this
X 43808504 G 1 ^]. <
X 43808505 C 3 . 4
X 43808506 T 8 . ?
X 43808507 G 5 . C
I want to get the max (8), min (1), and ...
0
votes
1answer
222 views
Calculating average in awk based on several column conditions in csv
I currently have the following csv format, it's much larger but I've taken a chunk of it for now.
EV,"Houston","-7.0"
AB,"Boston","19.0"
OO,"Mystic","13.0"
AB,"Boston","-12.0"
EN,"New York City","9....
2
votes
1answer
160 views
Awk inconsistent behavior filtering scientific notation numbers
I recently found that for some very small values, filtering using AWK doesn't seem to behave correctly. As in the following file test_loc.txt:
10:10000018 10 0.4505
X:99997421 X 0.95508
X:99997626 ...
1
vote
3answers
66 views
Find the maximum value of Column3 and print the values of Column1 and 2 only
In Unix, I am trying to find a command that would find the maximum value in Column3 and print the corresponding values from Column2 and Column1 (but not from Column3) in a new file.
Column1 ...
0
votes
1answer
19 views
re-write column's values to obtain previous line's value if exceed number 1
I have a file containing the following list of numbers:
0.1131492
0.1231466
0.1327564
0.1017683
5.4356130
0.1360532
5.4258129
0.1433982
0.1124752
.
.
.
I would like to re-write this list of numbers ...
0
votes
1answer
153 views
Sort file by row and with scientific numbers
I have a file F.tsv with 13 columns, the last column (the 13th columns) looks like this:
2.1e-06
0.58
10
8.7e-22
0.0014
0.034
9.5
0.67
0.67
0.68
9.2
8.4e-22
9.7
I've tried sort -k 13 F.tsv but it ...
1
vote
5answers
3k views
Sum of even and odd numbers of odd and even lines
I would like to write a shell script program that gives the sum of even and odd numbers in a given file's odd and even lines.
I would like to use:
sed -n 2~2p
and
sed -n 1~2p
but I am not even ...
0
votes
1answer
65 views
Compare two lines using second columns
http://0-0.latam.corp.yahoo.com/ 6656
http://0-0.latam.corp.yahoo.com/nonEtAk 6670
http://1.avatar.yahoo.com/ ...
0
votes
3answers
59 views
how to read file line by line and delete the line have number less than 20% shell scripting
i need help.
i have this file
i want read it line by line with for loops and delete the line less than 20% the number will with list %CPU .
%CPU PID USER UID COMMAND %MEM
71 9136 ...
2
votes
3answers
145 views
Generates random numbers with 11 digits
I need a command that generates random numbers with 11 digits. How can this be done?
4
votes
4answers
345 views
How can I adjust nth number in a line?
File contents:
RANDOM TEXT num1=400 num2=15 RANDOM TEXT
RANDOM TEXT num1=300 num2=10 RANDOM TEXT
RANDOM TEXT num1=200 num2=5 RANDOM TEXT
I would like to subtract 5 for ...
0
votes
1answer
37 views
How to add an average at then end of a table of values
I've been teaching myself Linux for 3 months now and I'm using the grep command but I also want an average of all of my extracted values.
Command:
grep -o "201[1-4].[0-9][ -9]" file1.txt file2.txt ...
6
votes
2answers
225 views
AWK if number less than previous number, how to add previous number to current number, and from then on
Have a field with distance data from GoPro. Need to concat the files together but the distance starts at 0 on each new csv file as video files are of only a set duration.
So how would it be possible ...
0
votes
2answers
525 views
How to calculate max and min for each unique entry
I need to find the max and min of each entry in my file:
#subset of my file
NEUTRON 20.900103
PION- 0.215176
PION- 22.716532
NEUTRON 8.043279
PION+ 1.374297
PION- 0.313350
...
0
votes
1answer
60 views
awk rounding error using int() in connection with log()
For later formatting I need to calculate the logarithm, base 10. Fine so far:
$ awk 'BEGIN { print log(1000) / log(10) }'
3
$ awk 'BEGIN { print log(1001) / log(10) }'
3.00043
I need to kick the ...
13
votes
4answers
3k views
How to increment the value of a (decimal) variable (with leading zero) by +1?
I have a file in the name of Build.number with the content value 012 which I need to increment by +1.
So, I tried this
BN=$($cat Build.number)
BN=$(($BN+1))
echo $BN >Build.number
but here I am ...
1
vote
2answers
51 views
Replace single field numbers with double field numbers (1->01)
I have the following file, beginning with
male 9
male 11
male 9
male 1
female 4
female 13
male 14
If I use
sort -u -k1,1 -k2,2n
this returns
female 13
female 4
male 1
male 11
male 14
male 9
male 9
...
0
votes
2answers
40 views
Highest Columns Number Records Issue
Highest Columns Number Records Issue
I have a test.txt file with contents as follows:
1:2:3
123:5
34589:5:0
34567:8:7
781:9:09
Could you please help me getting the following output from that test....
0
votes
1answer
38 views
add multiple rows for a range of variable and condtional column to a file
I have a file with 4 columns. First two columns are for x and y position (in integers) and third, fourth column for an arbitrary field value.
1 1 0.5 1.2
1 2 1.7 2.3
1 3 2.0 2.2
2 1 1.4 2.5
2 2 1.6 ...
1
vote
2answers
108 views
get count with grep
im working on sun10 Solaris os,i have a process that returns table as by using this command dmh -q 12 the below:
*PROFILE PRIORITY COMM_TYPE QID # OF MSGS ATTRIBUTES/VALUES*
13 ...
-3
votes
1answer
42 views
Adding a sequence of one hundred numbers to a starting number
If I have file like this:
1
19
24
119
201
230
... then I want output like this:
1 2 3 4 .........100
19 20 21 22 .......119
24 25 26 ...........124
119 120 121 .......219
201 ...
0
votes
3answers
295 views
multiply specific column in a file which consists of thousands columns
The in file looks like this
-17.3644 0.00000000 0.00000000 ....
-17.2703 0.00000000 0.00000000 ....
-17.1761 0.00000000 0.00000000 ....
-16.5173 0.00000000 0....
