2

I have a .txt file which is a collection of numbers. It looks like this:

[...]
6 2
7 999
8 105
9 78
10 45
11 6666
[...]

up to a 4-digit index.

Now I want to import the numbers as a list in python. That means I need a list which is

[2, 999, 105, 78, ...]

So I would like to get rid of the indices, the space after it and set a comma after every entry. I am not familiar with these, but I know it should work with a regular expression. Can you help?

12
  • 1
    Are the : part of the file? Do all lines have two numbers? Should you ignore lines that don't have two numbers? Or if the second number is bigger than 4 digits?
    – aviro
    Mar 4 at 10:15
  • And where is the "replace" part?
    – aviro
    Mar 4 at 10:16
  • @aviro No, the colons just mean that the file continues. It start with index 1 and ends with 4570. All lines have two numbers: The Index and the number that I want to.
    – Lereu
    Mar 4 at 10:20
  • 1
    So you don't actually need regular expression. You can just take the second column from all the lines and form them in the way you requested, right? And you're looking for a Linux shell command, not a python code?
    – aviro
    Mar 4 at 10:48
  • Python is off-topic here. This should be migrated to Stackoverflow.
    – Joooeey
    Mar 5 at 12:34

5 Answers 5

5

Using Python as requested:

file_path = 'file.txt'

with open(file_path, 'r') as file:
    c2 = [int(line.split()[1]) for line in file]

print(c2)
4

Since all of your lines include two numbers, there's no need to use regular expressions. You can just use awk command.

One liner:

awk 'NR==2 {SEP=", "} {RES=RES SEP $2} END {printf "[%s]\n", RES}' file.txt

Multi line for better readability:

awk '
  NR == 2 { SEP=", " } 
  { RES = RES SEP $2 } 
  END { printf "[%s]\n", RES }
' file.txt

Result:

[2, 999, 105, 78, 45, 6666]
4
  • Ah duh, of course! I would upvote, but I already had :)
    – terdon
    Mar 4 at 13:58
  • NR==2 {SEP=", "} {RES=RES SEP $2} = {RES=RES SEP $2; SEP=", "}. Don't use all upper case variable names though so that a) you can't clash with built-in variable names of which there are far more than most people realise and they vary between awk versions, and b) so your code doesn't look like you're using built-in variables when you aren't and so it obfuscates your code.
    – Ed Morton
    Mar 10 at 16:15
  • 1
    @EdMorton I usually uppercase variables in awk to differentiate "local" variables (used in only one scope) and "global". For instace, I use RES in every line, and also at the END section. But I accept your input.
    – aviro
    Mar 10 at 17:35
  • Understood. FWIW usually if it's a long script (and so benefits from a convention) then I start the variable with upper case, NumChars, if it's required to keep it's value across the whole execution (i.e. some globals), lower numChars, otherwise (i.e. the other globals and all locals). I don't find differentiating all globals from locals useful in awk other than declaring function-local variables as such by including them at the end of the function arg list so I can write function foo( i) {i=10} ...{for (i=1;i<=5;i++) foo()} without the global i clashing with the local i.
    – Ed Morton
    Mar 10 at 19:18
3

Using Perl

~$ perl -ane 'push @a, $F[1]; END { print "[", join(", ", @a), "]\n" };'  file
[2, 999, 105, 78, 45, 6666]

Above, Perl is used in -ane linewise, autosplit, non-autoprint mode. No BEGIN block is necessary. The @a array auto-vivicates and elements can be pushed onto it--in this case, the $F[1] second column after splitting on whitespace. (FYI, push @a, split /\s+/, $F[1] works as well, but the split /\s+/ is superfluous here). In the END block, the @a array elements are joined as requested, and print is called with opening and closing square-brackets.


Using Raku (formerly known as Perl_6)

~$ raku -ne 'my @a.push: .split(/\s+/).[1]; END @a.join(", ") andthen put "[$_]";'   file
[2, 999, 105, 78, 45, 6666]

Above, Raku is used in -ne linewise non-autoprint mode (fewer command-line flags in Raku as compared to Perl). No BEGIN block is necessary, however the @a array must be called with my (or our) to set scope. The input line $_ is split on \s+ whitespace (.split is short for $_.split), and [1] the second column is pushed onto @a. In the END block no parens are required: the @a array elements are joined as requested, andthen reloads the $_ topic, in order to simplify the final output call to "[$_]" with opening and closing square-brackets (put adds newline at the end).


Sample Input:

6 2
7 999
8 105
9 78
10 45
11 6666
3

Assuming that your input is a key-value two-column text file with no header, you can use the transpose function of datamash to convert the second column to a comma-separated row:

$ cat infile 
6 2
7 999
8 105
9 78
10 45
11 6666

$ awk '{print $2}' infile | datamash transpose -t, 
2,999,105,78,45,6666

Putting the output between square brackets is trivial. For example, pipe the result to this command:

$ sed 's/^/[/ ; s/$/]/'
1

I would use awk to print the second field, and then a simple sed replacement to add a [ at the beginning and replace the trailing , with a ]\n:

$ awk '{ printf "%s, ",$2}' file | sed 's/^/[/; s/, $/]\n/'
[2, 999, 105, 78, 45, 6666]

Note that this assumes that everything in the file is an index and its value. No headers, nothing that isn't desired in the output. If that is not the case, please update your question with an input example that includes these.

7
  • You rarely need a pipe to/from sed when you're using awk. If you ever think you do then you should take another look at your awk script :-). The only time I've found it useful would be if I had STRING-separated input but didn't have GNU awk so couldn't use a multi-char RS and it was convenient to do sed 's/STRING/X/g' file | awk -v RS='X' '...' but even then you don't NEED sed.
    – Ed Morton
    Mar 10 at 16:16
  • @EdMorton yes, I know you feel that way (you might have made that point once or twice already ;) ). Nevertheless, there are many things I find easier in sed and many I find easier in awk so I will often mix and match in whatever way makes the most sense for what I am trying to do.
    – terdon
    Mar 10 at 17:43
  • This may well be my ignorance, but if forced to do this in awk, I would have gone for something like awk '{ k = k ? k", "$2 : $2 } END{ sub($NF, /,/,""); printf "[%s]\n",k}' file and I find that both harder to parse, and harder to think up than just passing to sed for substitutions. Unless writing for a production system with very limited capacity, the tiny little change in efficiency that not starting a second process would bring pales before the extra weight of storing everything in memory anyway.
    – terdon
    Mar 10 at 17:49
  • That version would fail if $2 had the value 0 (or "" which may be valid in other contexts) so I wouldn't do that. It's also always worse to add something then remove it than to not add it in the first place. FWIW I'd write awk '{ k = k s $2; s=", " } END{ printf "[%s]\n", k }' or awk '{ k = (NR>1 ? k s : "") $2} END{ printf "[%s]\n", k }' or awk 'BEGIN{ s="[" } { printf "%s%s", s, $2; s=", " } END{ print "]" } or similar.
    – Ed Morton
    Mar 10 at 19:24
  • 1
    No worries, @EdMorton. I have been known to make very similar comments myself! Mostly for the grep foo file | awk antipattern. I don't mind, I just don't think it's always worth wrangling one tool only for little jobs like this.
    – terdon
    Mar 10 at 19:55

Not the answer you're looking for? Browse other questions tagged .