1

What is the simplest way to search for a word in a file given in the command line, with this format

./<file1> -f <file2> --edit <id> <column> <value>

I want to search a persons <id> in <file2> and change the word in <column> given with <value>.

I have tried

awk -F '|' -v ID="$4" -v Column="$5" \
           -v Value="$6" 'ID==$1 {$Column=Value ;}1' \
           OFS='|' $2>NewFile
mv NewFile $2 ;

But I want it done without a temporary file

For example:

1000|text1|text2|text3
1001|text4|text5|text6
1002|text7|text8|text9

After I execute

./<file> -f file2 --edit 1001 2 Marios

it should have this change:

1000|text1|text2|text3
1001|Marios|text5|text6
1002|text7|text8|text9
  • If you have GNU awk (gawk), you can change a file in-place (no temporary file) by using the adding the option -i inplace as in awk -i inplace -F '|' -v ID="$4" .... – John1024 Oct 16 '16 at 0:32
  • It's not clear whether the goal is to 1. avoid the bother of writing code to handle temporary files, (in which case sed -i would suffice, although it secretly creates and mvs a temporary file). 2. overwrite only the original file, (where something more low-level would be required, as in Kaz's answer). Please clarify. – agc Oct 17 '16 at 16:33
2

Editing text files without a temporary file is a bad idea and usually isn't done in Unix scripting. It requires rewriting the entire file, or at least the suffix portion which is affected by an edit. If the write is interrupted, the file is corrupt.

Of course, we do this everyday with text editors: we hold files in memory and overwrite them on disk when we save. The differences are that all decent editors keep at least one backup, unless that feature is explicitly disabled (and that is an extra file, not acceptable to you, presumably), and that editors are interactive: if a save fails for any reason (disk full, system crash, whatever) a person knows about it. In cases when it isn't a crash, the editor is still running and has the file in memory in spite of the failed save. The user can execute commands to save the file elsewhere, or to try saving again after taking some actions outside of the program to fix some situation.

Solution in TXR: overwrite from in-memory copy with no backup or recovery strategy:

#!/usr/local/bin/txr --lisp

(defvarl myname [*args-full* 2])

;; check for required arguments syntax
(unless (and (= (length *args*) 6)
             (equal [*args* 0] "-f")
             (equal [*args* 2] "--edit"))
  (put-line `usage: @myname -f <file> --edit <col1-key> <col-num> <replace>`)
  (exit 1))

;; do in-memory update and overwrite
(let ((file [*args* 1])
      (key [*args* 3])
      (col (pred (tointz [*args* 4]))) ;; pred, because [f #] is zero based
      (val [*args* 5])
      (ss (make-strlist-output-stream))) ;; in-memory string list stream

  ;; awk into memory
  (awk (:inputs file) ;; input from file
       (:output ss) ;; output stream is in-memory string list
       (:set fs "|") ;; field separator is pipe
       ((equal [f 0] key) (set [f col] val)) ;; do replacement
       (t)) ;; true condition with no action -> default print action

  ;; overwrite original file with string list
  (with-stream (out (open-file file "w"))
    (put-lines (get-list-from-stream ss) out)))

Session:

$ diff -u data.orig data
$ ./inplace 
usage: ./inplace -f <file> --edit <col1-key> <col-num> <replace>
$ ./inplace -f data --edit 1001 2
usage: ./inplace -f <file> --edit <col1-key> <col-num> <replace>
$ ./inplace -f data --edit 1001 2 Marios
$ diff -u data.orig data
--- data.orig   2016-10-16 08:05:03.233736781 -0700
+++ data    2016-10-16 08:15:57.412394022 -0700
@@ -1,3 +1,3 @@
 1000|text1|text2|text3
-1001|text4|text5|text6
+1001 Marios text5 text6
 1002|text7|text8|text9
  • You know, TXR does not look that scary when you look at it (+1). PS: Add a disclaimer, we care a lot about disclaimers here on SE. (and you're the author of the language, right?) – grochmal Oct 18 '16 at 14:32
0

One thing you seem to be looking for is command line parsing. A decent parsing can be done with a case in a POSIX shell.

Next AWK is completely fine to perform that transformation. To do it in place you have two alternatives: use GNU awk (with -i) as John suggests, or use a temporary file. Below is an example using mktemp, although mktemp is not POSIX it is around in almost any *nix system

#!/bin/sh

while test $# -gt 0
do
    case "$1" in
      -f)
      file="$2"
      shift
      shift
      ;;
      --edit)
      id="$2"
      column="$3"
      value="$4"
      shift
      shift
      shift
      shift
      ;;
      *)
      echo "Usage:"
      echo "  $0 -f <file> --edit <id> <column> <value>"
      exit
      ;;
  esac
done
# debug
echo "edit [$file] in [$id] change column [$column] to [$value]"

tmpf=`mktemp`
awk -v FS="|" -v OFS="|" "/^$id/ { \$$column = \"$value\" }1" "$file" > "$tmpf"
mv "$tmpf" "$file"

The idea is to escape the right characters when passing the program to awk. Assuming that the script above is called script.sh you can simply do:

./script.sh -f myfile --edit 1001 3 "It's a me Mario"

This still have some problems that I address below to not cluster it. First you should also check for an empty number of arguments:

if test $# -eq 0
then
    echo Usage
    exit
fi

And second, using a plain mv is sometimes dangerous. Notably when something goes wrong and the script produces no output. It is always good to add something like this around an mv that will overwrite the input:

if test -s "$tmpf"
then
    mv "$tmpf" "$file"
else
    echo Something went wrong
fi
  • I've already done what you've said but my problem is that i don't want a temporary file in order to modify my original . – Μάριος Τσοκανάς Oct 16 '16 at 13:32
  • @ΜάριοςΤσοκανάς - But that is only a question of killing mktemp (and the mv) and adding -i to the AWK line. I like mktemp because it is more portable than -i, and it make sure that the temporary file do not conflict. To be pedantic awk -i does create a temporary file anyway and move it over after completion, it just does it behind the scenes, – grochmal Oct 16 '16 at 18:32
0

Suppose that you have a file like below and you need to get the output as mentioned:

Input file:

1000|text1|text2|text3
1001|text4|text5|text6
1002|text7|text8|text9

Output should be :

1000|text1|text2|text3
1001|Marios|text5|text6
1002|text7|text8|text9

Please try this out:

grep -rn "1001" file1 | awk -F '|' '{export $2=<new value>;print $2}' \;
  1. Basically grep -rn "1001" file1 will give the line:

    1001|text4|text5|text6

  2. Once you get the above output,then use awk to change the value of the 2nd column, (which is Field separated by "|"), and print the value.

I don't have the environment to execute it at the moment, but I am sure the logic will be helpful for what you want to achieve.

Conclusion: I would suggest not to use temporary files in scripts, because if we use more of this temporary file assignments, the performance of the server will degrade because more I/O will happen in the server, in turn degrading its performance and making the server slow.

0

A one-liner shell function wrapper around some simple sed code:

# Usage: foo <file2> <id> <column> <value>
foo() { sed -i "/^$2|/s/[^|]*/$4/$3" "$1" ; }

Example:

 foo file2 1001 2 Marios  ;  cat file2

Output:

1000|text1|text2|text3
1001|Marios|text5|text6
1002|text7|text8|text9
  • 1
    sed -i uses a temporary file. I ran strace on it and see rename("./sedTZD1bN", "testfile") near the end. – Kaz Oct 16 '16 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.