I have test.json file with different lengths of rows. Some fictitious example:

{ a: 123, b: sd, c: x45, d: 1, e: '' }
{ a: 5, b: bfgg, c: x4c, d: 31, e: '' }

I want to cut the whole substring after d - part and get back just for every line:

{ a: 123, b: sd, c: x45 }
{ a: 5, b: bfgg, c: x4c }

I found here a similar question and tried to adapt my problem to it:

echo test.json |  sed 's/. d:/' > newtest.json

I need to do it for the whole file, not only one line.

  • Strictly speaking, that's not JSON (need [double] quotes around any strings) </pedantic> – PixnBits Nov 18 '14 at 20:33
sed '/d:/s/, d:[^}]*/ /' test.json

it will walk through the whole file and remove on each line with d: all , d:.* part till } symbol (} symbol will stay on the line).


@Rush's answer using sed is likely the best way to solve this but here's how you can do it using awk too:

$ awk -F ', d.* ' '{print $1, $2}' file.txt 
{ a: 123, b: sd, c: x45 }
{ a: 5, b: bfgg, c: x4c }

The above uses awk to split the data on , d.*. This produces 2 fields of data, $1 and $2 which contain the data resulting from AWK's splitting of the strings.


Through Perl which uses Positive lookahead assertion,

$ perl -pe 's/,\s+d:\s+.*?(?= })//' file
{ a: 123, b: sd, c: x45 }
{ a: 5, b: bfgg, c: x4c }

Through Python,

import sys
import re
file = sys.argv[1]
with open(file, 'r') as f:
    for line in f:
        line = re.sub(r',\s+d:\s+.*?(?= })', r'', line)
        print(line, end='')

Paste the above code in a file script.py and run the srcipt on your terminal by,

python3 script.py /path/to/the/input/file


{ a: 123, b: sd, c: x45 }
{ a: 5, b: bfgg, c: x4c }
  • Thank you, I don't use Perl, but do a lot of code in Python. Actually I have interested on specific solution in Linux (like awk, sed or probably grep), because I want collect some experience in this area. Thank you anyway. – Guforu Nov 17 '14 at 15:25

If you are doing much with transforming json on the command line, it is well worth your time to get the jq tool and learn to use it.


While the answers above show that you can do minimal transformations without actually parsing the json, eventually you will either reinvent a buggy json parser via regexp or revert to using the native json parser in your language of choice.

jq is fast, simple to use and a very handy tool to have in your toolbox.

BWT, your test data is invalid json, which will complicate the solutions above. If you fix it to

{ "a":123 , "b": "sd", "c": "x45", "d": 1, "e": "" }
{ "a":5 , "b": "bfgg", "c": "x4c", "d": 31, "e": "" }

Then this jq command will do want you want

 jq -c '{a,b,c}' test.json

  • thanks, this is cool. Sorry, my json is valid, I just create some example without strings "" (in the real file all keys and values in the dictionary are strings) – Guforu Nov 18 '14 at 7:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.