0

I have a text file containing columnar data and I would like to filter out lines that meet a specific criteria: the data in the second-to-last column begins at column 161 rather than 162. So given this:

EP10101 12011   SC0     NCI Application Security                                    DLS3270 SC040P20Maintain User Access                                        MF20170901150328000NGS2    20170901150328000            
EP10101 12011   SC0     NCI Application Security                                    DLS3270 SC040P20Maintain User Access                                        PF20170901150328000NGS2    20170901150328000            
OS10101 12011   DLS     NCI Dealer Systems                                          DLS3270 DLS20P10RDR Maintenance                                              B20171016171130000NGS2    20171016171130000            
OS10101 12011   DLS     NCI Dealer Systems                                          DLS3270 DLS07P10NCFI Lease/Loan Payout Query                                 B20171016134250000NGS2    20171016134250000            
OS10101 12011   DLS     NCI Dealer Systems                                          DLS3270 DLS20P10RDR Maintenance                                              B20171016143354000NGS2    20171016143354000

I'm left with this after processing:

OS10101 12011   DLS     NCI Dealer Systems                                          DLS3270 DLS20P10RDR Maintenance                                              B20171016171130000NGS2    20171016171130000            
OS10101 12011   DLS     NCI Dealer Systems                                          DLS3270 DLS07P10NCFI Lease/Loan Payout Query                                 B20171016134250000NGS2    20171016134250000            
OS10101 12011   DLS     NCI Dealer Systems                                          DLS3270 DLS20P10RDR Maintenance                                              B20171016143354000NGS2    20171016143354000

How can I achieve this with command line tools?

UPDATE

I want to delete the lines with the matched pattern and also need to delete the last columns in the file. So my ultimate output should be like below.

OS10101 12011   DLS     NCI Dealer Systems                                          DLS3270 DLS20P10RDR Maintenance          
OS10101 12011   DLS     NCI Dealer Systems                                          DLS3270 DLS07P10NCFI Lease/Loan Payout Query

closed as unclear what you're asking by Jeff Schaller, Stephen Rauch, G-Man, Satō Katsura, Anthony Geoghegan Nov 7 '17 at 14:23

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    Are the lines wrapped like that, such that groups of them should go together? – Jeff Schaller Nov 6 '17 at 16:16
  • 1
    In your example there is no line longer than 84 characters, and all lines have trailing spaces. Please post a real example. – AlexP Nov 6 '17 at 17:19
  • 1
    It's not very clear what you're asking. What is the criteria for lines to be deleted? – Time4Tea Nov 6 '17 at 20:57
  • @αғsнιη I have posted some input data from the file in the question. IF you look at the last 2 columns in the data "MF20170901150328000NGS2"(161-184) and "B20171016171130000NGS2"(162-184). I need to delete the whole row from the file if i find any column with positions (161-184). In the above input which i mentioned in the question the first 2 lines needs to get deleted.Layer provided a solution to this. I need to include one more logic in which it should delete the last 2 columns from the entire file also. So finally i need to delete the complete line if it matches (161-184) position..contd – Ravi Rock Nov 8 '17 at 11:56
  • and also it needs to delete the last 2 columns from the entire file i.e (B20171016171130000NGS2 20171016171130000). Please let me know whether my requirement is clear or not. In my updated question i have provided the final ouput. – Ravi Rock Nov 8 '17 at 11:57
1

Bit of a guess since the question is not entirely clear but

sed '/^.\{160\}\S\+/d' file

If this finds "data" (non-whitespace) at column 161 the line is deleted. All other lines are printed. This results in the output:

OS10101 12011   DLS     NCI Dealer Systems                                          DLS3270 DLS20P10RDR Maintenance                                              B20171016171130000NGS2    20171016171130000
OS10101 12011   DLS     NCI Dealer Systems                                          DLS3270 DLS07P10NCFI Lease/Loan Payout Query                                 B20171016134250000NGS2    20171016134250000
OS10101 12011   DLS     NCI Dealer Systems                                          DLS3270 DLS20P10RDR Maintenance                                              B20171016143354000NGS2    20171016143354000

(Can't help but think there is a better approach but this is all OP provides.)

  • Hi Layer. Is it possible to delete the last 2 columns if the same criteria matches? – Ravi Rock Nov 7 '17 at 15:30
  • You want something like this? sed 's/^\(.\{160\}\S.*\)..$/\1/g' file – B Layer Nov 7 '17 at 16:01
  • Actually, maybe what you mean is to delete the entire column blocks rather than one-character wide columns like those we have been talking about. If so try this: sed 's/^\(.\{160\}\)\S\+.*/\1/' file – B Layer Nov 7 '17 at 23:06
  • Thanks Layer. I got the correct scenario now. After successfully deleting the lines with the match i also need to delete the last columns in the file. So my ultimate output should be like I have edited the question. – Ravi Rock Nov 8 '17 at 8:45
  • @Layer Can you help on this one? – Ravi Rock Nov 8 '17 at 15:01
-1

Use below command as tested it worked fine

for i in $(awk '{if ((length($1) > "160") && (length($1) < "184"))print NR}' filename); do sed "$i"d p.txt; done
  • Why use sed if you’re already using awk? – Stephen Kitt Nov 7 '17 at 21:44
  • @StephenKitt Its Just an alternative As we can achieve using both sed and awk – Praveen Kumar BS Nov 8 '17 at 6:55
  • My point is that you can do the whole operation in AWK, no need to use sed in a loop afterwards. – Stephen Kitt Nov 8 '17 at 7:04
  • @StephenKitt Yes agree – Praveen Kumar BS Nov 8 '17 at 7:09

Not the answer you're looking for? Browse other questions tagged or ask your own question.