3

Todd Schneider uses the following command in his NYC Taxi Data Analysis to prep a CSV file for PostresSQL. 

sed $'s/\r$//' $filename | sed '/^$/d' | ...

The second sed command removes blank lines. What does the first one do?

Improve this question
2
  • he tries to remove carriage return char \r at the end of the line. Looks doubtful, I see the most windows files with CRLF at the end (not single CR)
    – RomanPerekhrest
    Oct 30, 2017 at 17:09
  • 1
    Not doubtful. The trailing newline, being the line terminator, will not be present in the current line as seen by sed. Hence, the last character on the line is \r. If the file contained CR line endings, sed would see the whole file as a single line.
    – glenn jackman
    Oct 30, 2017 at 17:24

1 Answer 1

Reset to default
8

According to man bash:

  Words of the form $'string' are treated specially.  The word expands to
   string,  with backslash-escaped characters replaced as specified by the
   ANSI C standard.  Backslash escape sequences, if present,  are  decoded
   as follows:                                      
          \a     alert (bell)                       
          \b     backspace                          
          \e                                        
          \E     an escape character                
          \f     form feed                          
          \n     new line                           
          \r     carriage return                    
          \t     horizontal tab                     
          \v     vertical tab                       
          \\     backslash                          
          \'     single quote                       
          \"     double quote                       
          \?     question mark                      
          \nnn   the  eight-bit  character  whose value is the octal value
                 nnn (one to three digits)          
          \xHH   the eight-bit character whose value  is  the  hexadecimal
                 value HH (one or two hex digits)   
          \uHHHH the  Unicode (ISO/IEC 10646) character whose value is the
                 hexadecimal value HHHH (one to four hex digits)
          \UHHHHHHHH                                
                 the Unicode (ISO/IEC 10646) character whose value is  the
                 hexadecimal value HHHHHHHH (one to eight hex digits)
          \cx    a control-x character              
  The  expanded  result  is  single-quoted, as if the dollar sign had not
   been present.

Consequently $'s/\r$//' deletes an ASCII CR at end of line. It's a crude way to convert Windows line endings to UNIX line endings.

Improve this answer
2
  • Ahh great. My plan is to analyze this in Windows so I shouldn't need to do this modification.
    – rhaskett
    Oct 30, 2017 at 17:47
  • Then you need a Windows-specific version of sed (and other similar utilities). Plain translations such as Cygwin won't work.
    – Satō Katsura
    Oct 30, 2017 at 17:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .