6

I recorded my current session using the script command, and all information was saved in a typescript file, but when I opened it using Vim there were a lot of ^Ms due to carriage returns.

I tried to convert this file to the Unix format using the dos2unix command, but I was unable to do so. It was giving this error:

dos2unix: Binary symbol 0x1B found at line 2,dos2unix: Skipping binary file typescript. 

I was just curious why it is happening. Why does script produce output in CR/LF form and not simply in LF form?

13

typescript saves everything what is sent to your terminal which may include escape sequences for positioning, colors, brightness etc. (0x1B is the ESC character.) The terminal output contains CR and LF even if the usual line ending in text files is different.

The character 0x1B makes dos2unix assume your input might be a binary file. Because modifying a binary file might not be useful, dos2unix rejects to do this by default. Apart from this there is no problem with the escape character.

You can try dos2unix -f to force conversion of the seemingly binary file. This way you tell it that you know that modifying the line endings in this file is safe.

Or use vim to remove the CR characters. :%s/CTRL+V CTRL+M ENTER

In case there might be more than one CR per line :%s/CTRL+V CTRL+M//g ENTER

|improve this answer|||||
  • i am not able to understand clearly what problem the ESC character is creating here and also why typescript is using the CR/LF format when it is actually a UNIX file – Noshiii Jul 24 '19 at 12:09
  • @Noshiii The escape character is present since you must at some point have pressed it (which makes the file "not an ASCII file"). The typescript contains CRLF linefeeds (as in a DOS text file) because that's what the terminal sees. – Kusalananda Jul 24 '19 at 13:02
  • @Kusalananda well i don't remember pressing ESC character, i have tried 2 3 times but i get the reason. Thanks a lot – Noshiii Jul 24 '19 at 13:04
  • 3
    @Noshiii Well, codes for changing colour also outputs ESC. – Kusalananda Jul 24 '19 at 13:05
  • 9
    @Kusalananda The escape character is probably part of the prompt or some command output. (cursor positioning, colors, ...) – Bodo Jul 24 '19 at 13:06
9

0x1b should be part of a VT100 terminal color code or similar.

http://www.termsys.demon.co.uk/vtansi.htm

<ESC> represents the ASCII "escape" character, 0x1B.

In general what script records is not what you see but what the terminal sees, so it's a raw terminal recording. It can even record timing information for scriptreplay to show what was going on in the same speed it originally happened...

And raw terminal just uses \r to move the cursor to the left. Even though it's no longer used in plain text files, it's still very much present in terminals. There's \r everywhere, it's just that it's invisible to you most of the time.

If you go through the kernel sources, you can find things like this:

static void puts_raw_fixed(int (*puts_raw) (const char *s, int len),
                           const char *s, int count)
{
        const char *s1;

        /* Output '\r' before each '\n' */
        while ((s1 = memchr(s, '\n', count)) != NULL) {
                puts_raw(s, s1 - s);
                puts_raw("\r\n", 2);
                count -= s1 + 1 - s;
                s = s1 + 1;
        }
        puts_raw(s, count);
}

Consider it a technical implementation detail of terminals and the like... and try to ignore it (as long as you don't do fancy terminal things).

If you don't want a raw recording, perhaps use the good old command > output.txt redirection instead of script, or simply... copy & paste from the terminal itself. That should produce output that is free of \r, unless the command itself produced raw data like that.

Otherwise perhaps see this question on Removing control chars (including console codes / colours) from script output for post-processing typescript files. Results might vary however. There's a lot of stuff going on in a terminal without you realizing, and script records it all.

|improve this answer|||||
  • ok i get it, just one doubt that what problem ESC character is creating while i try to convert the typescript file into UNIX format ? – Noshiii Jul 24 '19 at 12:45
  • 3
    dos2unix works with plain text files. typescript is not a plain text file, but a raw (binary) recording of terminal activity. For dos2unix that should just be a safety measure to not needlessly break your files (dos2unix is irreversible if you cannot guarantee all \n were \r\n before.) – frostschutz Jul 24 '19 at 12:47
  • 3
    @Noshiii when dos2unix detect non-ascii character (for example ESC) it assumes that it is a binary file. And generally modifying binary files will corrupt it. Try replacing all bytes that are 0x0d, 0x0a (\r\n) to 0x0a (\n) in Google Chrome and you will most likely cause the program to crash. Try doing the same in a JPEG file and you will see the file corrupted. Doing it to an mp3 file will result in the file sounding like it was recorded under water (sometimes called "cooked" mp3s). This is because 0x0a can either mean \n or the number 10 (ten) and dos2unix does not know how to guess – slebetman Jul 24 '19 at 20:46
0
try (BufferedReader reader = new BufferedReader(new FileReader("inputFile"));
            FileWriter writer1 = new FileWriter("outPutFile")) {
        String line = reader.readLine();
        String cleanText = "";
        while (line != null ) {
            if(!line.equalsIgnoreCase("[BEGIN TYPESCRIPT]")) {
                line = line.replaceAll("\u001B\\[[\\d;]*[^\\d;]", "");
                cleanText = cleanText + line.replaceAll("\\p{Cntrl}", "") + System.lineSeparator();
            }
            line = reader.readLine();
        }
        writer1.write(cleanText);
    } catch (IOException e) {
        e.printStackTrace();
    }
}

Using this we can remove the control charcaters and VT100 escape characters saved in the session recording typescript file. I have added a condition to skip the typeScript header "[BEGIN TYPESCRIPT]". Am replacing each special character by "".

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.