How to decompress jsonlz4 files (Firefox bookmark backups) using the command line?

Question

There seems to be various JavaScript+browser specific ways of decompressing this, but isn't there some way to transform jsonlz4 files to something unlz4 will read?

Answer 1

I was able to unpack the jsonlz4 by using lz4json:

apt-get install liblz4-dev
git clone https://github.com/andikleen/lz4json.git
cd lz4json
make
./lz4jsoncat ~/.mozilla/firefox/*/bookmarkbackups/*.jsonlz4

Answer 2

Save this script in a file, e.g., mozlz4:

#!/usr/bin/env python
from sys import stdin, stdout, argv, stderr
import os
try:
    import lz4.block as lz4
except ImportError:
    import lz4

stdin = os.fdopen(stdin.fileno(), 'rb')
stdout = os.fdopen(stdout.fileno(), 'wb')

if argv[1:] == ['-c']:
    stdout.write(b'mozLz40\0' + lz4.compress(stdin.read()))
elif argv[1:] == ['-d']:
    assert stdin.read(8) == b'mozLz40\0'
    stdout.write(lz4.decompress(stdin.read()))
else:
    stderr.write('Usage: %s -c|-d < infile > outfile\n' % argv[0])
    stderr.write('Compress or decompress Mozilla-flavor LZ4 files.\n\n')
    stderr.write('Examples:\n')
    stderr.write('\t%s -d < infile.json.mozlz4 > outfile.json\n' % argv[0])
    stderr.write('\t%s -c < infile.json > outfile.json.mozlz4\n' % argv[0])
    exit(1)

Answer 3

Sufficiently persistent Googling for this turns up a lot of solutions, but most of them seem to be either (a) broken by subsequent changes to underlying libraries, or (b) unnecessarily complex (at least to my personal taste), making them clunky to drop into existing code.

The following appears to work at least on Python 2.7 and 3.6 using a recent version of the Python LZ4 bindings:

def mozlz4_to_text(filepath):
    # Given the path to a "mozlz4", "jsonlz4", "baklz4" etc. file, 
    # return the uncompressed text.
    import lz4.block
    bytestream = open(filepath, "rb")
    bytestream.read(8)  # skip past the b"mozLz40\0" header
    valid_bytes = bytestream.read()
    text = lz4.block.decompress(valid_bytes)
    return text

Of course this does not attempt to validate inputs (or outputs), is not intended to be secure, etc., but if one just wants to be able to parse one's own FF data, it gets the basic job done.

Command line version here, which could be saved in the relevant directory and invoked from the command line as:

chmod +x mozlz4.py
./mozlz4.py <file you want to read> <file to save output to>

Answer 4

Actually almost all Firefox profile lz4 files are mozlz4 files. It means they have the same "file format header". Except one file. I talk about webext.sc.lz4 file. It has mozJSSCLz40v001\0 file header and maybe some sc packaging to pack group of files to on byte stream.

There is a Firefox addon to read or compress .mozlz4 text files mozlz4-edit

Answer 5

The accepted solution doesn't work on any non-Linux systems; I forked to https://github.com/cnst/lz4json to make it compile cleanly on all other UNIX® systems, including the BSDs; here's a sample for Mac OS X:

Mac OS X UNIX® w/ MacPorts

sudo port install lz4
git clone https://github.com/cnst/lz4json.git
cd lz4json
make
./lz4jsoncat ~/Library/Application\ Support/Firefox/Profiles/CHANGE\
THIS.default/sessionstore-backups/recovery.jsonlz4 \
| python -m json.tool | fgrep :textarea | more

---

I also fixed compilation issues on FreeBSD and NetBSD w/ pkgsrc at https://github.com/cnst/lz4json; adding support for other systems like Fink or Homebrew would be trivial in my version, too; I welcome pull requests for any other UNIX systems (the author of the original tool is only interested in GNU/Linux support, refusing a pull request to add non-Linux support).

---

Also, here's the solution that's using Firefox directly: https://superuser.com/a/1363748/180573.

Answer 6

Python 3 solution.
Needs lz4: pip install lz4

Save the code as .py file. Then you can drag & drop files on it or use the command line to access it.

It only accepts one file path. If that file has a Mozilla LZ4 extension (.lz4, .mozlz4, .jsonlz4, .baklz4) it will decode, otherwise it will try to encode the file (always to the same folder the original file was in). You will get an overwrite warning/question in case the output file already exists.

from sys import argv, exit
import os

try:
    import lz4.block as lz4
except ImportError:
    print("Please install lz4 via   'pip install lz4'.")


EXTENSIONS = {"1": ".lz4", "2": ".mozlz4", "3": ".jsonlz4", "4": ".baklz4"}



def check_overwrite(path):
    if os.path.exists(path):
        i = input(f"File {path} already exists. Overwrite? y/n\n")
        if i.lower() == "y":
            print("Overwriting file.")
            return True
        else:
            print("Exiting.")
            exit(1)
    else:
        return True


def print_info():
    print(f"Usage: {argv[0]} <infile>")
    print("Decompress Mozilla-LZ4 encoded <infile> or compress <infile> to Mozilla-LZ4.")
    print("Output file will be put in same folder as input file.")
    exit(1)


def choose_ext():
    print(f"Please choose file extension:")
    for k, v in EXTENSIONS.items():
        print(f"{k}: {v}")
    if (i := input()) in EXTENSIONS.keys():
        return EXTENSIONS[i]
    else:
        print("Invalid extension.\nExiting")
        exit(1)


def main():
    if len(argv) != 2:
        print_info()

    else:
        if not os.path.exists(argv[1]):
            print_info()
        else:
            in_full_name = os.path.basename(argv[1])
            in_folder_path = os.path.dirname(argv[1])
            in_name, in_ext = os.path.splitext(argv[1])  # "search.json", ".mozlz4"
            in_file_handle = open(argv[1], "rb")

            if in_ext.lower() in EXTENSIONS.values():
                ''' Decompress '''
                print(f"Trying to decompress {in_full_name}")
                out_file = os.path.join(in_folder_path, f"{in_name}")
                if check_overwrite(out_file):
                    with open(out_file, "wb") as f:
                        assert in_file_handle.read(8) == b"mozLz40\0"
                        f.write(lz4.decompress(in_file_handle.read()))

            else:
                ''' Compress '''
                print(f"Trying to compress {in_full_name}")
                ext = choose_ext()
                out_file = os.path.join(in_folder_path, f"{in_full_name}{ext}")
                if check_overwrite(out_file):
                    with open(out_file, "wb") as f:
                        f.write(b"mozLz40\0" + lz4.compress(in_file_handle.read()))


if __name__ == '__main__':
    main()