The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
2answers
70 views

parsing using loop and assigning row numbers to file

It is a bit complicated. i have a configuration file. it looks something like this. customer="airtel" model_name=fast programmer_typ_of="dev" Now the 1st column contains all the keys and whatever ...
5
votes
1answer
47 views

Why is fragmentation level so huge in files that contain other filesystems?

I've just found out what sparse files are and wanted to conduct some experiments on them. On wiki you can read that the files can get fragmented easily. I wanted to check how bad that is. I created a ...
2
votes
1answer
48 views

Understanding sparse files, dd, seek, inode block structure

At work we use sparse files as part of out Oracle VM environment for the guest disk images. After some questions from a colleague (which have since been answered) I am left with more questions about ...
1
vote
1answer
30 views

Create and use sparse file as non-privileged user

Here is my setup: I am non-privileged user (non-root). I cannot use sudo, nor I can use any other command to switch to root (e.g. su), or run commands as root. As a consequence, I cannot modify the /...
0
votes
2answers
76 views

How to get number of files in a 7z

I want to get the number of files (included directories) contained in the root of a 7z archive, but the 7z command on Linux don’t bring a simple way to only get the number and noting else. For ...
1
vote
1answer
63 views

rsync generating a destination directory of bigger size than source?

We are copying our data from source server to destination server using rsync : rsync -av --progress --sparse user@destination server After rsync, we observed that destination directory is bigger in ...
1
vote
0answers
20 views

xfs: writing to 17TB offset doesn't actually write it to specified offset

I have a situation wherein I want to write a few GBs at a very large offset - say 17TB. But when I use the dd command to do so, I notice that the filesystem is doing some kind of optimization - sparse ...
1
vote
0answers
115 views

Creating a sparse (automatically expanding and contracting) swap file with hole-punching support

I have this crazy idea, and want to know the pitfalls of such an approach (or whether it can even work with no deadlocks.. etc). The goal is to create an automatically expanding and contracting ...
0
votes
0answers
75 views

How to throttle system resources during transfer from btrfs on file image on cifs mount?

I have a btrfs on a sparse file image (4TB) on a cifs mount of a remote NAS (Synology DS214) with zlib compression enabled. Transferring file from and onto the btrfs causes freezes of Linux (where it ...
2
votes
1answer
227 views

How do I punch a hole (FALLOC_FL_PUNCH_HOLE) in a file from command line?

There is fallocate(2) system call to make sparse files by erasing a "holes" in them. Is there a command line tool for it? Expecting something like: makehole myfile.dat --offset=10G --length=1G
1
vote
1answer
64 views

file (vdi) increase (at double) size after split and then cat operation

I'd splitted a vdi file with 7 gigabytes and then joined with cat command. The file works normally on VBox, but get 14 gigabytes after be re-joined. The exact operation was: $ split -- bytes=2000M ...
3
votes
1answer
137 views

“Ghost” files in ext4? (ktorrent allocated disk space)

I'm downloading with KTorrent, and have noticed that when I select "allocate disk space before starting torrent", it creates all the files at start, presumably empty, to avoid fragmentation. When I'm ...
2
votes
1answer
214 views

What is the correct syntax for cloning a ntfs partition to a sparse file on another ntfs drive?

I have tried ddrescue -v -v -S /dev/sda2 /media/mint/otherhdd/img /media/mint/otherhdd/log and ddrescue -v -v -d -S /dev/sda2 /media/mint/otherhdd/img /media/mint/otherhdd/log but it does not work ...
2
votes
1answer
181 views

From a sparse file to a block device

I have a system image sparse file which actual size is only a few MBs but the "apparent size" is about 1GB. I'm trying to write it to a block device efficiently (without the holes). Here are some ...
7
votes
1answer
2k views

Converting sparse file to non-sparse in place

On Linux, given a sparse file, how to make it non-sparse, in place? It could be copied with cp --sparse=never ..., but if the file is say 10G and the hole is 2G (that is the allocated space is 8G), ...
4
votes
1answer
131 views

Linux: How to calculate/find the slackspace of a file?

How would I go about finding the slackspace of a file, for instance. fls -o 63 /images/diskimg.dd istat -o 63 /images/diskimg.dd 31 icat -o 63 /images/diskimg.dd | xxd | less In which of these ...
8
votes
1answer
149 views

How to display the non-sparse parts of a sparse file?

Imagine a file created with: truncate -s1T file echo test >> file truncate -s2T file I now have a 2 tebibyte file (that occupies 4kiB on disk), with "test\n" written in the middle. How would ...
9
votes
2answers
3k views

How can I speed up operations on sparse files with tar, gzip, rsync?

I have a sparse file. (du -h reports 3G and du -h --apparent-size reports 100G.) So far, so good. Now, when I want to compress the file using tar or send it over the network using rsync, it will ...
2
votes
1answer
103 views

Looking for a “undo” for file with holes (GNU)

Linux supports sparse files aka 'file with holes' Note following commands alias mystat='stat -c "%n: %B*%b blocks %s bytes"' dd if=/dev/zero bs=1024k seek=4096 count=0 of=file-with-holes ls -l ...
5
votes
2answers
814 views

zerofree verbose returns what?

zerofree -v /dev/sda1 returned 123642/1860888/3327744. The man page does not explain what those numbers are: http://manpages.ubuntu.com/manpages/natty/man8/zerofree.8.html I found some code on ...
1
vote
1answer
957 views

Ext4 exhibits unexpected write latency variance vs. ext2

I have a latency sensitive application running on an embedded system, and I'm seeing some discrepancy between writing to a ext4 partition and an ext2 partition on the same physical device. ...
1
vote
1answer
320 views

What parts of the filesystem to tmpfs/squashfs/sparseimage for USB live system

I want to create a live system on a USB key / SDCard with Linux Mint 14 (KDE or Debian Edition) that stores data persistently. The USB key needs to have at least one fat32 partition. As storage for ...
1
vote
1answer
393 views

Stripping null byte blocks with a sparse file?

I've created a sparse file with dd. How do I copy contents of another file there, leaving all the zero blocks unallocated?
17
votes
3answers
2k views

How can you trim files using the command line?

What is the console equivalent of the following Python code: target = file("disk", "w") # create a file target.seek(2*1024*1024*1024) # skip to 2 GB target.write("\0") target.close() Maybe some ...
2
votes
1answer
769 views

Can QEMU/KVM+OCFS2 be configured to punch holes in sparse image files when files are deleted in the guest?

I'm using sparse raw image files for guest VMs and OCFS2 on the host (local-only, not clustered) for the reflink feature. I understand from Googling around that OCFS2 is one of the filesystems that ...
19
votes
4answers
2k views

Can a file that was originally sparse and expanded be made re-sparse

I know that copying or transferring what was originally a sparse file without using a utility that understands sparse files will cause the 'holes' to be filled out. Is there a method or utility to ...
4
votes
3answers
1k views

from a sparse file to a block device over the network

If I have a sparse file representing a block device, then how can I efficiently restore the file over a network? Ideally, I'd use rsync -S host:file /dev/sdb, but rsync won't write to block devices. ...
13
votes
4answers
6k views

Creating an arbitrarily large “fake” file

I would like to create a special file similar to /dev/null or /dev/random, where the file doesn't actually exist but you can read from it all the same, except that I could actually set a cap on the ...
4
votes
1answer
2k views

Sparse files/file holes and unexpected block size

For my own learning, I've been playing around with creating files with file holes. I created a util that simply reads from stdin and writes to a file, but before writing to the file, it uses lseek to ...
3
votes
3answers
2k views

Is creating a sparse image using dd appropriate for backup/restore from a RAID?

I have two non-networked servers, each with 1.2TB of storage in a RAID5 config on an LSI MegaRaid 9240-8i controller. I have been trying without success to get Symantec Ghost to image one machine to ...
6
votes
1answer
1k views

Fragmentation and ext4

Suppose I have a sparse file F on a Linux ext4 filesystem, and process P1 is writing to a disjoint 50% subset of F while P2 writes to the other 50% of F. I would like to minimize fragmentation while ...
2
votes
1answer
2k views

What is the interaction of the rsync --size-only and --sparse options?

If I use rsync with the --size-only and --sparse options together, if a file already exists at the destination as a full file (otherwise the same) it's not overwritten. If I omit the --size-only ...
22
votes
5answers
12k views

Clone whole partition or hard drive to a sparse file

I like to clone a whole partition or a whole hard drive onto a larger external disk but like to create a sparse file. I often use dd for cloning, but it doesn't support sparse files. As a workaround I ...
10
votes
3answers
17k views

How to make ext4 filesystem sparse?

I am using Linux as guest OS in VirtualBox. I deleted huge number of files from its filesystem. Now i want to shrink the filesystem image file (vdi). The shrinking works by compressing filesystem ...
10
votes
2answers
2k views

Detailed sparse file information on Linux

I have a sparse file, in which only some blocks are allocated: ~% du -h --apparent-size example 100K example ~% du -h example 52K example I would like to know which blocks of the file are ...