What is :>filename.txt Doing?

Question

:>filename.txt

For example:

root@box$ dd if=/dev/zero of=file.txt count=1024 bs=1024
1024+0 records in
1024+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00536175 s, 196 MB/s
root@box$ ll
total 1024
-rw-r--r-- 1 root root 1048576 Nov 15 14:40 file.txt
root@box$ :>file.txt
root@box$ ll
total 0
-rw-r--r-- 1 root root 0 Nov 15 14:40 file.txt

Is this different from an rm? Does it operate faster or slower than other similar means of zeroing a file or deleting it?

Answer 1

As you have discovered, this just empties the file contents (it truncates the file); that is different from rm as rm would actually remove the file altogether. Additionally, :>file.txt will actually create the file if it didn't already exist.

: is a "do nothing command" that will exit with success and produce no output, so it's simply a short method to empty a file. In most shells, you could simply do >file.txt to get the same result. It also could be marginally faster than other methods such as echo >file.txt as echo could potentially be an external command.

Additionally, echo >file.txt would put a blank line in file.txt where :>file.txt would make the file have no contents whatsoever.

Answer 2

Yes, it's different from rm.

rm would remove the file.

:>filename.txt empties the file, leaving it still there, but zero bytes in size.

Answer 3

The shell invocation of >filename.txt redirects some output to the file "filename.txt" to completelly replace it. So, the shell has to clear all the contents of the given file before writting the output to it.

The output to be redirected is the output of the command executed. Like:

$ echo Hello >filename.txt

Will make the file of name filename.txt to contain exactly (and only) the string Hello.

Executing:

$ echo "New Value" >filename.txt
$ cat filename.txt
New Value

will erase everything inside the file and then write New Value to it.

If the command has no output, like the command true, the file will remain empty (truncated).

$ true >filename.txt
$ cat filename.txt

A command that is also a shell builtin is : (just a double dot (colon)) and that has no output. Being a builtin makes it faster than an external command such as true (which also has no output). So, either:

$ :  >  filename.txt
$ :  >filename.txt
$ :>filename.txt

will remove all content from the file named filename.txt or will create it as an empty file if it doesn't exist.

Is this different from an rm?

That is different than a rm, as a rm will make the file disappear from ls, not make the file to contain 0 bytes.

Does it operate faster or slower than other similar means of zeroing a file or deleting it?

Again, the file is not being deleted (disappear from the list given by ls) it transforms in an empty file (contains 0 bytes).

It should be faster than calling an external command to empty a file. Having to create a child shell to load the executable and exec the command makes external commands slower than the builtin :.

Similar solutions are (some set $? to 1):

[ ]         > filename.txt
builtin     > filename.txt
command     > filename.txt
printf ''   > filename.txt

Answer 4

re: performance: this is about as efficient as possible in the shell; just asking the kernel to truncate an existing file should be more efficient than unlinking the file and re-creating a new inode with the same name. Unless you want the file deleted, in which case rm or unlink it.

: is a shell built-in so it avoids fork/exec. So is the equivalent true in normal modern shells.

>foo or true > foo gets the shell to truncate the file by making an
open(path, O_WRONLY|O_TRUNC|O_CREAT, 0666) system call.

Or in practice with DASH on Linux, from the strace sh output:
openat(AT_FDCWD, "foo", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3 which is equivalent.

Then it has to close() that FD again. In fact, DASH doesn't special case this when you use :>, it jumps through the following hoops:

openat(AT_FDCWD, "foo", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
fcntl(1, F_GETFD)                       = 0
fcntl(1, F_DUPFD, 10)                   = 10          # save original stdout
fcntl(1, F_GETFD)                       = 0
fcntl(10, F_SETFD, FD_CLOEXEC)          = 0
dup2(3, 1)                              = 1           # redirect stdout to foo
close(3)                                = 0           # then close 3
dup2(10, 1)                             = 1           # then restore original stdout
fcntl(10, F_GETFD)                      = 0x1 (flags FD_CLOEXEC)
close(10)                               = 0

Using > foo in DASH leads to the same sequence of system calls, actually redirecting fd1 and then restoring it. I didn't check bash or other shells.

But that's still significantly cheaper than creating a new process to run truncate -s 0 foo which would (hopefully) make a single truncate("foo", 0) system call which would presumably be even more efficient than an open + close.

From a language like C (or anything with system call bindings), truncating a file you don't want open can be done most efficiently with a direct truncate syscall.

In Dash, 3>foo leads to this sequence of system calls:

openat(AT_FDCWD, "foo", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
close(3)                                = 0

Opening a new fd results in it already being fd 3, avoiding any duping. This is the most efficient way in dash, probably saving several microseconds vs. >foo. If that matters, shell scripting is the wrong language for your task!! But you did ask.

---

With Spectre+Meltdown mitigation enabled, even a -ENOSYS bad system call takes at least thousands of clock cycles, so microseconds, on modern Intel x86-64. Up from a couple hundred for the user->kernel->user round trip with just a syscall instruction. Of course path lookups and so on take significant time and so does filesystem code. And going to kernel mode and back often evicts some cache making user-space run slower upon return.

Actual I/O cost once the metadata does write-back to disk depends on the filesystem. An FS like XFS or modern ext4 using only 1 or a few large extents for the whole file can easily free huge amounts of space in O(1) time. Or O(n) where n is the number of extents (fragmentation) not the size in bytes.

Depending on the FS, if the extent info was stored right in the inode, instead of an indirect block, that's one less thing to add to the free list.

The I/O cost is similar to unlinking the file, but you don't have to free the inode or modify the directory entry. You do still have to update the mtime and ctime in the inode upon truncation, but you had to write it anyway if the size changed.