Tag Info

Hot answers tagged

33

The [ binary residing under the /bin tree in many GNU/Linux distributions is not something to be alarmed off. At least in my Fedora 19 it is a part of the coreutils package, as demonstrated below: $ rpm -qf /bin/[ coreutils-8.21-13.fc19.x86_64 and is a synonym for test to allow for expressions like [ expression ] to be written in shell scripts or even ...


26

It's usually plain C. The commands ls and pwd come from the GNU Coreutils package in (most?) Linux distributions (and maybe some other systems). You can find the code on their homepage. For coreutils specifically, you build them with the usual steps: after unpacking the source, issue: ./configure --prefix=/some/path # type ./configure ...


21

Here's another way to do locking in shell script that can prevent the race condition you describe above, where two jobs may both pass line 3. The noclobber option will work in ksh and bash. Don't use set noclobber because you shouldn't be scripting in csh/tcsh. ;) lockfile=/var/tmp/mylock if ( set -o noclobber; echo "$$" > "$lockfile") 2> ...


15

Izkata's comment revealed the answer: locale-specific comparisons. The sort command uses the locale indicated by the environment, whereas Python defaults to a byte order comparison. Comparing UTF-8 strings is harder than comparing byte strings. $ time (LC_ALL=C sort <numbers.txt >s2.txt) real 0m5.485s user 0m14.028s sys 0m0.404s How about ...


12

Sorting depends on the locale; specifically, it depends on $LC_COLLATE (possibly overridden by $LC_ALL), falling back to $LANG if it doesn't exist. The command locale will show you what values you're effectively working with. See man 3 strcoll, man 3 setlocale, etc. LC_COLLATE=C (or POSIX or no locale at all) results in a strict byte-by-byte comparison. ...


11

Don't know why it's not the default, maybe so that it behaves the same as other copying utilities (rsync, cpio, pax, tar...) which have no support for it (or when files are copied across an interface that doesn't allow that (like NFS, samba, fuse file systems layers...). I was in the same situation a few years ago, and looking at GNU cp code quickly, it's ...


10

You'll want to modify your assignment to read: var4="$(echo ztemp.xml | cut -f1 -d '.')" The $(…) construct is known as command susbtitution.


10

I think it's taking your + x as a time-zone specifier (e.g., consider 2013-04-25 19:52:36 +4 is a valid timestamp, in in a time zone 4 hours ahead of UTC). It's then seeing the word 'minutes', and treating it as a synonym of minute, so giving you one minute later. If you put in an explicit timezone specifier, it works: anthony@Zia:~$ TZ=utc date -d ...


9

stat: print timestamps to full resolution was committed to coreutils-8.6. $ git clone git://git.savannah.gnu.org/coreutils.git $ cd coreutils $ git log -1 --grep=time -- src/stat.c commit 9069af45e691d1252c727da66aa4f3f3c7f1ea29 Author: Eric Blake Date: Thu Sep 30 16:42:13 2010 -0600 stat: print timestamps to full resolution * src/stat.c ...


9

GNU coreutils do understand UTF-8 in general. For example echo 哈哈 | wc -m correctly outputs 3 in a UTF-8 locale (note that the option is -m, not -c which for historical reasons means bytes). This is a bug in cut. Looking at the source of cut, cut on characters is simply not implemented: the -c option is treated as a synonym of -b. A workaround is to use ...


8

tac is easier to understand in the case it's primarily designed for, which is when the separator is a record terminator, i.e. the separator appears after the last record. It prints the records (including each terminator) in reverse order. $ echo -n fooabara | tac -s a; echo rabafooa The input consists of three records (foo, b and r), each followed by the ...


7

I prefer to use hard links. lockfile=/var/lock/mylock tmpfile=${lockfile}.$$ echo $$ > $tmpfile if ln $tmpfile $lockfile 2>&-; then echo locked else echo locked by $(<$lockfile) rm $tmpfile exit fi trap "rm ${tmpfile} ${lockfile}" 0 1 2 3 15 # do what you need to Hard links are atomic over NFS and for the most part, mkdir is ...


7

Generally, there's a changelog. In fact, this (or other "prominent notices" of changes) is required by the GPL! (At least, effectively so for anything with multiple contributors — see GPLv2 section 2a.) For the GNU coreutils package — and for pretty much everything else from the GNU project directly — this file is definitely the first place to look, and ...


7

Depending on the shell you're using, you can use Parameter Expansion. For instance in bash: ${parameter%word} ${parameter%%word} Remove matching suffix pattern. The word is expanded to produce a pattern just as in pathname expansion. If the pattern matches a trailing portion of the expanded value of parameter, then the ...


7

POSIXly: $ awk 'BEGIN{ n = split(ENVIRON["PATH"], p, ":") while (n) i[p[n]]=n-- if (! (ARGV[1] in i)) print ARGV[1], "is not in $PATH" else if (! (ARGV[2] in i)) print ARGV[2], "is not in $PATH" else if (i[ARGV[1]] < i[ARGV[2]]) print ARGV[1], "is before", ARGV[2] else print ARGV[1], "is after", ARGV[2] exit}' /bin /usr/bin ...


7

As @samiam has stated the list is returned to you in a semi-random order via readdir(). I'll just add the following. The list returned is what I would call the directory order. On older filesystems, the order is often the creation order that the file entries in the directory's table were added. There is of course a caveat to this, when a directory entry is ...


7

This is more of an extra analysis than an actual answer but it does seem to vary depending on the data being sorted. First, a base reading: $ printf "%s\n" {1..1000000} > numbers.txt $ time python sort.py <numbers.txt >s1.txt real 0m0.521s user 0m0.216s sys 0m0.100s $ time sort <numbers.txt >s2.txt real 0m3.708s user ...


6

The point of -f is to try and avoid the need to stat every file entry, and to avoid the need to read them all before any are displayed. It is a "meta" option that just disables other options. So, yes, it should do what you expect. I can't answer why it isn't, but I would guess that you might have a shell alias or something else that inserts additional ...


6

Ksh, Zsh and Bash all offer another, perhaps clearer syntax: var4=$(echo ztemp.xml | cut -f1 -d '.') The backticks (a.k.a. "grave accent") is unreadable in some fonts. The $(blahblah) syntax is a lot more obvious at least. Note that you can pipe values into a read command in some shells: ls -1 \*.\* | cut -f1 -d'.' | while read VAR4; do echo $VAR4; done ...


6

I understand that mkdir is atomic, so perhaps: lockdir=/var/tmp/myapp if mkdir $lockdir; then # this is a new instance, store the pid echo $$ > $lockdir/PID else echo Job is already running, pid $(<$lockdir/PID) >&2 exit 6 fi # then set traps to cleanup upon script termination # ref http://www.shelldorado.com/goodcoding/tempfiles.html ...


6

busybox the favorite of Embedded Linux systems. BusyBox combines tiny versions of many common UNIX utilities into a single small executable. It provides replacements for most of the utilities you usually find in GNU fileutils, shellutils, etc. The utilities in BusyBox generally have fewer options than their full-featured GNU cousins; however, the options ...


6

Wikipedia, "Factor (Unix)" with an interesting take: factor first appeared on 5th edition Research Unix in 1974, as a "user maintained" utility (section 6 of the manual). In the 7th edition in 1979, it was moved into the main "commands" section of the manual (section 1). From there, the factor utility was copied to all other variants of Unix, including ...


6

cp doesn't have this option. You could write a wrapper script, but it's pretty simple. ln -f $^ $@ 2>/dev/null || cp -f $^ $@ GNU Coreutils 7.5 introduced the --reflink option. If you pass --reflink=auto and the underlying filesystem supports copy-on-write (e.g. Btrfs or ZFS) and the copy happens to be on the same device, then cp will create a new ...


6

By default, du will only count each file once if it is linked to multiple times. If you run du -L bar it will count the file because it only reaches it once. However, if you run du -L * it will only count it the first time it sees it. For example: $ du -L foo bar 16K foo 4.0K bar $ du -L bar foo 16K bar 4.0K foo Notice that swapping the ...


6

You would use --no-target-directory if you don't want a source directory copied underneath an existing destination directory, you want the source directory copied onto the destination directory. Here is an example of a directory copy with and without --no-target-directory: $ mkdir a $ touch a/b a/c $ find . ./a ./a/c ./a/b $ cp -r a b # b does not ...


6

From the man page for du: -b, --bytes equivalent to '--apparent-size --block-size=1' -h, --human-readable print sizes in human readable format (e.g., 1K 234M 2G) The -h switch will display the output using units, blocks have no units, so these 2 switches aren't suppose to be used together! NOTE: The command du will therefore ...


6

readdir() bascially. When tar finds out what files are in a directory, it directly asks the kernel for a file listing via opendir() followed by readdir(). readdir() does not return the files in any particular order; the way the files are ordered depends on the file system being used by the Linux kernel. There, alas, isn't an option for tar to sort files ...


6

You may want to check this: ls -l --time=atime atime — updated when file is read mtime — updated when the file changes. ctime — updated when the file or owner or permissions changes. Have fun! :)


5

Just to add a bit more to Mat's answer, although Unix implements these as standalone programs, many commands are also implemented within bash (to save it having to spawn a new process) see the bash man page for more details, or have a look at the shell-builtin tag. bash is also written in C It may be worth noting that these commands do not have to be ...


5

Normally, they are the same binary with hard links. Depending on the shell, an internal implementation may be used for test and [ tests rather than the binary. This is more efficient in terms of process spawning, and may provide different options than the binary program test provides. Other than the the differences in call format, they both provide the ...



Only top voted, non community-wiki answers of a minimum length are eligible