19

I'm working with large number of files, which I keep in a directory. Every time I go in to that directory and accidentally press Tab twice, it takes too long (may be over a minutes) to show files that match the pattern, and I'm quite annoyed with this behavior.

For example, my directory structure is:

my-project/
├── docs/
├── data/        <---- contains around 200k files.
└── analyser/

Since I still love completion, is there anyway to disable this feature only at the data/ directory? Like set timeout for 5 sec, or a script that automatically switch off completion when inside specific directory?

  • Would you switch to zsh over this? (I don't know if it's possible in bash. I know it's possible in zsh, but it might not be easy, I haven't checked.) – Gilles Oct 13 '14 at 21:31
9

This isn't perfect, but then again bash completion is quite a tricky thing...

The very simplest way is on a per-command basis, it's slightly more flexible than FIGNORE, you can do:

 complete -f -X "/myproject/data/*" vi

This instructs autocomplete that completion for vi is for files, and to remove patterns matching the -X filter. The downside is that the pattern isn't normalised, so ../data and variations won't match.

The next best thing might be a custom PROMPT_COMMAND function:

# associative arrays of non-autocomplete directories
declare -A noacdirs=([/myproject/data]=1 )

function _myprompt {
    [[ -n "${noacdirs[$PWD]}" ]] && {
       echo autocomplete off
       bind 'set disable-completion on'
    } || {
       echo autocomplete on
       bind 'set disable-completion off'
    }
} 

PROMPT_COMMAND=_myprompt

This disables completion (completely) when you are in the directory, but it disables it for every path not just files in that directory.

It would be more generally useful to selectively disable this for defined paths, but I believe the only way is to use a default completion function (bash-4.1 and later with complete -D) and a lot of messing about.

This should work for you, but it may have unintended side effects (i.e. changes to the expected completion in some cases):

declare -A noacdirs=([/myproject/data]=1 )

_xcomplete() {
    local cur=${COMP_WORDS[COMP_CWORD]} # the current token

    name=$(readlink -f "${cur:-./}")  # poor man's path canonify
    dirname=$(dirname "$name/.")

    [[ -n "${noacdirs[$dirname]}" ]] && {
        COMPREPLY=( "" )   # dummy to prevent completion
        return
    }
    # let default kick in
    COMPREPLY=()
}

complete -o bashdefault -o default -F _xcomplete vi

This works for completion of vi, other commands can be added as needed. It should stop completion for files in the named directories regardless of path or working directory.

I believe the general approach with complete -D is to dynamically add completion functions for each command as it is encountered. One might also need to add complete -E (completion of command name when input buffer is empty).


Update Here's a hybrid version of the PROMPT_COMMAND and completion function solutions, it's a little easier to understand and hack I think:

declare -A noacdirs=([/myproject/data]=1 [/project2/bigdata]=1)

_xcomplete() {
    local cmd=${COMP_WORDS[0]}
    local cur=${COMP_WORDS[COMP_CWORD]} # the current token

    [[ -z "$cur" && -n "$nocomplete" ]] && {
        printf "\n(restricted completion for $cmd in $nocomplete)\n"  
        printf "$PS2 $COMP_LINE"
        COMPREPLY=( "" )   # dummy to prevent completion
        return
    }
    COMPREPLY=()       # let default kick in
}

function _myprompt {
    nocomplete=
    # uncomment next line for hard-coded list of directories
    [[ -n "${noacdirs[$PWD]}" ]] && nocomplete=$PWD
    # uncomment next line for per-directory ".noautocomplete"
    # [[ -f ./.noautocomplete ]] && nocomplete=$PWD
    # uncomment next line for size-based guessing of large directories
    # [[ $(stat -c %s .) -gt 512*1024 ]] && nocomplete=$PWD
} 

PROMPT_COMMAND=_myprompt
complete -o bashdefault -o default -F _xcomplete vi cp scp diff

This prompt function sets the nocomplete variable when you enter one of the configured directories. The modified completion behaviour only kicks in when that variable is non-blank and only when you try completing from an empty string, thus allowing completion of partial names (remove the -z "$cur" condition to prevent completion altogether). Comment out the two printf lines for silent operation.

Other options include a per-directory .noautocomplete flag file that you can touch in a directory as needed; and guessing of directory size using GNU stat. You can use any or all of those three options.

(The stat method is only a guess, the reported directory size grows with its contents, it's a "high water mark" that won't usually shrink when files are deleted without some administrative intervention. It's cheaper than determining the real contents of a potentially large directory. Precise behaviour and increment per file depends on underlying filesystem. I find it a reliable indicator on linux ext2/3/4 systems at least.)

bash adds an extra space even when an empty completion is returned (this only occurs when completing at the end of a line). You can add -o nospace to the complete command to prevent this.

One remaining niggle is that if you back up the cursor to the start of a token and hit tab, the default completion will kick in again. Consider it a feature ;-)

(Or you could futz around with ${COMP_LINE:$COMP_POINT-1:1} if you like over-engineering, but I find bash itself fails to set the completion variables reliably when you back up and attempt completion in the middle of a command.)

6

If your data directory contains files with a specific suffix, e.g. .out, then you can set your bash variable FIGNORE to ".out" and these will be ignored. Although it is possible to use directory names too, this does not help on current working directory names.

Example:

Create thousands of 100KB test files on physical host with raid 1 HDDs:

for i in {1..200000}; do dd if=/dev/urandom bs=102400 count=1 of=file$i.json; done

Set the bash variable:
$ FIGNORE=".json"

Create the test file:
$ touch test.out

Test at 5,000 files:

$ ls *.json|wc -l
5587
$ vi test.out

No delay between the vi and single tab before test.out appears.

Test at 50,000 files:

$ ls *.json|wc -l
51854
$ vi test.out

Single tab creates a fraction of a second delay before test.out appears.

Test at 200,000 files:

$ ls *.json|wc -l
bash: /bin/ls: Argument list too long
$ ls | awk 'BEGIN {count=0} /.*.json/ {count++} END {print count}'
200000
$ vi test.out

Delay of 1 second between the vi and single tab before test.out appears.

References:

Excert from man bash

FIGNORE
              A  colon-separated  list  of  suffixes to ignore when performing filename completion (see READLINE below).  A filename whose suffix matches one of the entries in FIGNORE is
              excluded from the list of matched filenames.  A sample value is ".o:~".
  • 1
    won't help, try create dir w/ about ~100k files. then I do $ vi <tab><tab>, it's take lots of time anyway. :\ – neizod Oct 13 '14 at 15:03
  • still not solve the issue, suppose I have 100k .json files, and a single text file named foo.txt. turning FIGNORE='.json' on, I type $ vi <tab> and still need to wait half a minute so it complete me to $ vi foo.txt. --- real scenario I'm not gonna edit, or mix file types inside that dir, but if I turn on FIGNORE and (accidentally) press tab, my keyboard will be freeze for a long time without hint of something like Display all 188275 possibilities? (y or n), which tell me that I can have my keyboard back. – neizod Oct 14 '14 at 4:20
  • @neizod - Sorry, I tried many scenarios to get FIGNORE to work on a directory level, to no avail. I guess a custom solution is required, or you simply disable bash completion on that host, or try the zsh solution detailed on our sister site Gilles mentioned. – geedoubleya Oct 14 '14 at 5:44
  • Sorry, it's only 1 second on your system, it's around half a minute on my system. So I guess I'd go buy a new PC to make this solution work. – neizod Oct 17 '14 at 3:30
0

Guess this is what you want (borrowed some code from @mr.spuratic)

Note this doesn't prevent you from pressing TAB using full path (e.g. vim /directory/contains/lots/files<tab><tab>

function cd() {
  builtin cd "$@"
  local NOCOMPLETION=( "/" "/lib" )
  local IS_NOCOMPLETION=0
  for dir in "${NOCOMPLETION[@]}"
  do
    echo $dir
    if [[ $(pwd) == "$dir" ]]
    then
      echo "disable completion"
      bind "set disable-completion on"
      IS_NOCOMPLETION=1
      return
    fi                                                                                                                                                                                              
  done
  if [[ $IS_NOCOMPLETION -eq 0 ]]
  then
    echo "enable completion"
    bind "set disable-completion off"
  fi
}  
  • Don't forget about pushd/popd. Since this only uses a regular array, not an associative array it will work in bash 3.x too. – mr.spuratic Oct 16 '14 at 13:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.