2

I've configured Bash to save multiline history entries with embedded newlines.

shopt -s cmdhist lithist
export HISTTIMEFORMAT='%F %T '

When I exit the shell, my history is appended to ~/.bash_history with timestamps separating the entries, like:

#1501293767
foo() {
echo foo
}
#1501293785
ls

I experimented with setting HISTIGNORE to filter out some commands,

HISTIGNORE='ls[ ]*:man[ ]*:cat[ ]*'

but quickly realized I prefer to include everything in the current session's (in memory) history, I just want to filter what gets written to disk for future sessions.

I'd like to leverage ~/.bash_logout to do this filtering (I've been looking at these answers to a couple related questions).

The multiline entries make filtering a little complicated. I need to identify entries where the first line following the timestamp matches one of the ignorable patterns, and exclude the entire entry, including the timestamp.

I've come up with a way to do this with awk, but my awk skills are not great, so I wonder if there's a better way?

filter_history() {
  tmpfile=$(mktemp)
  history -w $tmpfile

  awk '/^#[[:digit:]]{10}$/  { timestamp=$0; ignore=0; next }
       length(timestamp) > 0 && /^(ls|man|cat)([^[:alnum:]]|$)/ { timestamp=""; ignore=1 ; next }
       length(timestamp) > 0 { print timestamp; timestamp=""; print; next }
       ignore == 0           { print }' \
      $tmpfile >> $HISTFILE && rm $tmpfile
}

Edit: Actually, I can't think of a case where I'd want to filter out a multi-line entry. Even if it starts with ls, for instance, if it span multiple lines it's probably doing something interesting, and worth remembering.

  • .bash_logout is only executed in a login shell, it isn't useful here. Run this code from an exit trap instead (trap filter_history EXIT). – Gilles Jul 29 '17 at 12:52
  • Oh good point. I overlooked that because I'm on macOS, where the convention seems to be to use login shells for all interactive sessions. – ivan Jul 29 '17 at 13:22
  • @Gilles It seems that trapping EXIT causes my filter to run too late, i.e. after the history has already been written to disk. I could adjust my script to filter the entire contents of HISTFILE after the fact, but it seems smarter to filter incrementally, i.e. just the contents I want appended to the existing file. Maybe I should trap SIGHUP and unset HISTFILE after manually appending to it? – ivan Jul 29 '17 at 16:16
  • Ah, sorry, I didn't know that an EXIT trap would run after writing the history. If you're doing complex stuff, you may want to switch to zsh, which is generally more flexible. – Gilles Jul 31 '17 at 0:44
1

The requirement to keep multi-line entries even if the first line matches an "ignore" pattern added some complexity. I ended up writing a finite-state machine in Awk to filter HISTFILE after it was written to disk (I couldn't find a way to trigger filtering before it had already been written).

~/.bashrc:

# if shell is interactive, filter history upon exit
if [[ $- == *i* ]]; then
  trap '$HOME/.bash_history_filter >/dev/null 2>&1 &' EXIT
fi

~/.bash_history_filter:

tmpfile=$(mktemp)
trap 'rm -f "$tmpfile"' EXIT

filter_script="$HOME/.bash_history_filter.awk"
persisted_history="${HISTFILE:-$HOME/.bash_history}"

if [[ -r "$filter_script" && -r "$persisted_history" ]]; then
  awk -f "$filter_script" "$persisted_history" > "$tmpfile"

  mv "$tmpfile" "$persisted_history"
fi

Finite-state machine

~/.bash_history_filter.awk:

/^#[[:digit:]]{10}$/ {
  timestamp = $0
  histentry = ""
  next
}
$1 ~ /^(ls?|man|cat)$/ {
  if (! timestamp) {
    print
  } else {
    histentry = $0
  }
  next
}
timestamp {
  print timestamp
  timestamp = ""
}
histentry {
  print histentry
  histentry = ""
}
{ print }

A couple related posts (here and here) lead me to suspect this could also be done using sed. I haven't figured that out yet, but I'd be curious to see how it compares.

0

for the completness , here is how i deal with bash history! (filter & backup)

however you filter or not the HISTFILE isn't enouth for someone who cares about his history, also having a big HISTFILE isn't recommanded , so sure someday you will need to keep your history somewhere else!

function shellHist () #find old hist
{ 
    if [[ -f ${HISTFILE}.${1} ]]; then
        cat ${HISTFILE}.${1};
    else
        grep -h "${@:-.}" ${HISTFILE}.*;
    fi
}
function shellHistBackup () 
{ 
    [[ -n ${HISTFILE} ]] || return;
    # pidKill0 && return; # do nothing if the job is actually running elsewhere
    {
    while read histLine; do
        if ! grep -q "^${histLine}$" ${HISTFILE}.${histLine%% *} 2> /dev/null; then
            echo "${histLine}" >> ${HISTFILE}.${histLine%% *};
        fi;
    done < ${HISTFILE}
    for fltr in ls cat any ; do
        rm ${HISTFILE}.${fltr}
    done
    } & # pidLog # related to pidKill0
}

i include shellHistBackup in "trap shellExit EXIT"

i hope this will be useful for somebody!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.