123

In zsh, I want to have unlimited history. I set HISTSIZE=, which works in bash. Now I import an old history

mv old_history .history

which is pretty big

wc -l .history
43562 .history

If I now close and start zsh again, I see

wc -l .history
32234 .history

Can't I have unlimited history in zsh?

Improve this question
1

4 Answers 4

Reset to default
150

There is the limit and the possibilities of your machines.

HISTFILE="$HOME/.zsh_history"
HISTSIZE=10000000
SAVEHIST=10000000
setopt BANG_HIST                 # Treat the '!' character specially during expansion.
setopt EXTENDED_HISTORY          # Write the history file in the ":start:elapsed;command" format.
setopt INC_APPEND_HISTORY        # Write to the history file immediately, not when the shell exits.
setopt SHARE_HISTORY             # Share history between all sessions.
setopt HIST_EXPIRE_DUPS_FIRST    # Expire duplicate entries first when trimming history.
setopt HIST_IGNORE_DUPS          # Don't record an entry that was just recorded again.
setopt HIST_IGNORE_ALL_DUPS      # Delete old recorded entry if new entry is a duplicate.
setopt HIST_FIND_NO_DUPS         # Do not display a line previously found.
setopt HIST_IGNORE_SPACE         # Don't record an entry starting with a space.
setopt HIST_SAVE_NO_DUPS         # Don't write duplicate entries in the history file.
setopt HIST_REDUCE_BLANKS        # Remove superfluous blanks before recording entry.
setopt HIST_VERIFY               # Don't execute immediately upon history expansion.
setopt HIST_BEEP                 # Beep when accessing nonexistent history.

From the ZSH Mailing list:

You should determine how much memory you have, how much of it you can allow to be occupied by the history (AFAIK it is always fully loaded into memory) and act accordingly. Removing the limit is not wiser as it leaves you with an idea that there is no limit while it is always limited by available resources.

Or if you do not think you will ever hit a problem with resource exhaustion you can just set HISTSIZE to LONG_MAX from limits.h: it is the maximum number HISTSIZE can have.

Which explain the Gentoo solution:

export HISTSIZE=2000
export HISTFILE="$HOME/.history"

History won't be saved without the following command:

export SAVEHIST=$HISTSIZE

To prevent history from recording duplicated entries (such as ls -l entered many times during single shell session), you can set the hist_ignore_all_dups option:

setopt hist_ignore_all_dups

A useful trick to prevent particular entries from being recorded into a history by preceding them with at least one space.

setopt hist_ignore_space
Improve this answer
12
  • 2
    I don't see how that answers my question. The size of my history file is 683 kB, surely not too large for my memory?
    – pfnuesel
    Apr 2, 2016 at 15:09
  • 3
    user $export HISTSIZE=99999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 will be out of the possibilities of your computer. You have to set that number to fit as best as possible to your wish and computer possibilities.
    – aurelien
    Apr 2, 2016 at 15:11
  • 4
    Your question is Can't I have unlimited history in zsh? The ZSH author respond No but you can set it as you wish. Gentoo explain just correctly how do it.
    – aurelien
    Apr 2, 2016 at 15:17
  • 11
    none of these solutions work in zsh. It still cuts it off after about a week or two. I've been dealing with this problem for years and it's so absurd. I finally used logrotate to do it.
    – kristopolous
    Oct 3, 2018 at 21:50
  • 2
    not sure if this helps as its not really what you ask, but in zsh you need to add a "1" after the history command to show the full history, otherwise you get the last 15 entries.
    – m4rinos
    Mar 4, 2020 at 9:27
55

You need to set both HISTSIZE and SAVEHIST. They indicate how many lines of history to keep in memory and how many lines to keep in the history file, respectively.

I don't think zsh has a setting that means “infinite”, but for all practical purpose, one billion is infinite, while remaining machine-representable on 32-bit machines. Add to ~/.zshrc:

HISTFILE=~/.zsh_history
HISTSIZE=999999999
SAVEHIST=$HISTSIZE
Improve this answer
4
  • Should I have to worry about keeping such a large number of lines in memory slowing down my computer?
    – a06e
    Jan 11, 2018 at 17:58
  • 29
    You are not going to type enough history lines in your life time to slow down a 2010s PC.
    – Gilles 'SO- stop being evil'
    Jan 11, 2018 at 18:56
  • @Gilles'SO-stopbeingevil' so why is the default size so small?
    – talz
    Jan 5 at 10:19
  • 2
    @talz Because the default zsh configuration is over thirty years old (zsh seems to favor backward compatibility over friendliness to new users). Even today, there are (increasingly rare) cases where a large size can be a problem, for example if your home directory is on a network filesystem, or if you choose zsh on a small embedded device (as in, a wrt router, not a Raspberry Pi).
    – Gilles 'SO- stop being evil'
    Jan 5 at 16:30
13

I was having a problem with history being capped at 10000 until I realized oh-my-zsh was setting its own HISTSIZE and SAVEHIST values. TL;DR, make sure to put the lines at the bottom of your .zshrc.

Another problem I was having was my history getting truncated to 1024 lines. While I'm not sure why, I solved it doing the following:

  • Setting HISTFILE=$HOME/.zsh_hist (something different than it was before)
  • Setting HISTFILE=$HOME/.bash_history explicitly in ~/.bashrc
  • Doing the normal settings like HISTSIZE=100000 and SAVEHIST=100000

I think this could have been related to me using export to set HISTFILE which caused bash to use the same HISTFILE variable when launched interactively from ZSH which would truncate it. Setting bash history file probably isn't necessary as long as you make sure not to export HISTFILE.

Improve this answer
4
  • 3
    Thank you! I was having the same problem with zprezto. "Why is my history going away so quickly? I have HISTSIZE/SAVEHIST at 2 million, but wc -l .zhistory is only about 10,000." Oddly, in the runtime shell I see my .zshrc value, but zprezto's history module indeed initializes SAVEHIST to 10,000 and that must be somewhat sticky.
    – Conrad Meyer
    Feb 20, 2020 at 18:15
  • You should do this before sourcing oh-my-zsh. That way it sees what values you set and make adjustments accordingly
    – smac89
    Mar 3, 2021 at 9:18
  • If you're using oh-my-zsh and want to avoid this confusion, just keep your own config file in ~/.oh-my-sh/custom. Make sure the file extension is .zsh though. oh-my-zsh will leave the file(s) there alone, and load them at the end, so your own settings override any of theirs. Then you don't need to worry about oh-my-zsh removing any of your settings after an update.
    – nivek
    Sep 5, 2021 at 6:18
  • Darned thank you! This was driving me crazy. I moved my own exports immediately after sourcing oh-my-zsh and now I finally have bigger history.
    – dimitarvp
    Mar 18, 2022 at 23:40
8

The problem was not with zsh or my .zshrc, but with my history. There were some oddball entries like

^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@#1453692179

that confused zsh and everything after these entries got truncated. I don't know what character ^@ corresponds to, or how these entries ended up in my history, but removing them fixed the problem.

Improve this answer
8
  • which terminal and editor do you use?
    – aurelien
    Apr 2, 2016 at 16:06
  • 2
    ^@ is the ASCII NUL (\0) character.
    – muru
    Apr 3, 2016 at 1:22
  • 10
    The ^@^@^@^@ characters are nulls, and they frequently show up when two programs try writing to the same file at the same time, and aren't using atomic writes (eg. they're buffered writes) or locking. Or it can be caused by filesystem corruption, bad restoration from backup, or similar.
    – user222658
    Mar 24, 2017 at 18:50
  • 2
    +1 user222658; These are NULs and can easily be caused by a non-atomic "append" filesystem operation and power loss, if the filesystem (or disk!) writes the updated inode size before writing the appended data. (Some disks perform their own write caching, and those caches reorder writes, which breaks some filesystems' internal consistency.)
    – Conrad Meyer
    Feb 20, 2020 at 18:32
  • 2
    This is an interesting clue that some kind of values in the command could cause the history to get truncated. I think I've had this happen a few times. It's pretty rare. I do track my history file regularly in git via a cronjob so I can get it back. Still a huge hassle though. Incidents like this are why I set that up to begin with. It's not a paranoid measure at all (although my efforts perhaps should have gone toward a full system backup setup instead). I wonder if we can get the bug fixed with zsh though.
    – Steven Lu
    Jul 16, 2021 at 23:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.