Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

When I want to install a new package, I'm usually greeted with a response like:

oliver@cloud:~$ sudo apt-get install unison
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package unison

Already knowing this pattern, I continue:

oliver@cloud:~$ sudo apt-get update
Hit http://downloads-distro.mongodb.org dist Release.gpg
Ign http://downloads-distro.mongodb.org/repo/debian-sysvinit/ dist/10gen Translation-en
Ign http://downloads-distro.mongodb.org/repo/debian-sysvinit/ dist/10gen Translation-en_US
Hit http://downloads-distro.mongodb.org dist Release
Ign http://downloads-distro.mongodb.org dist/10gen amd64 Packages
Ign http://downloads-distro.mongodb.org dist/10gen amd64 Packages
Ign http://http.debian.net/debian/ squeeze/contrib Translation-en_US
Ign http://http.debian.net/debian/ squeeze/main Translation-en_US
Ign http://http.debian.net/debian/ squeeze/non-free Translation-en_US
...

oliver@cloud:~$ sudo apt-get install unison
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  unison
0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.
Need to get 693 kB of archives.

If I want to install a new package tomorrow, the server will not know about it until I update again. I've only noticed this on Debian servers and I'm not aware of any adjustments that would cause this.

I understand the need for the local cache, my problem is that it seems to be reset whenever I want to install something new. I would expect the cache to stay populated for longer periods of time. Usually, when I would want to install something new the next day, apt-get will tell me it's unable to locate the package.

On this server, cron-apt is installed. I must assume it's installed by default. cron-apt is set to run nightly at 4:00. It seems to be responsible for my lost cache:

oliver@cloud:/var/lib/apt/lists$ sudo apt-get update
...
Fetched 12.1 MB in 11s (1,023 kB/s)
Reading package lists... Done
oliver@cloud:/var/lib/apt/lists$ ls /var/lib/apt/lists/ | wc -l
24
oliver@cloud:/var/lib/apt/lists$ sudo /usr/sbin/cron-apt
oliver@cloud:/var/lib/apt/lists$ ls /var/lib/apt/lists/ | wc -l
8

I'm not really sure why this is happening, because, from what I can tell, one of the jobs of cron-apt is to actually invoke apt-get update:

oliver@cloud:/var/lib/apt/lists$ cat /etc/cron-apt/action.d/0-update
update -o quiet=2
share|improve this question
    
Are the servers always connected to the internet (no strict firewall) ? –  BatchyX Feb 14 '13 at 21:29
    
@BatchyX: Yes, they are –  Oliver Salzburg Feb 14 '13 at 21:48
    
“If I want to install a new package tomorrow, the server will not know about it until I update again.” That's not a behavior I've ever seen or heard of. You need to run apt-get update frequently to update the list of available versions, but the list of available package names doesn't change often. Could you give an example of apt forgetting the name of a package? –  Gilles Feb 14 '13 at 23:50
1  
Do you have APT::Periodic::Enable set to 1? There's a nightly cron job (/etc/cron.daily/apt) that runs apt-get update. Have you tweaked your apt configuration? Has cron sent you emails from that job? Do files in /var/lib/apt/lists disappear overnight? –  Gilles Feb 14 '13 at 23:56
    
@Gilles: You pointed me in the right direction. /etc/cron.daily/apt lists the defaults for all settings. The default for Update-Package-Lists (which controls if apt-get update is called) is 0. On this server, there is only minimal config in /etc/apt/apt.conf.d and the setting is untouched. On my other systems, there's a 10periodic or similar which sets Update-Package-Lists to 1. –  Oliver Salzburg Feb 15 '13 at 0:14

5 Answers 5

up vote 2 down vote accepted

I was finally able to solve this issue. As previously assumed, cron-apt was to blame. Or more specifically its configuration.

The first thing that confused me, was that cron-apt was even installed. Because it wasn't installed by default on my other machines. Only on the machines from the VPS provider where I had this issue. I found it confusing that it exists, because I assumed a similar mechanism is already provided through the /etc/cron.daily/apt script. Either way, as shown in the question, running cron-apt reduced the amount of cached information.

I first couldn't see why this was the reason, because cron-apt was actually performing an update every day.

oliver@cloud:/$ cat /etc/cron-apt/action.d/0-update
update -o quiet=2

So what's the deal here?

I only found out what the problem is, by stepping through the /usr/sbin/cron-apt script line by line. I realized that $APTCOMMAND always gets additional $OPTIONS tacked onto it. And the resulting call looked like this:

/usr/bin/apt-get -o quiet=1 -o Dir::Etc::SourceList=/etc/apt/security.sources.list update -o quiet=2

So, I had a look into /etc/cron-apt/config and found the $OPTIONS block:

# General apt options that will be passed to all APTCOMMAND calls.
# Use "-o quiet" instead of "-q" for aptitude compatibility.
#  OPTIONS="-o quiet=1"
# You can for example add an additional sources.list file here.
#  OPTIONS="-o quiet=1 -o Dir::Etc::SourceList=/etc/apt/security.sources.list"
# You can also set an alternative sources.list file here.
#  OPTIONS="-o quiet=1 -o Dir::Etc::SourceList=/etc/apt/security.sources.list -o Dir::Etc::SourceParts=\"/dev/null\""
OPTIONS="-o quiet=1 -o Dir::Etc::SourceList=/etc/apt/security.sources.list"
# If you want to allow unauthenticated and untrusted packages add the
# following to your options directive.
#  OPTIONS="-o quiet=1 -o APT::Get::AllowUnauthenticated=true -o aptitude::Cmdline::ignore-trust-violations=yes"
# To limit the bandwidth used use the following line. This example limit the
# bandwidth usage to 25 kB/s.
#  OPTIONS="-o Acquire::http::Dl-Limit=25"

One line sticks out. The one that isn't commented. And it's pretty obvious that this line isn't there by default (which I also quickly confirmed on another system).

In case it isn't apparent, the cause of those options is that the cached sources are overwritten every night with the contents of the security lists (and only those).

The VPS provider has since confirmed that this is a customization of their Debian installation image. I have removed the problematic setting and now everything is back to normal.

share|improve this answer

Of course, you know you don't have to update until you need it.....What is happening is that the part of the catalog of information gets transmitted locally to you computer to make apt-get hopefully a little faster. Instead of multiple requests to the remote server for the entire catalog, it only makes one and then updates it as necessary. Again this process does make it faster for you, allows you to see all the packages currently available, and takes the load off of all the people simply trying to look at the catalog by loading it only once locally.

If you are only doing one package during a large period of time, it probably does appear to be a giantic time waster...but its implementation really helps everybody.

share|improve this answer

Usually it just means that exact version, information about which you fetched with last update, is no longer available on Debian servers as it was replaced by new one. But it shouldn't happen that much on Squeeze (even considering security updates). Other option is that some weird script (I don't know of any such one) periodically cleans your /var/lib/apt/lists directory that caches packages lists available on servers (and which apt-get update fetches). Unless you are following unstable or even experimental on your servers (which seems little weird) in which case packages are updated quite often, so cached lists are getting outdated pretty fast. So my advice would be to train your muscle memory to call apt-get update first, each time you are starting some installation session.

share|improve this answer
    
It seems like your assumption regarding /var/lib/apt/lists is correct. When I apt-get update the folder will be filled with package lists. At 4:00 every night, cron-apt is invoked. Afterwards, the folder is noticeably more empty :P This is highly confusing because, to my understanding, one of the set jobs for cron-apt is to invoke apt-get update. I'll see what else I can find. –  Oliver Salzburg Feb 19 '13 at 13:03

To avoid manually updating your package list, you could automate this task every night with a cron job. Use 'crontab -e' and enter the following line.

0 23 * * * apt-get update

share|improve this answer

Your question is a little vague. If the question is "why do I have to run apt-get update" the answer is that the updating of apt can be a time consuming process. Doing it every time I had to install a package would add considerably more time. Usually updating the cache once a day (ideally, in an automated manner) is more than sufficient.

"If I want to install a new package tomorrow, the server will not know about it until I update again. I've only noticed this on Debian servers and I'm not aware of any adjustments that would cause this."

I'm fuzzy on this part. You're saying that if a new version of unison is available tomorrow, how do you know? Again, if you have apt-get update running at least once a day (say, via a cron script) then apt will learn it has new packages. That said, it's not going to automatically tell you that new packages are available; this is with good reasons, since upgrading packages can occasionally cause system stability issues, and the user is necessary to resolve them.

Generally speaking, on a Debian based system, incremental newer versions of software isn't necessarily better. I generally don't care about minor version upgrades except when there's a security issue, or if there's a feature in a new version that I know I need.

share|improve this answer
    
I installed a different package on that server yesterday and, thus, had to run apt-get update. Today, I want to install unison, but the server tells me he doesn't know the package. Shouldn't the apt-get update I invoked yesterday have populated my local cache with the information about the unison package? –  Oliver Salzburg Feb 14 '13 at 21:51
    
Yes, it should have. There's a chance your sources.list file may have been changed by you or someone else. unison is in the 'universe' component, so your /etc/apt/sources.list file should show something like: deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ oneiric universe –  Stephan Feb 15 '13 at 0:04
    
I'm on Debian. I'm currently assuming that I was just used to systems having Update-Package-Lists enabled. –  Oliver Salzburg Feb 15 '13 at 1:06

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.