What is the benefit of /etc/apt/sources.list.d over /etc/apt/sources.list

Question

I know this question has been asked before, but I do not accept the answer, "you can clearly see custom additions". When I add ppa's (which I have not done in years), I hit a key on my keyboard labeled "Enter" which allows me to add an empty line before the new entry (I would even add an explanatory comment, but I am a tech writer, so ....). I like my sources.conf clean and neat.

/etc/apt/sources.d

Means I have half a dozen files to parse instead of just one.

AFAIK, there is "absolutely" no advantage in having one configuration file vs 6 (for sake of argument, maybe you have 3 or even 2, doesn't matter ... 1 still beats 2).

Can somebody please come up with a rational advantage, "you can clearly see custom additions" is a poor man's excuse.

I must add, I love change, however, ONLY when there are benefits introduced by the change.

Edit after first response:

It allows new installations that need their own repos to not have to search a flat file to ensure that it is not adding duplicate entries.

Now, they have to search a directory for dupe's instead of a flat file. Unless they assume admin's don't change things ...

It allows a system administrator to easily disable (by renaming) or remove (by deleting) a repository set without having to edit a monolithic file.

Admin has to grep directory to find appropriate file to rename, before, he would search ONE file and comment out a line, a sed one-liner for "almost" any admin.

It allows a package maintainer to give a simple command to update repository locations without having to worry about inadvertently changing the configuration for unrelated repositories.

I do not understand this one, I "assume" package maintainer knows the URL of his repository. Again, has to sed a directory instead of a single file.

Answer 1

Having each repository (or collection of repositories) in its own file makes it simpler to manage, both by hand and programmatically:

• It allows new installations that need their own repos to not have to search a flat file to ensure that it is not adding duplicate entries.
• It allows a system administrator to easily disable (by renaming) or remove (by deleting) a repository set without having to edit a monolithic file.
• It allows a package maintainer to give a simple command to update repository locations without having to worry about inadvertently changing the configuration for unrelated repositories.

Answer 2

On a technical level, as someone who has had to handle these changes in a few large and popular system info tools, basically it comes down to this:

For sources.list.d/

# to add
if [[ ! -e /etc/apt/sources.list.d/some_repo.list ]];then
  echo 'some repo line for apt' > /etc/apt/sources.list.d/some_repo.list
fi

# to delete
if [[ -e /etc/apt/sources.list.d/some_repo.list ]];then
  rm -f /etc/apt/sources.list.d/some_repo.list
fi

Note that unless they are also doing the same check as below, if you had commented out a repo line, these tests would be wrong. If they are doing the same check as below, then it's the same exact complexity, except carried out over many files, not one. Also, unless they are checking ALL possible files, they can, and often do, add a duplicate item, which then makes apt complain, until you delete one of them.

For sources.list

# to add. Respect commented out lines. Bonus points for uncommenting
# line instead of adding a new line
if [[ -z $( grep -E '\s*[^#]\s*some repo line for apt' /etc/apt/sources.list ) ]];then
  echo 'some repo line for apt' >> /etc/apt/sources.list
fi

# to delete. Delete whether commented out or not. Bonus for not
# deleting if commented out, thus respecting the user's wishes
sed -i '/.*some repo line for apt.*/d' /etc/apt/sources.list

The Google Chrome devs didn't check for the presence of Google Chrome sources, relying on the exact file name their Chrome package would create to be present. In all other cases, they would create a new file in sources.list.d named exactly the way they wanted.

To see what sources you have, of course, it's not so pretty, since you can't get easier to read and maintain than:

cat /etc/sources.list

So this was basically done for the purpose of automated updates, and to provide easy single commands you could give to users, as far as I can tell. For users, it means that they have to read many files instead of 1 file to see if they have a repo added, and for apt, it means it has to read many files instead of one file as well.

Since in the real world, if you were going to do this well, you have to support checks against all the files, regardless of what they are named, and then test if the action to be carried out is required or not required.

However, if you were not going to do it well, you'd just ignore the checks to see if the item is somewhere in sources, and just check for the file name. I believe that's what most automated stuff does, but since in the end, I simply had to check everything so I could list it and act based on if one of those files matched, the only real result was making it a lot more complicated.

Bulk Edits

Given running many servers, I'd be tempted to just script a nightly job that loops through /etc/apt/sources.list.d/ and checks first to make sure the item is not in sources.list already, then if it is not, add that item to sources.list, delete the sources.list.d file, and if already in sources.list, just delete the sources.list.d file

Since there is NO negative to using only sources.list beyond simplicity and ease of maintenance, adding something like that might not be a bad idea, particularly given creative random actions by sys admins.

As noted in the above comment, inxi -r will neatly print out per file the active repos, but will not of course edit or alter them, so that would be only half the solution. If it's many distributions, it's a pain learning how each does it, that's for sure, and randomness certainly is the rule rather than the exception sadly.

Answer 3

If you're manually managing your servers I'll agree it makes things more confusing. However, it benefits programmatic management (i.e. "configuration as code"). When using config management software like Puppet, Ansible, Chef, etc., it's easier to just drop or remove a file in a dir and run apt update, instead of parsing a file to add or remove certain lines.

Especially since that avoids having to manage the contents of a single 'file' resource, e.g: /etc/apt/sources.list, from multiple independent modules that have been written by third parties.

I appreciate Ubuntu's broad use of ".d" dirs for this particular reason, i.e. sudoers.d, rsyslog.d, sysctl.d., cron.d, logrotate.d, etc.

Answer 4

As nemo pointed out in a comment, one of the key advantages of a directory is it allows for the notion of "ownership".

Modern Linux distributions and installers are all based around the idea of packages - independent pieces of software which can, as much as possible, be added and removed atomically. Whenever you install a package with dpkg (and therefore apt), it keeps track of which files on the system were created by that installer. Uninstalling the package can then largely consist of deleting those files.

The currently accepted answer takes it as a bad thing that a Google Chrome installer assumed that it should only create or delete an entry in the location it expected, but automating anything else leads to all sorts of horrible edge cases; for instance:

• If the line exists in sources.list when installing, but is commented out, should the installer uncomment it, or add a duplicate?
• If the uninstaller removes the line, but the user has added or edited comments next to it, the file will be left with broken commentary.
• If the user manually added the line, the installer could know not to add it; but how would the uninstaller know not to remove it?

Separate files are not required for ownership; for instance, the installer could include a block of comments stating that it "owns" a particular set of lines. In that case, it would always search the file for that exact block, not for some other mention of the same repository.

All else being equal, automating edits to a single configuration file will always be more complicated than automating creation and deletion of a separate file. At the very least, removing lines requires use of some pattern-matching tool such as sed. In a more complex file, both adding and removing lines might require a scripting tool with knowledge of the file format, to add to an appropriate section, or remove without damaging surrounding formatting.

Since an installer would need to avoid messing with manually edited configuration anyway, it makes sense to put automated, tool-owned, configuration in a format that is easy for automated tools to manage.

Answer 5

This allows packages to add extra sources without resorting to scripts.

For example, when you install Microsoft's Skype package, a source for skype.com is automatically configured to download updates; removing the Skype package from the system also disables this package source again.

If you wanted to have the same effect with a flat file, then the installation scripts for Skype would need to modify your sources.list, which probably a lot of system administrators would find slightly unnerving.

Answer 6

I'm not convinced that there is a good reason - other than it seems fashionable. To me, it breaks a rule that a directory should either be a leaf or a branch - ie that it should contain only files or directories, not a mixture of both.

I suppose that it does make files smaller, so easier to read - in the case for instance of sudo rules, which can be quite long, it does make it easier to have a standardized set of rules for one type of user (say a developer), and add those to the config directory if devs should be allowed to sudo on this machine; thus you need to maintain fewer files - just a file for devs, for admins, for sysops, etc, rather than for every possible combination thereof.

There, I've contradicted myself.

I think that I was unclear.

To my mind, there should be a directory /etc/.../list.d, but all files should be in there. It is ugly, and harder to parse when part of the config is in a separate directory. Separate files for each repo, but all in a single directory, or all in subdirectories, so

/etc/
/etc/${system}
/etc/${system}/common
/etc/${system}/use_1
/etc/${system}/use_2

and so-on.

This allows for the ownership (as others have mentioned) to be fine-grained. the directory /etc/${system} (where system might be yum/apt, or whatever) would be owned by root, group what ever system expects, with rw for group, and --- for other. the use_{1,2} directories could be owned by more appropriate users.

As Seamus has commented, whatever works for you is best.