How to modify a PKGBUILD which uses git sources to pull only a shallow clone?

Question

The other day I tried installing opencv-git from the AUR with makepkg on Arch Linux. Of course it pulls from the git repository as the name indicates. This pulls 1Gb. I am reading about making a shallow clone with git. When I look at the PKGBUILD file, using grep git PKGBUILD, I see:

pkgname="opencv-git"
makedepends=('git' 'cmake' 'python2-numpy' 'mesa' 'eigen2')
provides=("${pkgname%-git}")
conflicts=("${pkgname%-git}")
source=("${pkgname%-git}::git+http://github.com/Itseez/opencv.git"
    cd "${srcdir}/${pkgname%-git}"
    git describe --long | sed -r 's/([^-]*-g)/r\1/;s/-/./g'
    cd "${srcdir}/${pkgname%-git}"
    cd "${srcdir}/${pkgname%-git}"
    cd "${srcdir}/${pkgname%-git}"
    install -Dm644 "LICENSE" "${pkgdir}/usr/share/licenses/${pkgname%-git}/LICENSE"

Is there a way to modify the recipe or the makepkg command to pull only a shallow clone (the latest version of the source is what I want) and not the full repository to save space and bandwidth? Reading man 5 PKGBUILD doesn't provide the insight I'm looking for. Also looked quickly through the makepkg and pacman manpages - can't seem to find how to do that.

Answer 1

This can be done by using a custom dlagent. I do not really understand Arch packaging or how the dlagents work, so I only have a hack answer, but it gets the job done.

The idea is to modify the PKGBUILD to use a custom download agent. I modified the source

"${pkgname%-git}::git+http://github.com/Itseez/opencv.git"

into

"${pkgname%-git}::mygit://opencv.git"

and then defined a new dlagent called mygit which does a shallow clone by

makepkg DLAGENTS='mygit::/usr/bin/git clone --depth 1 http://github.com/Itseez/opencv.git'

Also notice that the repository that is being cloned is hard-coded into the command. Again, this can probably be avoided. Finally, the download location is not what the PKGBUILD expects. To work around this, I simply move the repository after downloading it. I do this by adding

mv "${srcdir}/../mygit:/opencv.git" "${srcdir}/../${pkgname%-git}"

at the beginning of the pkgver function.

I think the cleaner solution would be to figure out what the git+http dlagent is doing and redfine that temporarily. This should avoid all the hack aspects of the solution.

Answer 2

Personally I modified the makepkg script and it's working like a charm:

# vim `which makepkg` +/clone
...
541         msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "git"
542         if ! git clone --mirror "$url" "$dir"; then
543             error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "git"
...

Appending --mirror --single-branch --depth 1 to the git clone command:

541         msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "git"
542         if ! git clone --mirror --single-branch --depth 1 "$url" "$dir"; then
543             error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "git"

Here is a diff view:

--- makepkg ...
+++ makepkg-patched ...
@@ -539,7 +539,7 @@

    if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then
        msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "git"
-       if ! git clone --mirror "$url" "$dir"; then
+       if ! git clone --mirror --single-branch --depth 1 "$url" "$dir"; then
            error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "git"
            plain "$(gettext "Aborting...")"
            exit 1

Answer 3

According to https://bugs.archlinux.org/task/23065 (credit to jasonwryan), adding shallow cloning functionality to AUR PKGBUILD was a wishlist item that was closed on Saturday, 05 March 2011 with the comment:

Reason for closing: Won't implement

This suggests that it is not going to happen unless someone submits a patch.

As I suggested to the poster in comments, what he is trying to do can almost certainly be accomplished by breaking the process up into two steps:

1. Clone the git repository using shallow clone
2. Run the PKGBUILD recipe, but point it at the local clone. I am not an Arch User, so don't know whether this is the case, but I'd be very surprised at a package building system that forced users to clone repositories from remote in order to build packages.

Answer 4

If you make a shallow mirror of the repository in the same directory as the PKGBUILD, you can use makepkg --holdver to prevent makepkg from updating the rest of the repository. This removes the need to modify the PKGBUILD, makepkg.conf, or makepkg itself; however, cloning/updating the repository needs to be done manually.

As an example with cling-git, which would normally clone the entirety of llvm and clang:

$ git clone --mirror --depth=1 --branch=cling-patches http://root.cern.ch/git/llvm.git llvm
$ git clone --mirror --depth=1 --branch=cling-patches http://root.cern.ch/git/clang.git clang
$ git clone --mirror --depth=1 http://root.cern.ch/git/cling.git cling

$ makepkg --holdver

---

From the makepkg man pages:

--holdver
    When using VCS sources (PKGBUILD(5)) any currently checked out source
    will not be updated to the latest revision.

Note that makepkg will still clone repos that are not already present, which means that I could have omitted manually cloning the cling repository in the above example since it's not so large.

Answer 5

If you don't want to modify makepkg scripts.

as outlined here, point DEVELSRCDIR in /etc/yaourtrc or ~/.yaourtrc file to some persistent folder. Then all the repository checkouts (git/svn/...) will happen in this folder. Once repository is cloned, then only quick fetch with newest revisions will be performed instead of full clone each time.

Answer 6

Sorry for answering a answered question, but I don't have enough point to add a comment.

Inspiring by @StrongBad, I added a new protocal for makepkg and it looks ugly but work:

'sgit::/usr/bin/emacs --batch --eval (shell-command\ (concat\ \"git\ clone\ --depth=1\ \"\ (replace-regexp-in-string\ \"#\"\ \"\ --\"\ (substring\ \"%u\ \"\ 1))\ \"%o\"))'

Besides Emacs, any other interpreter is possible to be used here. Then, empty spaces are needed to be escaped to be considered as a single argument, and this is the point that Bash fails to archieve.