Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
I found a collection of slackbuilds, some i need there are on GitHub. https://github.com/PhantomX/slackbuilds/ I don't want to get all git.
git clone https://github.com/PhantomX/slackbuilds.git
But only get a slackbuild, for this one.
How to do this? Is it possible?
You will end up downloading the entire history, so I don't see much benefit in it, but you can checkout specific parts using a "sparse" checkout. Quoting this Stack Overflow post:
The steps to do a sparse clone are as follows:
mkdir <repo> cd <repo> git init git remote add -f origin <url>
I'm going to interrupt here. Since I'm quoting another post, I don't want to edit the quoted parts, but do not use -f with git remote add. It will do a fetch, which will pull in the entire history. Just add the remote without a fetch:
git remote add origin <url>
And then do a shallow fetch like described later.
This creates an empty repository with your remote, and fetches all objects but doesn't check them out. Then do:
git config core.sparseCheckout trueNow you need to define which files/folders you want to actually check out. This is done by listing them in
.git/info/sparse-checkout, eg:echo "some/dir/" >> .git/info/sparse-checkout echo "another/sub/tree" >> .git/info/sparse-checkout[...]
You might want to have a look at the extended tutorial and you should probably read the official documentation for sparse checkout.
You might be better off using a shallow clone. Instead of a normal git pull, try:
git pull --depth=1 origin master
I had an occasion to test this again recently, trying to get only the Ubuntu Mono Powerline fonts. The steps above ended up downloading some 11 MB, where the Ubuntu Fonts themselves are ~900 KB:
% git pull --depth=1 origin master
remote: Enumerating objects: 310, done.
remote: Counting objects: 100% (310/310), done.
remote: Compressing objects: 100% (236/236), done.
remote: Total 310 (delta 75), reused 260 (delta 71), pack-reused 0
Receiving objects: 100% (310/310), 10.40 MiB | 3.25 MiB/s, done.
Resolving deltas: 100% (75/75), done.
From https://github.com/powerline/fonts
* branch master -> FETCH_HEAD
* [new branch] master -> origin/master
% du -hxd1 .
11M ./.git
824K ./UbuntuMono
12M .
A normal clone took about 20 MB. There's some savings, but not enough.
Using the --filter + checkout method in Ciro Santilli's answer really cuts down the size, but as mentioned there, downloads each blob one by one, which is slow:
% git fetch --depth=1 --filter=blob:none
remote: Enumerating objects: 52, done.
remote: Counting objects: 100% (52/52), done.
remote: Compressing objects: 100% (49/49), done.
remote: Total 52 (delta 1), reused 35 (delta 1), pack-reused 0
Receiving objects: 100% (52/52), 14.55 KiB | 1.32 MiB/s, done.
Resolving deltas: 100% (1/1), done.
From https://github.com/powerline/fonts
* [new branch] master -> origin/master
* [new branch] terminus -> origin/terminus
% git checkout origin/master -- UbuntuMono
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (1/1), 1.98 KiB | 1.98 MiB/s, done.
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (1/1), 581 bytes | 581.00 KiB/s, done.
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (1/1), 121.43 KiB | 609.00 KiB/s, done.
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (1/1), 100.66 KiB | 512.00 KiB/s, done.
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (1/1), 107.62 KiB | 583.00 KiB/s, done.
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (1/1), 112.15 KiB | 791.00 KiB/s, done.
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (1/1), 454 bytes | 454.00 KiB/s, done.
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (1/1), 468 bytes | 468.00 KiB/s, done.
% du -hxd1 .
692K ./.git
824K ./UbuntuMono
1.5M .
TL;DR: Use all of --filter, sparse checkout and shallow clone to reduce the total download, or only use sparse checkout + shallow clone if you don't care about the total download and just want that one directory however it may be obtained.
error: Sparse checkout leaves no entry on working directory after doing a git pull after making the modification to core.sparseCheckout and .git/info/sparse-checkout
Aug 7, 2017 at 12:00
git init; git remote ... just set up things, nothing is downloaded. The whole point of sparse is to get only objects you are interested in.
-f option is for: "With -f option, git fetch <name> is run immediately after the remote information is set up."
git remote add -f origin <url> git is still downloading the whole repo. In my case, I don't want to spend time waiting for that, even if it doesn't checkout the entire repo.
Apr 28, 2020 at 20:01
git clone --filter + git sparse-checkout downloads only the required files
E.g., to clone only files in subdirectory small/ in this test repository: https://github.com/cirosantilli/test-git-partial-clone-big-small-no-bigtree
git clone -n --depth=1 --filter=tree:0 \
https://github.com/cirosantilli/test-git-partial-clone-big-small-no-bigtree
cd test-git-partial-clone-big-small-no-bigtree
git sparse-checkout set --no-cone small
git checkout
You could also select multiple directories for download with:
git sparse-checkout set --no-cone small small2
This method doesn't work for individual files however, but here is another method that does: https://stackoverflow.com/questions/2466735/how-to-sparsely-checkout-only-one-single-file-from-a-git-repository/52270527#52270527
In this test, clone is basically instantaneous, and we can confirm that the cloned repository is very small as desired:
du --apparent-size -hs * .* | sort -hs
giving:
2.0K small
226K .git
That test repository contains:
big/ subdirectory with 10x 10MB files0, 1, ... 9 on toplevel (this is because certain previous attempts would download toplevel files)small/ and small2/ subdirectories with 1000 files of size one byte eachAll contents are pseudo-random and therefore incompressible, so we can easily notice if any of the big files were downloaded, e.g. with ncdu.
So if you download anything you didn't want, you would get 100 MB extra, and it would be very noticeable.
On the above, git clone downloads a single object, presumably the commit:
Cloning into 'test-git-partial-clone-big-small'...
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (1/1), done.
and then the final checkout downloads the files we requested:
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 3 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), 10.19 KiB | 2.04 MiB/s, done.
remote: Enumerating objects: 253, done.
remote: Counting objects: 100% (253/253), done.
Receiving objects: 100% (253/253), 2.50 KiB | 2.50 MiB/s, done.
remote: Total 253 (delta 0), reused 253 (delta 0), pack-reused 0
Your branch is up to date with 'origin/master'.
Tested on git 2.37.2, Ubuntu 22.10, on January 2023.
TODO also prevent download of unneeded tree objects
The above method downloads all Git tree objects (i.e. directory listings, but not actual file contents). We can confirm that by running:
git ls-files
and seeing that it contains the directories large files such as:
big/0
In most projects this won't be an issue, but the perfectionist in me would like to avoid them.
I've also created a very extreme repository with some very large tree objects (100 MB) under the directory big_tree: https://github.com/cirosantilli/test-git-partial-clone-big-small
Let me know if anyone finds a way to clone just the small/ directory from it!
About the commands
The --filter option was added together with an update to the remote protocol, and it truly prevents objects from being downloaded from the server.
The sparse-checkout part is also needed unfortunately. You can also only download certain files with the much more understandable:
git clone --depth 1 --filter=blob:none --no-checkout \
https://github.com/cirosantilli/test-git-partial-clone-big-small
cd test-git-partial-clone-big-small
git checkout master -- d1
but that method for some reason downloads files one by one very slowly, making it unusable unless you have very few files in the directory.
Another less verbose but failed attempt was:
git clone --depth 1 --filter=blob:none --sparse \
https://github.com/cirosantilli/test-git-partial-clone-big-small
cd test-git-partial-clone-big-small
git sparse-checkout set small
but that downloads all files in the toplevel directory: https://stackoverflow.com/questions/75311408/how-to-prevent-git-clone-filter-blobnone-sparse-from-downloading-files-on-t
The dream: any directory can have web interface metadata
This feature could revolutionize Git.
Imagine having all the code base of your enterprise in a single monorepo without ugly third-party tools like repo.
Imagine storing huge blobs directly in the repo without any ugly third party extensions.
Imagine if GitHub would allow per file / directory metadata like stars and permissions, so you can store all your personal stuff under a single repo.
Imagine if submodules were treated exactly like regular directories: just request a tree SHA, and a DNS-like mechanism resolves your request, first looking on your local ~/.git, then first to closer servers (your enterprise's mirror / cache) and ending up on GitHub.
I have a dream.
The test cone monorepo philosophy
This is a possible philosophy for monorepo maintenance without submodules.
We want to avoid submodules because it is annoying to have to commit to two separate repositories every time you make a change that has a submodule and non-submodule component.
Every directory with a Makefile or analogous should build and test itself.
Such directories can depend on either:
Until git starts supporting this natively (i.e. submodules that can track only subdirectories), we can support this with some metadata in a git tracked file:
monorepo.json
{
"path": "some/useful/lib",
"sha": 12341234123412341234,
}
where sha refers to the usual SHA of the entire repository. Then we need scripts that will checkout such directories e.g. under a gitignored monorepo folder:
monorepo/som/useful/lib
Whenever you change a file, you have to go up the tree and test all directories that have Makefile. This is because directories can depend on subdirectories at their latest versions, so you could always break something above you.
Related:
Try this:
svn export https://github.com/PhantomX/slackbuilds/trunk/${directory}
working solution :
you need to turn off the compression:
git config --global core.compression 0
then you need to use shallow clone
git clone --depth=1 <url>
then most important step is to cd into your cloned project
cd <shallow cloned project dir>
now deopen the clone,step by step
git fetch --depth=N, with increasing N
eg.
git fetch --depth=4
then,
git fetch --depth=100
then,
git fetch --depth=500
you can choose how many steps you want by replacing this N,
and finally download all of the remaining revisions using,
git fetch --unshallow
upvote if it helps you :)
I got this working finally thanks to the above. I was editing the file directly, getting errors by accidently including the quotes. Your .git/info/sparse-checkout file should include just the directory, no quotes. Here's an example: This gigantic project: https://github.com/SharePoint/sp-dev-fx-webparts, I wanted just the react-script-editor in the samples directory. https://github.com/SharePoint/sp-dev-fx-webparts/tree/master/samples/react-script-editor
I followed the directions above and got it working when my .git/info/sparse-checkout file had just this in it
samples/react-script-editor
This will clone a specific folder and remove all history unrelated to it.
git clone --single-branch -b {branch} [email protected]:{user}/{repo}.git
git filter-branch --subdirectory-filter {path/to/folder} HEAD
git remote remove origin
git remote add origin [email protected]:{user}/{new-repo}.git
git push -u origin master
--depth=1 to avoid), and all other directories (??? to avoid?).
For GitHub repos, you can clone any sub-directories of any GitHub repository (at any reference) using https://github.com/HR/github-clone
I'll answer with specific restrictions I faced.
Here's what I did:
git clone --depth=1 --single-branch --branch emptyBranch <remote>
(pretty quick)git checkout <branchToSyncTo> <directoryToSyncTo>
(checks-out only specific folder)