0

I have the problem listed here: Oct 2019 upgrade: pip upgrade failure · Issue #5844 · msys2/MINGW-packages: basically, pip on MSYS2 refuses to update because it sees some files. pacman -Syu basically dumps this:

...
:: Proceed with installation? [Y/n] y
:: Retrieving packages...
 mingw-w64-x86_64-gl...     4.6 MiB  4.97M/s 00:01 [#####################] 100%
 mingw-w64-x86_64-gn...  1942.9 KiB  1045K/s 00:02 [#####################] 100%
(18/18) checking keys in keyring                   [#####################] 100%
(18/18) checking package integrity                 [#####################] 100%
(18/18) loading package files                      [#####################] 100%
(18/18) checking for file conflicts                [#####################] 100%
error: failed to commit transaction (conflicting files)
python2-pip: /c/msys64/usr/lib/python2.7/site-packages/pip/_internal/commands/debug.py exists in filesystem
python2-pip: /c/msys64/usr/lib/python2.7/site-packages/pip/_internal/commands/debug.pyc exists in filesystem
...

I don't want to uninstall pip, and it seems (pacman "exists on filesystem" error) that I can "just" move the files elsewhere, then have the update run. Of course, I'd like to preserve the original file locations, of the files that I'm moving.

So, I thought, maybe I could tar those files in an archive (using --remove-files so the originals are also deleted, so they don't trip the update any more), then restore them after update - if any other packages I need, have installed them.

Unfortunately, I really cannot find the right invocation, so that a tar archive is update with files that come in from a while loop (that is, I don't have a "one-line", space-separated list of files).

This results with only a single file in archive:

$ pacman -Syu | grep exists | awk '{print $2}' | while read f; do tar -czf /tmp/bckp.tar "$f"; done
:: Proceed with installation? [Y/n] y
error: failed to commit transaction (conflicting files)
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
...

$ tar tzvf /tmp/bckp.tar
-rw-r--r-- user/None       13854 2019-09-02 09:07 c/msys64/usr/lib/python3.7/site-packages/pip/_vendor/urllib3/packages/rfc3986/validators.py

This won't even run:

$ pacman -Syu | grep exists | awk '{print $2}' | while read f; do tar -czvf --strip-components=1 --append /tmp/bckp.tar.gz "$f"; done
:: Proceed with installation? [Y/n] y
error: failed to commit transaction (conflicting files)
tar: You may not specify more than one '-Acdtrux', '--delete' or  '--test-label' option
Try 'tar --help' or 'tar --usage' for more information.
tar: You may not specify more than one '-Acdtrux', '--delete' or  '--test-label' option
Try 'tar --help' or 'tar --usage' for more information.
...

This won't either:

$ pacman -Syu | grep exists | awk '{print $2}' | while read f; do tar -zvf --strip-components=1 --append /tmp/bckp.tar.gz "$f"; done
:: Proceed with installation? [Y/n] y
error: failed to commit transaction (conflicting files)
tar: Cannot update compressed archives
Try 'tar --help' or 'tar --usage' for more information.
tar: Cannot update compressed archives
Try 'tar --help' or 'tar --usage' for more information.
...

This ends up complaining "no such file or directory" for existing files:

$ pacman -Syu | grep exists | awk '{print $2}' | while read f; do tar -vf --strip-components=1 --append /tmp/bckp.tar.gz "$f"; done
:: Proceed with installation? [Y/n] y
error: failed to commit transaction (conflicting files)
tar: Removing leading `/' from member names
tar: /tmp/bckp.tar.gz: Cannot stat: No such file or directory
/c/msys64/usr/lib/python2.7/site-packages/pip/_internal/commands/debug.py
tar: Removing leading `/' from hard link targets
tar: Exiting with failure status due to previous errors
tar: Removing leading `/' from member names
tar: /tmp/bckp.tar.gz: Cannot stat: No such file or directory
/c/msys64/usr/lib/python2.7/site-packages/pip/_internal/commands/debug.pyc
...

So, assuming I have a list of files (absolute filepaths), of the form /c/msys64/usr/lib/python2.7/site-packages/pip/_internal/commands/debug.pyc, in a while loop - how can I add them all to a .tar archive, such that they end up in it with the first part of the absolute path stripped (i.e. mentioned file would end up in tar archive as msys64/usr/lib/python2.7/site-packages/pip/_internal/commands/debug.pyc)?

  • 1
    Do you need a loop? Using xargs, you can run tar on a number of filenames at once. – muru Oct 8 '19 at 9:27
  • Thanks, @muru - not sure I need a loop, but then, how else would I extract the output from pacman - which contains the files I want to move? I'm trying pacman -Syu | grep exists | awk '{print $2}' | xargs tar -vf --strip-components=1 /tmp/bckp.tar and it fails with tar: You must specify one of the '-Acdtrux', '--delete' or '--test-label' options; also tried pacman -Syu | grep exists | awk '{print $2}' | xargs tar -cvf --strip-components=1 /tmp/bckp.tar, fails with tar: /tmp/bckp.tar: Cannot stat: No such file or directory – sdbbs Oct 8 '19 at 9:41
1

Ok, I think I finally found the right invocation - note, the sed is added because --strip-components works on extraction only; so here we strip the /c/ manually from the filepath strings, and then instruct tar to change directory to /c/ using the -C switch:

$ pacman -Syu | grep exists | awk '{print $2}' | sed 's_/c/__' | xargs tar -cvf /tmp/bckp.tar -C /c/
:: Proceed with installation? [Y/n] y
error: failed to commit transaction (conflicting files)
msys64/usr/lib/python2.7/site-packages/pip/_internal/commands/debug.py
msys64/usr/lib/python2.7/site-packages/pip/_internal/commands/debug.pyc
msys64/usr/lib/python2.7/site-packages/pip/_internal/distributions/__init__.py
msys64/usr/lib/python2.7/site-packages/pip/_internal/distributions/__init__.pyc
...

... and to double-check:

$ tar tvf /tmp/bckp.tar
-rw-r--r-- user/None        3360 2019-09-02 09:08 msys64/usr/lib/python2.7/site-packages/pip/_internal/commands/debug.py
-rw-r--r-- user/None        4154 2019-09-02 09:08 msys64/usr/lib/python2.7/site-packages/pip/_internal/commands/debug.pyc
-rw-r--r-- user/None         861 2019-09-02 09:08 msys64/usr/lib/python2.7/site-packages/pip/_internal/distributions/__init__.py
-rw-r--r-- user/None         986 2019-09-02 09:08 msys64/usr/lib/python2.7/site-packages/pip/_internal/distributions/__init__.pyc
...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.