Questions tagged [file-copy]
Questions regarding copying files from one place to another
664
questions
0
votes
0
answers
24
views
Periodic 10 second pauses/freezes every 15 seconds when copying files on all(!) LUKS encrypted ext4 hard drives (NVMe not affected)
I copied a 10TB file from a LVM on LUKS encrypted ext4 drive to another identically set up drive and noticed that the copy process goes for 15 seconds at full speed (~220MB/s), then pauses for 10 ...
- 33
-2
votes
3
answers
78
views
Transfer files from windows to Kali linux
hello there I have set up a Linux virtual machine installed on my pc from Microsoft store up and running with windows also installed on the same pc. I cannot access files in the windows directory and ...
0
votes
1
answer
47
views
cp AND rsync exclude all files of all types from all subdirectories, only keep directories [duplicate]
when using find . -type d it excludes all files whether in parent directory or in subfolders from the search. I want to extend this into a -exec cp -p or rsync "{}" destination \; so that ...
1
vote
2
answers
87
views
How to break syslog into smaller files without losing data
Here we try to break syslog file into smaller chunks. For that we use following script.
#!/bin/bash
date=$(date +%Y%m%d_%H%M)
cp /path/to/sys.log /path/to/chuck/file.log
cat /dev/null > /path/to/...
-4
votes
1
answer
58
views
copy multiple directories in multiple directories without loop [duplicate]
I want to copy directories from one variable to directories from another one but without loop.
from="fromdir1 fromdir2"
to="todir1 todir2"
I mean fromdir1 to todir1, fromdir2 to ...
0
votes
1
answer
65
views
Copy new files from NAS to External Hard Drive with rsync
I want to copy new files from my NAS to my external hard drive. I did copy the new files manually up to this point. Now I want to use a script to automate this for me. However everything I tried so ...
- 103
10
votes
5
answers
1k
views
Copy file along with creating directory
I have a file in a directory as /a/b/c/xxxxx.sql
I have another directory, /a/b/d, having no further subdirectories.
How should I copy xxxxx.sql into /a/b/d, to have the new file with directory as /a/...
- 101
0
votes
0
answers
2k
views
How to solve the error "cp: cannot create regular file : No such file or directory" in centos7
I tried to copy my WRF output files to the /run/media/maslab-3/exthdd/ directory where my 8 TB external HDD is mounted. But, it is not copying; rather, the following error is throwing:
cp: cannot ...
- 1
0
votes
1
answer
408
views
How to recursively create .sha256 hash files for every file in a folder?
Example I have these files
/sdcard/testfolder/file1
/sdcard/testfolder/file2
/sdcard/testfolder/file3
/sdcard/testfolder/file4.ext
I would like to create .sha256 files for each
/sdcard/testfolder/...
- 101
0
votes
0
answers
45
views
Slow copy speeds from SD card to SSD?
I have a Debian (with kernel 5.10.110 #178) on aarch64 architecture.
I'm trying to copy an SD card (SanDisk Extreme, microSD XC UHS-I V30, 64GB) to an SSD (Intel Optane, 16GB).
Based on the Sandisk's ...
- 319
0
votes
0
answers
42
views
Copying all files and folders of a directory using tar
I am trying to copy my files from ssd to external hdd using linux live boot.
The command I'm writing is:
root@ubuntu:/media/ubuntu/4bd92f4b-6b48-4f4a-83ad-9d82d3be488b/home/pegasus# tar cf - Downloads ...
0
votes
2
answers
54
views
Methodology to copy data from one machine to many others in a fast way
I found some bottleneck on my workflow which is as follows. I do have one master computer which needs to send data to other node machines. This is done in a for loop such as:
for all nodes: rsync <...
- 155
0
votes
1
answer
52
views
How to Tranfser huge amount of Files to slow USB Drive (dd, cp, rsync, 7z, tar)
I have like 5GB of data and a very slow USB-Drive.
Should I use dd, cp or rsync?
Should I compress them first into for example 7z / tar or not?
In short: What is the best way and the best practices ...
- 540
0
votes
2
answers
71
views
Does Unix SCP command continues to copy files althouth the response code is > 0
There was a case where an scp command returned response code > 0 but the files where successfully copied and finally loaded in a db which means that they were not corrupted.
Does Unix SCP command ...
- 1
-1
votes
1
answer
47
views
REQUEST: Appending {number} to end of file upon transfer
I have thousands of directories, within each directory is a subdirectory with a file called file.jpg. Since each subdirectory has a file called file.jpg I cannot transfer all those files into the same ...
1
vote
1
answer
216
views
How to enabling big_writes option during mounting of ntfs drives in kubuntu 22.10
During copying of big files i am suffering from high cpu usage. I read multiple linux forums saying adding big_writes flag during mount of ntfs drives results is less cpu usage.
Below is my ...
1
vote
1
answer
231
views
SCP not working with RegEx
I've been trying to copy some remote files to localhost using regex expression, but it reads the regex as if it would be a regular string and does not find any files matching that.
Any ideas why?
file-...
- 11
2
votes
1
answer
609
views
why is NFS copy speed half that of SSH scp?
RHEL 7.9 x86-64
high end dell servers with Xeon cpu's, 512gb ram, intel nic card
I am the only user on the server(s), and there is no other work load on them
cisco 1gbps wired LAN
data.tar is ~ 50 gb
/...
- 5,806
0
votes
0
answers
634
views
Diagnosing slow sshfs transfer speeds (as compared to http)
I've been trying to set up a way to transfer a reasonable amount of data between two servers (remote and local, both running ubuntu server), including some backups (zipped), pictures, home movies, ...
- 1
0
votes
1
answer
69
views
File bytes was increased after copying files from centos to centos server
I copied files from one server to another server using rsync. The folder file byte size, as measured with du -sb <folder>, increased from 31,755,289,904 bytes to 31,755,505,579.
I'm not sure how ...
- 1
0
votes
1
answer
243
views
Change owning user and group on file creation
I have an issue where I want to connect a very old system (UNIX) to a new machine. This old machine logs in by scp using a hardcoded oldsystemaccess user and copies a file into a subdirectory of the ...
- 95
0
votes
2
answers
21
views
Device name changes without any indication resulted of massive data copying
I have Linux machine with two external ntfs-formatted hard drives. One of the drives contains data, the other one is supposed to be a back-up.
I run Rsync to copy data from the first drive to the ...
- 155
1
vote
1
answer
113
views
Instantly copy files from one location to the other
I have a situation where there are few autosys jobs monitoring the files and then based on the file availability its triggering the dependent jobs.
What I am looking for: As soon as a file arrives ...
- 11
0
votes
1
answer
42
views
Why did rsync hose my root file system when trying to merge in drivers?
On a computer I've been working on trying to get going for a LONG time now and still have not completely managed to do so, I tried the following, based on a suggestion from here:
How to copy-merge two ...
- 131
0
votes
1
answer
22
views
Program/method to repair a partially unreadable file from a backup copy
I have a partially damaged file on an integrity-verifying filesystem (btrfs):
$ pv -c \{30939216-15e2-4563-b77e-2ab8687ec333\}.vdi >/dev/null
pv: {30939216-15e2-4563-b77e-2ab8687ec333}.vdi: read ...
- 5,141
0
votes
0
answers
109
views
in linux, system is slowed very much and CPU 100% when I copy from (non-system-based) usb HDD to another usbHDD. Why?
in linux, system is slowed very much and CPU 100% when I copy from (non-system-based) usb HDD to another usbHDD. Why?
My CPU is 100%. I was told that CPU can be 100% without system be slowed down....
- 212
5
votes
2
answers
397
views
How to find out if the filesystem supports reflink copies?
I try to create a snapshot copy of a directory recursively, but it returns errors for every single file if the filesystem (custom shfs) does not support reflink copies:
# cp -a --reflink=always /mnt/...
- 377
1
vote
0
answers
560
views
Copy from Docker to the host
I'm newbie in docker world. I have a docker inside a Virtualbox image, and I want to copy files to the host. I tried as it is written here, and on many sites:
https://linuxhandbook.com/cp-command/
me@...
- 11
0
votes
1
answer
2k
views
Are there other Linux-to-Linux network mount method besides NFS? [duplicate]
In Linux,
specifically RHEL/CentOS 7.9, but anything equivalent or of a later version...
if I wanted to mount a folder/directory from another Linux system across a network,
all I am aware of is NFS, ...
- 5,806
1
vote
0
answers
815
views
Copying a file with cp is stuck (copying with Dolphin doesn't finish either)
I had a problem with copying files from my hard drive to an USB stick with the Dolphin file-manager in Debian11/KDE. Some were stuck near the middle and some were stuck after the full file-size was ...
- 4,068
1
vote
0
answers
68
views
why top command in linux shows high us and low wa while cp many large files simultaneously?
I am copying 10 large files(which may costs 1 hour) simultaneously, and I used top command to check the cpu usage, but I found out 100.0 us and 0.0 wa. shouldn't it be IO-bound instead of CPU-bound? I ...
- 11
0
votes
1
answer
115
views
How can I clone an entire remote directory tree and file structure but have empty files, via the command line?
On a remote production server I have a directory that has more than 15 Terabytes of data. Also, in addition to Linux permission, ACLs are heavily used to allow departments, users, and daemons fine ...
- 107
0
votes
1
answer
24
views
Copied folder is somehow connected to main folder
I am trying to backup my google drive data because I'm out of storage and have no intention of buying cloud storage (also read as "cheap").
What I did was, I downloaded the Google Drive ...
- 1
2
votes
1
answer
457
views
How to create an "exact" copy of a directory using the shell
Using Bash, how can I copy a directory to another place while keeping all properties exactly the same, including timestamp and permission and all other properties?
user495498
4
votes
0
answers
87
views
How to fully copy linux instance to another machine
I have a remote google cloud instance which runs Debian GNU/Linux 11 (bullseye), and I would like to create not nessesarily a copy of it, but rather I would like to copy everything that is installed ...
- 141
2
votes
1
answer
328
views
rsync hangs after 100% to-chk output
I have an NFS mount of /backup and it is
type nfs4 (rw,nosuid,noexec,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,,clientaddr=192.168.1.2,...
- 5,806
3
votes
1
answer
1k
views
Copy a file and preserve its creation date
I need to copy files and preserve all their metadata, including creation date.
I found ways to use rsync options to preserve modification date, but I also need the creation date to be the same, and ...
- 31
1
vote
2
answers
2k
views
What is cp Command to copy all files + directory structure AND cp Command to skip copying files when they already exist
I need to copy all of the files in a directory while preserving the directory structure to a backup hard drive. Will this work:
find . -name '*.*' -exec cp --parents \{\} /target \;
Then I want to ...
- 11
1
vote
2
answers
141
views
fixing just ownership when forgetting cp -rp [duplicate]
I have a /data folder that has a gazillion files and folders, took all night to copy.
I copied it as root via cp -r /data/* /data2 forgetting the p
All my stuff at the destination is owned by root.
...
- 5,806
9
votes
2
answers
892
views
Is rsync --append able to resume an interrupted copy process without reading all the copied data?
I need to copy one very large file (3TB) on the same machine from one external drive to another. This might take (because of low bandwidth) many days.
So I want to be prepared when I have to interrupt ...
- 529
1
vote
0
answers
379
views
Using server-side copy on a SMB share
According to the SMB docs server-side copy should work on recent versions of SMB via cp --reflink
When trying this with a server running smbd 4.15 (via crazy-max/docker-samba) and a share mounted with ...
- 111
0
votes
1
answer
191
views
cp cannot access the files in a directorty
I have a script that copies the missing files from one machine to another. The code is
#!/bin/bash
MACHINE=machine2
DIRECTORY=/test/oracle/
rsh $MACHINE "cd /test/oracle/"
scp -rp ...
1
vote
2
answers
483
views
Why is the amount rsync sent greater than received, without compression?
I just finished copying a lot of data with rsync:
rsync -avv --progress /src ./dst
And at the end, got this report:
total: matches=0 hash_hits=0 false_alarms=0 data=116101578747
sent 116,174,399,...
- 149
0
votes
3
answers
694
views
Does Unix have a command to read from stdin and write to a file (like tee without sending output to stdout)?
I want to copy files that require root to read/write from one system to another.
My current solution is to use sudo on each system and use tee as shown.
ssh host sudo cat /etc/somefile | sudo tee /etc/...
- 562
22
votes
4
answers
2k
views
How can I copy a /tmp/ directory that is created & deleted by a process?
I am investigating the behavior of a binary on Oracle Linux 9 (XFS filesystem). This binary, when called by a process, creates a directory under /tmp and copies some files to it. This directory gets a ...
- 423
0
votes
1
answer
742
views
Selecting n random files from one directory and copying them to another folder + other files with same same, but different filetype
I have two directories, lets call them X and Y
Within them I have 100k+ files, .jpg files in X and .txt files in Y
I want to randomly select N files from X and copy to folder Z
This should be ...
0
votes
1
answer
84
views
how can I detect if my infinite cp command has failed using bash
I have a little data collector, a Raspberry Pi Zero W, which, on boot, mounts a remote folder on mnt using sshfs. The data transfer is effected using a cp command, e.g., cp device file & where ...
- 11
2
votes
1
answer
2k
views
Copying files from USB drive fails - I/O error or Invalid Argument
I'm trying to copy files from an USB stick to another drive. At least the file names appear to be corrupt, ls shows them as:
'ZHECMIv'$'\027''.PDF'
'ZHEKMI>2.P─F'
ZHENIL~1.PDF
'эeloѤyfɯrɥvdr....
- 2,347
0
votes
2
answers
192
views
In AIX, how to avoid overwrite a file with cp?
We need to periodically archive some big files older than 2 days to a NAS while keeping their directory tree structure. Those files are kept for 7 days in the source directory.
At first we used find ...
- 163
0
votes
1
answer
144
views
How can I optimize the transfer of files between two systems and also trim the file
I have computer 1 logging voltage data to a file volts.json every second.
My second computer connects via ssh and grabs that file every 5 minutes. Splunk indexes that file for a dashboard.
Is scp ...
- 129