Tag Info

New answers tagged

0

I'm not sure it matters that much. You could fdisk -l /dev/sda ; However, usually /dev/sda is your hard or SSD disk, and the USB stick would be e.g. /dev/sdi (then run fdisk -l /dev/sdi). Do a dmesg just after plugging the stick to find out. Be careful: very often /dev/sda is your system disk. Once a file system is mounted from the USB key, you might run ...


2

The Linux kernel has options to restrict what physical address ranges it will use as RAM, but that won't prevent buggy drivers or access through /dev/mem to escape those ranges. You won't gain anything from modifying the MMU configuration during the boot stage, because the kernel will take control of the MMU after that anyway. If you want to be absolutely ...


0

No, It don't. I checked it myself many times, but don't worry, Linux kernel takes care of your precious memory very smartly. read Content cached in RAM while writing to disk - Linux and http://linuxatemyram.com


2

It seems like the -o flag will take the actual column name. So if the top command shows only "mem" then the command should be "top -o mem". For the ubuntu machine I am testing with, the column is called "%MEM". On the OSX Yosemite I tried, it is "mem".


3

I figured out that a large portion of the memory usage was, in fact, attributable to inactive memory used by exited processes. The most accurate way to determine how much memory is available post-January 2014 is to look at MemAvailable in /proc/meminfo. You can also see the amount of inactive memory is this file.


1

Huge pages would be useful in a situation where you needed a huge amount of information to be written in the same block. It can relate to the strategy for disk writes and can be significant for caching. Like all configuration options, it makes no sense if your use case does not fit. So the answer is, workload that actually needs a huge ton of data in the ...


1

One handy way is to use atop. In atop, you can type P to see per program statistics like this: PAUSED NPROCS SYSCPU USRCPU VSIZE RSIZE RDDSK WRDSK SNET MEM CMD 1/4 17 1.14s 8.06s 14.3G 2.7G ? ? ? 35% chrome 1 0.30s 0.30s 2.6G ...


0

You can use qps. It shows the total usage of each process.              Reference qps man page


2

I right clicked the folder and clicked "create archive" and selected the .tar.gz option. The directory structure is deep, over 500,000 directories Yeah, good luck getting that to package up. And the GUI tool will try to do that on the same volume, which means a) you need another 1Tb of free space and b) the head thrashing of reading one file and ...


2

7z would be my choice. It allows auto-splitting of archives and supports multi-threaded compression. No, xz doesn't, despite what the help message says. Try with: 7za a -v100m -m0=lzma2 -mx=9 -ms=on -mmt=$THREADS archive.7z directory/ The output is split in 100MB blocks (change it with the -v switch). The only real downside is that 7z does not retain ...


4

Did you consider torrent? Peer-to-Peer might be your best option for an over-the-internet transfer: At least as fast as other internet transfers: your upload speed will determine the transfer speed No data corruption Choose which files to transfer first No extra local/cloud storage space needed Free You didn't tell which OS you were using, but as you're ...


27

Additional information provided in the comments reveals that the OP is using a GUI method to create the .tar.gz file. GUI software often includes a lot more bloat than the equivalent command line equivalent software, or performs additional unnecessary tasks for the sake of some "extra" feature such as a progress bar. It wouldn't surprise me if the GUI ...


12

"five million" files, and 1TB in total? Your files must be very small, then. I'd simply try rsync: rsync -alPEmivvz /source/dir remote.host.tld:/base/dir If you don't have that - or your use-case doesn't allow for using rsync, I'd at least check if 7z works with your data. It might not, but I think it's still worth a try: 7z a archive.7z /source/dir Or ...


7

Unless you can do better than 25:1 compression you are unlikely to gain anything from compressing this before snail-mailing, unless you have some hardware tape format that you can exchange the the third party. The largest common storage is blue ray and that will roughly get you 40Gb. You would need 25 to 1 compression on your data to get it to fit on that. ...


0

I suggest you to use something like : tar -c -L 104857600 -f file1.tar -f file2.tar -f file3.tar -f file4.tar -f file5.tar -f file6.tar -f file7.tar -f file8.tar -f file9.tar -f file10.tar gzip file*.tar which will create 10 files which will be at most 100GB big. But that wouldn't answer your "OOM" issue since tar is supposed to stream it's read/write so ...


0

How about that: find /path | tar -T - -czf - | ssh remotehost "cd /target/dir/; tar xzf -" find searches lists the directory tree tar -T - reads the list from stdin -czf - creates an archive and writes to stdout, z gzips the stream ssh remotehost login to remotehost using ssh cd /target/dir/ change to the target directory tar xzf - extract the incoming ...


0

I think that you can't prevent this crash, if you don't change operating system or computer(related to memory) on which you executed your program(mdworker). In practice, allocatable memory is equal to RAM+(SWAP SPACE), but I guess that your program, mdworker, is over limitation (RAM+SWAP SPACE). Did you allocate too large data(array, etc...)?


0

The manual page of top says: 2c. MEMORY Usage This portion consists of two lines which may express values in kibibytes (KiB) through exbibytes (EiB) depending on the scaling factor enforced with the 'E' interactive command. It also contains definitions of kibibytes, which is discussed in this Super User answer. The manual page for free says ...


0

See the k after each number? Kilobytes. (That's also the default for free, if you don't give -m.)


-1

use this script wget https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py chmod +x ps_mem.py ./ps_mem.py it will show you the ram usage :)


0

You will have to write a short script to parse the output and divide the numbers. Some solutions can be found here http://stackoverflow.com/questions/10585978/linux-command-for-percentage-of-memory-that-is-free Just be careful. What exactly do you mean by "total used"? The first line (Mem) is not what you want... second line is better, because cached memory ...


2

If you are only interested in the memory used after the fact, then use GNU time: command time -v myprogram (the above uses the bash way of invoking the external time command rather than the bash builtin, your shell may vary). Or, GNU memusage: memusage -T ./myprogram If you are interested in the memory used on an ongoing basis (i.e. during a long ...


3

It would depend on what kind of stats you want, but if you're writing a program in C running on Linux, you'd definitely better know about Valgrind. Valgrind can, not only profile detailed memory usage of your program, but also detect memory access violations which are common in C and possibly very hard to debug. For your profiling purpose, take a look at ...


1

Here's the resident set size and virtual memory size of all sshd processes on one system: ulric@qvp2:~$ ps -eo rss,vsz,args|grep sshd|grep -v grep 448 55292 /usr/sbin/sshd -D 5176 147460 sshd: ulric [priv] 2776 149704 sshd: ulric@pts/3 Or perhaps easier: ulric@qvp2:~$ ps aux|head -n 1&&ps aux|grep sshd|grep -v grep USER PID %CPU %MEM ...


0

I have a 4GB System running Gentoo. For comparison purposes, here's my output: MemTotal: 4052288 kB MemFree: 485856 kB MemAvailable: 1895624 kB Buffers: 371420 kB Cached: 1176368 kB SwapCached: 31280 kB Active: 2382336 kB Inactive: 834060 kB Active(anon): 1323456 kB Inactive(anon): 423400 kB ...



Top 50 recent answers are included