Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [data]

The tag has no usage guidance.

0
votes
0answers
11 views

DDRescue Rescue Rate Concern

I'm in the process of using DDRescue on a 896 GB partition of my 1 TB Toshiba HD. After almost 5 days it's rescued only 7% of my data (according to terminal input). I paused the program for the first ...
0
votes
2answers
66 views

Replace User Name on one file with User Name plus Company from output of another file

I have 2 text files. One is the list of user names along with the company they work on the same line but separated by a tab space. The second is a output from a weekly generated report that includes ...
0
votes
2answers
54 views

df -h results different than VGdisplay / LVdisplay

Im trying to remove the added disk to the VM, as the disk space is not needed anymore. However, it looks to be that the VM has in the VG almost all the data used what is available there. However, i'...
0
votes
3answers
44 views

Grep/Awk for Value Between

So I have a text file with lines like this: http://sg.ovies.m/.ss/ (Status: 200) [Size: 128] http://sg.mo.v/.dev/ (Status: 200) [Size: 12328] http://som.b/.hal/ (Status: 200) [Size: 1238] http://m.cb/...
11
votes
5answers
1k views

Check data integrity after copying thousands of files [duplicate]

I copied thousands of files into an exFAT MicroSD card. The number of files and bytes is identical, but how do I know whether the data is corrupt or not? It would be good if the JackPal Android ...
2
votes
1answer
129 views

What are data integrity / bit-rot protection options on CentOS 7?

I have a 2 disk CentOS 7 machine build that that I need data integrity / bitrot protection on. How can I achieve this? Note from my reading btrfs,zfs and DM-Integrity does not seem to be options. ...
0
votes
1answer
42 views

How to move physical disks content (rsnapshot content in there) to another physical disk?

Situation: simple linux system, with external removable usb media (rdx), single primary partition with ext4 filesystem on them for backups. Using some external usb disk media (rdx disks) for ...
1
vote
1answer
74 views

Data Cleaning Using Shellscript

I want a combined query suitable for selecting rows where lang = en i.e. language is equal to English and removing the other rows while I convert this file from JSON to CSV. json2csv -i Downloads/30....
0
votes
0answers
29 views

HAproxy / rewrite local ip to external ip

I have a local cluster and an external client on the internet. The solution is communicating on several ports, but though a HAproxy upstream LB. The cluster handles many other solutions on the local ...
3
votes
1answer
76 views

How to copy partitions from one device to another, including boot partitions?

Basic Explanation How could I copy partitions sdh1 though shd4 from my 600gb HDD (sdh) to my 1 TB SSD (sdg)? Which are my Windows partitions and still keep it bootable? I used GPT. Below is a ...
1
vote
0answers
13 views

I Node and Data Retrieval

Is there a way possible to open a data block without the file if we know the i node number that it's linked to ? Is it possible to open the content in data block without the file ?
24
votes
2answers
3k views

Purpose of cp -x (stay on file system)?

If I wanted to stay on the same file system, couldn't I just specify an output path for the same file system? Or is it to prevent accidentally leaving the current file system?
1
vote
0answers
17 views

How to see the file revisions in Tux3 and NILFS?

Tux3 and NilFS are file systems made for file history/revisions. How can I see these revisions? On Windows, there is a file history manager and Shadow Copies (enabled by default only on Windows 7 ...
0
votes
0answers
17 views

How to delete backed files only from original storage device?

I have a backup of an SD card but have not yet deleted the original files to free up space. Now, they got mixed up with new files. How do I delete all files that are already backed up (same file name ...
4
votes
0answers
105 views

Is there a standard text mode tool to render heatmaps for arbitrary numeric matrix data?

For the curiosity's sake I created a script to visualize Git history week/hour-of-day basis timestamps in order to collect some stats. The following script #!/bin/bash # Declare the global heatmap ...
0
votes
0answers
25 views

Deactivating disk cache for specific device or partition? (play 2160p video off SD card without sacrificing RAM space other programs need)

I would like to deactivate the disk cache for /dev/mmcblk0p1 in order to preserve more RAM for other system parts while playing 2160p video off /dev/mmcblk0p1. 2160p videos ravenously wipe the RAM ...
2
votes
1answer
41 views

Generating dead LBA list from gddrescue mapfile/logfile?

I ultimately would like to know which files in images generated by (g)ddrescue are affected by damage. gddrescue (command ddrescue) is a tool for recovering data from damaged media. When specifying ...
0
votes
1answer
73 views

Can dvdisaster recover optical discs better than gddrescue?

gddrescue is more universal. But both tools are just limited by hardware restrictions. If a CD's sector is unreadable, the drive usually refuses to let the computer know about the subcode. For ...
0
votes
1answer
16 views

List changed files and folders compared to disk image

Let's call the image Disk.img, taken from /dev/sda. How do I list every file and folder with differences? (Let's assume the time stamps of the files are incorrect.)
0
votes
1answer
65 views

Differentiate bad logical and physical blocks? (list seperately)

The tool badblocks can give a list of unreadable LBA's, including logical errors I guess. How can I differentiate between logical (soft) bad blocks and physical (hard) bad blocks? List logical and ...
0
votes
0answers
17 views

Back up and sync to mirror disk or image file

Let's assume a computer with 1 TB SSD and 2 TB HDD and an empty external 4 TB HDD for backup purposes. Now, to prevent data loss if one of the drives fails, a disk image or a copy of each file can be ...
0
votes
1answer
40 views

Recover Data while installing Liniux by option : “Erase disk and install Linux Mint”

I tried to intsall the Linux Mint by selecting the option by option : "Erase disk and install Linux Mint" assuming that the installation will be on C drive (where windows was installed ) but the ...
0
votes
0answers
13 views

Output data directly to MTP/FTP storage?

My phone (only MTP data access) has a MicroSD card inserted into it Now, I wish to save a ddrescue disk image directly onto it while the phone is connected to the PC.That disk is not damaged at all, ...
0
votes
1answer
376 views

Data getting overwritten with same timestamp in InfluxDB

There are some data in a csv file and it is getting overwritten if the timestamp is similar, for example: ip,time,name 1.1.1.1,2018-08-31 11:12:32,python 1.2.1.2,2018-08-31 11:05:28,oracle 1.3.1.3,...
0
votes
1answer
332 views

Understanding direct and indirect blocks?

Why can't we use use indirect blocks instead of direct blocks. Indirect blocks can store much more memory addresses which results in more data blocks. So why waste those 10 direct blocks in Inode ...
-1
votes
1answer
38 views

Error Displaying last 10 minutes log with awk [duplicate]

I am using below command to display log data for last 10 mins. It works fine if the month stays same however when month is changed it doesn't show any data. awk -v d1="$D1" -v d2="$D2" '$0 > d1 &...
2
votes
5answers
688 views

Grep specific data from a string [duplicate]

Here is the data: Valid from: Fri Nov 24 12:11:28 EST 2017 until: Sun Nov 24 12:11:28 EST 2019 Valid from: Fri Feb 24 13:21:19 EST 2017 until: Thu Feb 24 13:31:19 EST 2022 Valid from: Tue Dec 20 12:...
0
votes
2answers
32 views

Segregate Data in Log files

I want to segregate error types in my log file. I am unable to make a logic for it. Suppose i have these different types of errors in a log file. Now i want to extract those types only. Like in below ...
3
votes
1answer
349 views

Use Variable Date in Sed Command

I want to use sed command as below: $ sed -n '/MONTH_TODAY DATE_YEST/,/MONTH_TODAY DATE_TODAY/p' \ /home/crmiib/TESTUSERLOG/XYZ/user.log I can print logs using: $ sed -n '/Jul 27/,/Jul 28/p' /...
0
votes
1answer
48 views

Combine files into one [duplicate]

If I have a folder with the following file names: cluster_sizes_0.txt cluster_sizes_1.txt cluster_size_2.txt etc. Each file contains a single column of values. Is there a command within linux such ...
1
vote
1answer
108 views

Two consecutive OPs after pipe or two jq OPs in one run?

I have to extract data from a slightly mis-formatted JSON string, hence I first pass it through sed & awk. What I have is a command like: `sed 's/},/},\n/g' test.json |awk '/"characater"/ { gsub("...
2
votes
3answers
246 views

extract data from JSON string

I've got to extract a license plate digit and it's associated confidence from a JSON string that looks like: { "response": { "container": { "id": "0df307bc-06b2-45cf-b7ff-ce07fd04e04d", ...
2
votes
1answer
23 views

Prepare Live-Stick WITH extra data on 2nd partition

I have a USB Stick of 64 GB. I have a debian ISO. I can make myself a bootable USB stick with sudo cp debian.iso /dev/sdb Problem 1: I cannot copy various files to the stick preventing them to be ...
2
votes
1answer
46 views

Verify disk integrity without reading data?

There is an application called “HDDscan” for Windows: http://hddscan.com/ It can also verify disks by reading them in a way they only get transferred into the internal buffer of the drive, and ...
0
votes
2answers
1k views

Removing EFI System Partition from data disk

I have one SSD and one HDD in my computer. I use the SSD as the drive which holds the OSes (Ubuntu and Windows). I installed them on my SSD in BIOS/legacy mode (MBR format) and I still have an ESP on ...
2
votes
0answers
116 views

Home NAS: Best setup to ease data recovery in case of hardware failure? Should I get rid of HW raid 1?

I'm not quite sure how to formulate the question here. I built myself a low energy NAS with an Intel N4200 with 8GB RAM. For the hard drives I'm using two WD red in a Raidon GR3660-B3 which is running ...
1
vote
1answer
58 views

Secure temporary storage

I'm trying to find a way to securely store data for the duration of a session - specifically it's for passwords on a jump box - so the user only needs to enter the password for a given target once in ...
-1
votes
1answer
217 views

Problems with external hard disk

I can't overcome a problem I have with my hard disk! I don't know if it is the right section, but the problem I have started suddenly while I was copying files on Debian. The problem is that my ...
0
votes
2answers
161 views

Using if and shell variables inside an awk script

I am trying to get data from a file that is like this: 6 6 1 0 0.1166667E+02 0.4826611E-09 0.4826611E-09 0.3004786E-09 0.5000000E-15 1.000000000000000E-004 CAR system-001 ...
0
votes
1answer
197 views

Handle if statements and variables with awk

I am trying to get data from a file that is like this: 6 6 1 0 0.1166667E+02 0.4826611E-09 0.4826611E-09 0.3004786E-09 0.5000000E-15 1.000000000000000E-004 CAR system-001 ...
-2
votes
3answers
192 views

Ubuntu 18.04 with a PC tracking mechanism is still a “free software”? [closed]

I read this article that basically says the Ubuntu 18.04 will include a mechanism that collects the following basic data on a user: Which version of Ubuntu you’re installing (including which flavour) ...
3
votes
0answers
319 views

Do rsync --checksum and diff usually have different outputs or is my hdd broken?

I set up a NanoPi Neo 2 with OpenMediaVault (Debian Jessie) as its OS and have an hdd attached to it (the OS is on an SD card). When I now copy my data from my external hard drive onto my system with ...
0
votes
0answers
20 views

Forcing a clean uninterrupted baud when receiving data via serial port

I made a project with a micro-controller that allows me to send data to the PC where the low nibbles of data always follow the same sequence. For example, My data is sent like this: x0 x1 x2 x3 x4 ......
1
vote
1answer
92 views

Pull multiple rows of data from 2 columns in several locations

I have some really poorly formatted data files, and unfortunately the format of the files produced cannot be altered. I was using GAWK to parse the data into a manageable format but now I've been ...
2
votes
1answer
2k views

Ubuntu 16.04 LTS: Value too large for defined data type

I am trying to run Partiview software which is a visualization tool for particle data sets. I have run a large simulation that includes millions of particles and the resultant data file that I am ...
1
vote
1answer
63 views

Transpose a 3D tensor reprensented by files

I have a 3D tensor of dimension MxNxD data object. And this data object is decomposed and stored in different files. Initially, each slice/matrix of dimension MxN (first two dimension) is saved in a ...
4
votes
2answers
5k views

diff two large CSV files (each 90GB) and output to another csv

This is a follow up question from Sort large CSV files (90GB), Disk quota exceeded. So now I have two CSV files sorted, as file1.csv and file2.csv each CSV file has 4 columns, e.g. file 1: ID Date ...
2
votes
0answers
285 views

SMART data of SanDisk Extreme Pro

I have RHEL 6.8 with smartmontool 5.43 on Dell OptiPlex 9020 with SanDisk Extreme Pro 240 GB SSD as OS disk used for quite disk i/o intensive work. I am concerned about durability of SSD. After one ...
3
votes
0answers
488 views

*REAL* zerofree equivalent for btrfs

is there any zerofree equivalent for btrfs ? and please don't give me sfill or cat /dev/zero >/tmp/file, this is not the same thing at all, be cause Zerofree finds the unallocated, non-zeroed blocks, ...
1
vote
1answer
57 views

I need to delete . in my data

I have a data file with 4 columns. 4th column is real variable with 5 decimals followed by 0. I need to remove the "." in this column. I tried using sed command in R system(paste('sed s/./""/g <"...