Questions tagged [data]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2 votes
1 answer
22 views

What is the use/purpose of struct map_segment

I was looking through source code of the Linux kernel (version v2.6.38) and I stumbled upon an undefined struct map_segment defined in exec_domain: struct exec_domain { const char *name; ...
user avatar
0 votes
0 answers
33 views

How can I recover data from a failing drive if the drive has been encrypted with bitlocker

I have an ssd that has failed. I keep getting i/o errors when I try to open the drive. The drive is a gpt drive with windows installed on partition 3 and the windows partition has been encrypted using ...
user avatar
  • 283
0 votes
0 answers
17 views

Instrumentation Tools for Collecting Data in Memory Accesses

We can use instrumentation tools, such as Pin tool, to collect memory accesses of an application. The type (read or write) and memory address can be collected. How about data? Can we use Pin tool to ...
user avatar
0 votes
0 answers
23 views

Data recovery for a specific mp3 file

I am looking for some help for the recovery of data that was erased. This concerns files with mp3 format. I know that a part of the file has been deleted or separated from the main file, the end part, ...
user avatar
  • 1
0 votes
2 answers
34 views

Extracting just GPS location to new files named their date stamp from one huge log file

So, i've huge (over 100k records) log file, and need to extract all GPS locations based on their datestamp. ./production.log.109.gz:I, [2022-02-10T10:00:59.703529 #25190] INFO -- : #<Event::...
user avatar
  • 3
-1 votes
1 answer
22 views

Is there a standard location where executables dependencies are stored?

When I list the contents of the bin folders on my system, I often only see the executable files. Some of the executables require additional files such as data and configuration files in order to run. ...
user avatar
  • 137
0 votes
1 answer
171 views

read raw data from disk and convert

please help to find a way to convert the data as follows: I read from disk using dd utility dd if=/dev/sdb skip=8388608 count=560 iflag=skip_bytes,count_bytes |hexdump -C and I am getting ...
user avatar
  • 1
-1 votes
1 answer
70 views

average of nth line from mutiple files into one average master file

I have 3 files which contain 8 lines of numerical values and text. I'm trying to take the average of each rows across all three files and print a new file with those averages. The three example files ...
user avatar
0 votes
0 answers
49 views

Is it overkill to shutdown a VM before taking an LVM snapshot for backup purposes?

As part of a backup strategy for hundreds of VMs built on LVM volumes, I am considering: Shutdown the VM. Take a snapshot of the LVM volume containing user and application data, including MySQL and ...
user avatar
2 votes
3 answers
489 views

Finding an appropriate backup strategy (borgbackup, restic etc)

I am fully aware that questions like this one have been asked multiple times. However, since reading lots of documentation of various backup tools didn't answer all my questions, I think its fine to ...
user avatar
  • 103
1 vote
2 answers
100 views

How to calculate the size of data transferred between two machines in Ubuntu?

I have two computers. The first one is as a workstation. It runs the deep learning code and connects an AP by a wired connection. The second one is as a robot's computer. It sends the video and other ...
user avatar
  • 111
0 votes
0 answers
150 views

How do I access my files from PMBR disk

I was creating a partition on my external hard disk via the 'Disks' app in Linux mint and I accidentally changed the partition scheme. Now the disk is not recognised in either Windows or Ubuntu and in ...
user avatar
0 votes
1 answer
184 views

Copying an entire directory structure and/or skeleton

How do we copy an entire directory template/structure or skeleton ? As nothing in use instruction: Copy SOURCE to DEST, or multiple SOURCE(s) to DIRECTORY. -a, --archive same as -dR -...
user avatar
0 votes
0 answers
27 views

Removing RAID 1 with no dataloss

I have a hetzner dedicated machine with 2x512gb drives (running debian minimal) and the drives are in RAID 1. I would like to disable the raid on the second drive, so i can use it for other things, ...
user avatar
0 votes
1 answer
128 views

Copy data from AWS S3 bucket to Linux SFTP

I am writing bash script to copy data from AWS S3 bucket to Linux SFTP. #!/bin/bash #!/usr/bin/expect set CURRENT_DATE [lindex $argv 0] CURRENT_DATE=`date +"%Y%m%d"` cd /home/ubuntu/test/...
user avatar
  • 1
1 vote
1 answer
55 views

Does a process' kernel virtual memory contain process specific data?

I don't quite understand this picture from the CS:APP book. It shows how the kernel virtual memory of a process has a region different from other processes. Does this mean, that the kernel in the ...
user avatar
  • 13
3 votes
1 answer
2k views

How secure is blkdiscard?

How secure is blkdiscard when using it on a ssd? I tried blkdiscard /dev/sda on a ssd running Ubuntu and after executing the command the OS doesn't survive a reboot and simply plugging the disk into a ...
user avatar
  • 434
0 votes
1 answer
34 views

Pasting files and concatating files together

I have various years and months of data for numerous site locations. I have computed the average for all months for each needed year. Each Month/Year has its own its own file. Not all sites have data ...
user avatar
2 votes
0 answers
72 views

How to check the accumulated data written by each process

I have pihole running on a single board computer and i'm trying to reduce as much as possible data being written to the sd card. I already have logs and pihole database running in a ramdisk but awk '/...
user avatar
  • 21
-1 votes
4 answers
133 views

Combining different terms in third column into one row according to sample code

I have some genomic data with three columns. Column 1 is samples, Column 2 is cancer type, and Column 3 is mechanism. Each sample has one or more gene mechanisms assigned to it (e.g. 6 ecDNA) in the ...
user avatar
1 vote
1 answer
69 views

How can I separate these two columns in this csv file in Linux/Bash?

I am looking to separate these two columns, each into their own separate text files. This data is from a csv file on Kaggle that contains Titanic passenger data. The first column is the number of ...
user avatar
0 votes
3 answers
81 views

How would I format filtered data to new file?

Hello all I'm trying to read data from file, and check for each state and increment total transactions counts from data: REDACTED The end result should be printed to a new file in this format This ...
user avatar
  • 5
0 votes
1 answer
30 views

How to keep blank data or assign missing data when recompiuting data for unit shift

I have a series of data files with format below. I have a script to read just the data to the needed units (ppm to ppb) thus the *1000 of column $3. The issue which I am running into when awk runs and ...
user avatar
1 vote
1 answer
102 views

Use GNU Parallel when file has a single (long) line

I have a file which is an HTML document, containing a <table> I want to extract data from and output into a csv. This file has 544609657 characters, is about 545 megabytes, all in a single line. ...
user avatar
0 votes
0 answers
54 views

how to pass a string with spaces as argument in shell script [duplicate]

Below is my code to which i'm trying to pass a 3 argument like "client name with datacenter" hostname ip so, "client name with Datacenter" is the 1st argument likewise hostname and ...
user avatar
0 votes
1 answer
199 views

Nextcloud bulk loading new data with Linux command line

I have nearly 40,000 folders with over 187,000 files to add to a nextcloud server. I did quite a bit a research and finally find an answer which I give below. I added this here so others did not need ...
user avatar
0 votes
1 answer
61 views

Transpose large table

guys I am sorry if this has been asked before, but I will give it a try. I have a table (tab delimited text file) that is formatted like this Species1 Species2 Species3 Gene 1 Value ...
user avatar
0 votes
0 answers
170 views

Move data between partition

I have a slow drive with a big chunk of data that I want to cut/paste in an other partition (of the same drive obviously, otherwise I shouldn't speak about partition but drives directly), but this ...
user avatar
  • 83
0 votes
0 answers
127 views

How can I make a dummy directory tree with dummy files that contain realistic data?

I'm looking for preferably a one-liner that drops a directory tree a few layers deep with files that contain "realistic" data, as in, not /dev/zero nor /dev/urandom. Data that noticeably ...
user avatar
  • 303
0 votes
2 answers
191 views

Merging two CSVs based on matching first column (with header)

I have two CSV files. One of which contains a lot of data: main.csv Order No.,invoiceNumber,invoiceAmount,invoiceAmountNet 20001,20010,500,5 20002,20011,400,4 20003,20012,300,3 20004,20013,200,2 20005,...
user avatar
  • 3
1 vote
1 answer
161 views

Computations on matrix and avoiding division by zero

I want to compute average of values in data table Example data matrix S1 1 3 4 5 16 3 3 4 6 0 S2 1 6 4 5 16 3 4 6 6 0 S3 1 3 4 3 4 7 3 5 16 9 S4 ...
user avatar
  • 105
0 votes
2 answers
30 views

how can i pull data based on certain flags for rows of data and create a new file with them

I'm working with Upper Air Balloon data with a format of .... **254 12 1 JAN 2002 1 94703 72501 40.87N 72.87W 20 1116** 2 100 100 250 40 32767 3 **3 ...
user avatar
-1 votes
1 answer
28 views

How to view all the list that can be viewed by update-alternatives?

Tabbing gives me ls data but i want something to be shown like x-session-manager , x-terminal-emulator and so on. How can i get list what can i set?
user avatar
0 votes
1 answer
188 views

How to I remove the all meta-data, identify objects, embedded-data, non-pdf/embedded binary, identity(IDs), embedded-character etc in PDF file?

I want convert the PDF to only TEXT(as unicode) & IMAGE based PDF. So the pdf document should consist only of images and text. I want to do this in a GNU/Linux environment, it would be better for ...
user avatar
  • 184
1 vote
1 answer
500 views

Is it possible to merge 2 hard drives into 1 big volume without losing data on of them?

I got 2 connected hard drives in my CentOS 7 dedicated server, though only one is being used (already have data on it), while the other just sits there unpartitioned and unused. I want to 'add' the ...
user avatar
  • 11
0 votes
0 answers
15 views

Reducing amount of data on remote server

I have a vague question.. which is hard to Google on, so using stackexchange might be more useful. The current situation is local computer: Ubuntu 16.04, not much storage space (~100 Gb) Remote ...
user avatar
  • 101
0 votes
1 answer
218 views

date tz converts with GMT and City Names with same GMT get differ result

I want convert custom string time on UTC between different time zones defined as GMT. I general all answers that is here ok. But some question is with this examples: 1.TZ='GMT-07' date --date='TZ="...
user avatar
  • 1
0 votes
1 answer
49 views

I would like to analyze a TXT file, but the data is a 1 column array, and it needs to be a in a multi-column array

Is there a way for me to manipulate the data (i.e.; change some feature of, so that it looks like it should, a multi-column array) without having to manually go through it? There are thousands of rows ...
user avatar
  • 23
1 vote
1 answer
81 views

awk - arithmetic operation, use different formatting in case field contains a string instead of a number

I am trying to subtract data sets from each other in a single file. However when the script hits a value of na the output is incorrect. Input file: # <header> 2002 328 0.1345 2002 ...
user avatar
1 vote
1 answer
274 views

Is it safe to copy a directory while the contents are constantly changed?

I want to backup a shared directory with lots of smalls files, which are constantly changed by my users (add/delete/edit). As usual, I use tar and filter the content through a compression program like ...
user avatar
  • 445
0 votes
1 answer
719 views

Sort on substring of line (not using a delimiter)

I need some help understanding how the sort command is working in the following examples: cat testfile The input data: thisfilehasduplicates thissnowhasfallen thisduckhasdied thishallwasfull ...
user avatar
  • 103
0 votes
1 answer
47 views

Making use of second (ext3) hard drive

I have two hard drives. Both are FireCudas, "half ssd, half hdd". One has 1TB of space and the other, with my Manjaro on it, has 480GB. Up until now (for 2 years) i have not used the second drive, ...
user avatar
1 vote
0 answers
99 views

Wipe disk with bad sectors [duplicate]

I'm trying to wipe a bad disk using dd with the following command sudo dd if=/dev/zero of=/dev/sda status=progress, but it exits prematurely and outputs the following: 13160186368 bytes (13 GB, 12 ...
user avatar
  • 623
4 votes
1 answer
3k views

Metered network

My mobile ISP handles a data limit of 12,5GB, and I'm using Manjaro Linux. I'm asking myself how a "metered" connection will save data. In the NetworkManager, there are three different options under ...
user avatar
  • 43
0 votes
1 answer
120 views

Data sanitisation question - regular expression for removing combinations of trailing spaces and commas

I have address strings which were (due to poor initial programming) allowed far too much entry of free text. In the course of cleaning up this inherited mess, I have noticed that at the end of the ...
user avatar
  • 3
0 votes
0 answers
211 views

Trying to save a 2012 macbook pro failing disk with ddrescue

I have a Macintosh HD disk that failed suddenly (system would not boot anymore - diskutil complained about a problem with the B-tree. Tried various diskutil commands in the past without success) which ...
user avatar
5 votes
1 answer
412 views

how to sort based on numbers on column with seperator on bash

how to sort these data by fifth column PREPAID|085319338982|Replacement Card|2016-05-31 15:15:29|218BKSAT||14121837 | 0 PREPAID|08128352315|Replacement Card|2016-05-31 15:15:33|218KKSAT||14121842 | 0 ...
user avatar
0 votes
1 answer
118 views

Scrub filesystem to check all used sectors are readable

I'd like to have a way to periodically check that I can indeed read all the data on my drives, so I can re-write any files and/or investigate further if there is an issue. Both ZFS and Btrfs have a ...
user avatar
  • 25.4k
-2 votes
1 answer
597 views

How change the mount point on an external hdd that has data on it, without deleting any data?

I need to change the mountpoint from "/media" to "/mnt" on an external hdd that has data that can not be deleted. I read that I can create a directory using "sudo mkdir", but that is all I have ...
user avatar
-2 votes
1 answer
711 views

Cloning a disk to a larger disk without overwriting

If I clone a hdd to another (larger) hdd that already has data on it, will the older data get deleted or will new data get added on top of old data? I need to move data from 2 smaller hard drives to ...
user avatar