1

I'm using a cloud-based Linux server: Linux seller-huge.novalocal 3.10.0-862.11.6.el7.x86_64 #1 SMP Tue Aug 14 21:49:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux.

I use this to do data scraping, it worked well it the past, but all my python3 scripts break down today with the following error:

Traceback (most recent call last):
  File "feedback_1.py", line 82, in main
    scrape_wr.writerow(result_scrape)
OSError: [Errno 28] No space left on device

During handling of the above exception, another exception occurred:

OSError: [Errno 28] No space left on device

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "feedback_1.py", line 85, in <module>
    main(0, 200000, 0)

I thought this should be caused by unavailable disk space, So I searched online and figured how to mount new volumes to my instance:

[gaojia@seller-huge feedback_scrape]$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        20G   20G   20K 100% /
devtmpfs         32G     0   32G   0% /dev
tmpfs            32G     0   32G   0% /dev/shm
tmpfs            32G   33M   32G   1% /run
tmpfs            32G     0   32G   0% /sys/fs/cgroup
tmpfs           6.3G     0  6.3G   0% /run/user/1000
/dev/vdc        197G   61M  187G   1% /myspace3
/dev/vdb         50G   53M   47G   1% /myspace2

vdc and vdb are newly added volumes, judging on the result of df-h, I think they are mounted correctly, but how could I use them then? I mean how could I put data into these volumes?

3
  • go to the directory that is named in the last column
    – jsotola
    Commented Nov 1, 2018 at 21:29
  • Then what? like I put all my scraping scripts, like scraping.py under this directory and run it from there?
    – Jason Goal
    Commented Nov 1, 2018 at 21:33
  • you asked how to put data into the mounted volumes ..... you can put whatever you want into those directories ..... it seems like you are using comments to ask a question that is unrelated to your original question ..... or your original question is misleading and does not ask the true question
    – jsotola
    Commented Nov 1, 2018 at 21:38

3 Answers 3

2

You can write data to those volumes at /myspace2 and /myspace3 as normal.

The issue is that /dev/vda, which is mounted at / and contains the operating system itself, is completely full.

You need to clear space off of / first or else you're going to have a ton of problems. For example, you aren't going to be able to install any software with a package manager because it's going to need to write to /. Your logs are normally located at /var/log and there is a lot of software that needs to write to locations such as subdirectories of /var/lib and /opt. 20GB really isn't enough for /. As for running your data scrapes, you'll need to modify them to write to /myspace2 and /myspace3.

6
  • Thanks for your reply, just like you said, I deleted a bunch of files from the boot device, now there is 14G free space. To avoid the same problem, I want to put my scripts under /myspace2 and run them from there, will this automatically store whatever data produced by the scripts on this disk mounted on /myspace2? If so I had a problem sending my scripts there(/myspace2)scp: Permission denied
    – Jason Goal
    Commented Nov 1, 2018 at 21:39
  • @JasonGoal If you have root on the server then you can create a directory on /myspace2 with mkdir /myspace2/mydirectory and then make yourself the owner of it with chown so that you can write to it: chown jason:jasongroup /myspace2/mydirectory. I don't know enough about your scripts to know whether or not they will write to where they are located so you'll have to look at them and make sure that they are set to write to /myspace2/mydirectory. Commented Nov 1, 2018 at 21:47
  • thanks, now I can successfully scp some files to /myspace2/mydirectory under usergaojia rather than root, but when I use shell to check this directory, nothing is there.
    – Jason Goal
    Commented Nov 1, 2018 at 22:02
  • My fault, used the wrong directory name.
    – Jason Goal
    Commented Nov 1, 2018 at 22:13
  • @JasonGoal Make sure that the code in your scripts are set to write to that location. Commented Nov 1, 2018 at 22:16
1

By saying "put data into these volumes" I am guessing you just want your script to write data to these newly added disks. In that case, you'd need to edit your script to point the destination directory to the mount points to which these disks are mounted on. From the output of 'df -h', /dev/vdc is mounted on /myspace3 and /dev/vdb is mounted on /myspace2.

/myspace{2,3} are just like any other directory on your linux filesystem.

$ cp something.txt /myspace3

will copy 'something.txt' file in '/myspace3' directory; and

cp something1.txt /myspace2

will copy 'something1.txt' in /myspace2 directory.

-1

If your provider allows you to increase the size of vda, do that (~50G), then reboot.

Use fdisk /dev/vda to delete the vda1 and create a new partition starting at the same sector (should be default), taking up the entire disk.

Then assuming the root filesystem is ext3 or ext4, resize2fs /dev/vda1.

6
  • That is a lot more dangerous than copying date to the new volumes! At least recommend making a backup first!
    – Law29
    Commented Nov 1, 2018 at 21:57
  • the mount says its root in the question. fdisk only alters partition information. Provided the new partition just needs to be bigger and start at the same location. Its actually not that risky. Taking a backup of the partition information is a good step however.
    – danblack
    Commented Nov 1, 2018 at 22:31
  • @danblack /dev/vda1 contains boot and the OS itself. It is extremely risky because you told him to delete it. That isn't even possible in the way that you suggested because the filesystem would be in use at the time. If he were to delete it by another means then he'd wipe out his entire OS. Commented Nov 1, 2018 at 23:25
  • @nasir-riley, I suggest trying it and see. I'm recommending this based on doing it at least 20 times (over a number of years, the most recent of which was last week, which involved LVM too), most of the times had the filesystem mounted, and I understand the underlying mechanisms. I didn't say to 'delete it by another means' so the OP is hopefully smart enough to follow the directions.
    – danblack
    Commented Nov 1, 2018 at 23:56
  • @danblack He doesn't have an LVM. If he did then the output of the df -h command would have /dev/mapper/name_vg-lv_name under Filesystem. He stated that he added new volumes to his system. Nowhere did he state that he added new volume groups. I don't know what Linux you've been working with but you can't use fdisk to delete the partition that the OS is on while the system is up and running. If he were able to do that then he'd wipe his entire installation out. If you have an understanding of this then it shouldn't be that hard to see why your suggestion is a horrible idea. Commented Nov 2, 2018 at 0:34

Not the answer you're looking for? Browse other questions tagged .