2

I have a Linux machine that I own and administer (CLIENT). I have access to a UNIX server, with no administrative privileges at all (SERVER).

Previously, I used to copy a lot of files from this server, processes them in my client, and then push them back to the server.

Lately, I'm working remotely, and to move 5~10 Gb, many times a day; so this approach is not an option anymore.

So what I want do is to:

  1. (optional, if you have a better idea) transfer a binary (compiled to that Unix, I have most of then, but a reference to cross compile could be helpful)
  2. run my binaries against those files, on the SERVER (using their computing power, and memory, since the tasks are huge).
  3. save the results on the server (with a redirection "> file.txt")
  4. not leave (or even install) these apps on the server.

How this kind of result could be accomplished?

Below some additional back ground.

On this specific server, there is a long process to request, and almost certainly not to be granted, authorization to install new programs. Believe-me I've tried. One of the questions taken into consideration for this request was "how will this work be done without this software?", to which my reply was "I've been doing it on my workstation with copies of the files", to which they replied "so continue working that way."

Since I have started these analysis by UNIX, the results are great for the main business; so demand is increasing. I've managed to maintain a solution scalable that is performing well. But the transfers are a big concern now.

I stand to lose my ability to work from home if this analysis cannot be done anymore.

4
  • Definitely doable, the main complicating issue I can see is the complexity of the programs you are using. What libraries do the depend on? Are these installed on the server or do you need to transfer them too?
    – Graeme
    Mar 3, 2014 at 19:13
  • The "do not leave or even install..." is a red flag to me, is there some prohibition on doing this work on the server? Why? No way to convince them it's counterproductive?
    – vonbrand
    Mar 3, 2014 at 19:17
  • Perhaps something like rsync(1) cuts down the amount of data to be shipped from server to client enough?
    – vonbrand
    Mar 3, 2014 at 19:18
  • If the binaries etc have to be removed every time, tar will be faster. Probably easiest to keep a tar ball of everything you need and just transfer over as required.
    – Graeme
    Mar 3, 2014 at 20:29

1 Answer 1

3

To run your program on the server, you will need to have the program installed there. You don't need administrator privileges to do this but you will need remote shell access to actually run the program. Your 4th requirement to not leave or install software on the server makes your proposition impossible. If you want to do processing on the server, you will need to have your code running there.

You have 2 methods to get your program onto the server

  1. Copy your source to the server and compile on the server (easier)
  2. Cross-compile for the server on your home machine and copy the resulting binaries over (harder)

You mention that you cannot install software on the server and that "you have tried", but what exactly did you try? It is commonplace that the only place you can write will be your $HOME and that default installation options to /usr or /usr/local/ or /opt will fail, but this is not a problem. You can install whatever programs you need into your $HOME e.g. $HOME/bin and then modify your shell initialization (e.g. .bashrc) to add the neccessary paths to $PATH and $LD_LIBRARY_PATH so that you can execute program installed there.

As long as you do not have a disk quota (or it is large enough) you should not have a problem installing your entire analysis workflow into your home directory and running the code remotely.

6
  • 'Your 4th requirement to not leave or install software on the server makes your proposition impossible.' - how so? The software can always be removed after running. In fact there will likely be no problem with deleting binaries while the software is running, once the process(es) finish, it will be gone from memory too.
    – Graeme
    Mar 3, 2014 at 20:37
  • 1
    @Graeme before you can delete the software, you must install it, and he emphasized the "no installs" part of the requirement in his post.
    – casey
    Mar 3, 2014 at 20:40
  • ???? The programs are only files, they can be run and deleted locally.
    – Graeme
    Mar 3, 2014 at 21:00
  • 1
    @Graeme I'm not the one who stipulated the files could not be installed on the server. Take it up with the OP. As he worded it, it is unfeasible if you cannot install (e.g. to copy) files to the server. I am merely pointing out that to run a program on the server, it must reside (e.g. be installed) there. Whether you can delete them later is no relevant to whether they can be installed in the first place. I do however agree that being able to delete them does mitigate the other half of the requirement that they cannot be left there.
    – casey
    Mar 3, 2014 at 21:02
  • Ok, I see the confusion now. I don't think the OP expects to be able to run the programs without transferring them over (plus the first point of the question says 'transfer a binary'). In most cases 'install' means putting the binaries somewhere that is in the PATH defined by the system (eg /usr/bin). I wouldn't call dumping the files in $HOME or /tmp before running an install.
    – Graeme
    Mar 3, 2014 at 21:12

Not the answer you're looking for? Browse other questions tagged or ask your own question.