Tag Info

Hot answers tagged

23

You probably want to use the ServerAlive settings for this. They do not require any configuration on the server, and can be set on the command line if you wish. ssh -o ServerAliveInterval=5 -o ServerAliveCountMax=1 $HOST This will send a ssh keepalive message every 5 seconds, and if it comes time to send another keepalive, but a response to the last one ...


7

Creating an ha environment has a lot of caveats and is complicated, and often times depends on the actual software (e.g. creating a master-slave environment for mysql is different than for postfix0 If you want to get started and only want to have two systems and don't have time to configure all your daemons accordingly you should have a look at drbd, raid-1 ...


5

There's the rocks linux distro which is made for clustering, and is based on CentOS/RHEL. The strong point of rocks is that it'll for the most part manage and do a lot of the minutia for you. It'll do automatic installation and reinstallation, and if your computers can boot via PXE, the initial install will consist of PXE booting your nodes. If you have a ...


5

You can't have multiple processes adding to the same tar archive (or any other usual archive format, compressed or not). Each file is stored contiguously, and there is no way to insert data in a file, only to append or overwrite, so continuing to write to a file that isn't the last one would overwrite subsequent files. If you know the file size in advance, ...


4

It isn't normal for mandb to run continuously. It is typical to run mandb once a day in a cron job, to perform maintenance task such as updating an index of installed man pages and building or trimming a cache of formatted man pages. The daily job should run in a few seconds, perhaps a few minutes if you have a lot of man pages and a slow disk. If the job ...


4

Using a compute cluster? Since you're asking about qsub I'm going to assume you're using GridEngine or some computer clustering product. qsub When you run the command qsub script_name.sh you're telling the cluster's head node that you want to submit a script to run on that server. So that server would need to have access to the same filesystem's that ...


3

You can use Redhat Cluster Suit for the same. Let's understand little bit of clustering There is a different cluster for every problem. Generally speaking though, there are two main problems that clusters try to resolve. Performance and High Availability. and As per your requirement ( will continue to work even if a node in the cluster becomes inoperable ...


3

I'm a little confused what exactly you want your cluster to do. It sounds a bit like all you want is starting programs and running them on different boxes. If that's the case, SSH/X-forwarding/NFS should do you ok. IMHO this doesn't really have anything to do with clustering, its simply remote controlling different machines. I really have to ask one thing ...


3

It really depends on what you are trying to accomplish, and what you mean by "Distributed Computing Cluster." I did a similar thing once in Uni using old machines and PVM that's the "Cluster" in the sense of a bunch of machines acting as one single computer to do parallel processing - think Beowulf clusters. Of course, you will need code that is written to ...


3

As already mentioned in the comments, since memory and CPU usage are negligible as you said, idle users won't hurt anyone. If you still want to get rid of them, you could automatically hunt down idle users and kick them off, either by e.g. a shell script you call in regular intervals via cron, or by setting ClientAliveInterval 1800 ClientAliveCountMax 0 ...


3

I suppose that when you say that myprogram writes in a fixed file, your mean that you cannot modify this program to make it write to another file (maybe you don't have the source code). I'm also pretty sure that what you want to do is not possible with NFS. Anyway the solution you mentioned using different mount points will work, but isn't very practical if ...


3

Ok try with this command line: pdsh -w n032 sed -i.bak -e '"\$aGATEWAY=10.0.10.1"' test/test1; With ssh it works. If it works locally (on the nodes) the simple command, it should work this line too. When you try without quote ', the shell locally (on the current shell on entry computer) will expand and transform for the node the line in: sed -i.bak -e ...


2

We have a small cluster that has openSUSE as its base distro, but I do not think it is too important. Ubuntu looks like a viable alternative and has quite a bit of documentation and community support. On top of linux, we run Sun Grid Engine (and our cluster even includes Mac OS machines pretty seamlessly), but slurm would probably work for a simple setup. ...


2

What you're talking about is called a Single System Image cluster, or sometimes a distributed shared memory system (in a more limiting context). There are some projects listed on the linked Wikipedia pages you should look into. I've used the SGI Altix cluster (the NUMAlink ones), and it can be quite powerful if you have a process that requires a huge ...


2

While @wnoise's answer is a the nicer solution, it might not be possible for you to implement it (i.e. do you administer the cluster?)... so, why not have a look at one of the 'cluster SSH' solutions @Chaleb mentioned here (pssh, pdsh, clusterssh, clusterit) or Fabric (also mentioned in this thread, by @Crankyadmin) to gather usage statistics. Add a ...


2

There are many "batch systems" that are designed to handle this sort of problem. One specifically tailored to handle "cycle stealing" from otherwise unoccupied systems is condor, a long running research project at the University of Wisconsin.


2

The rup command from the rstatclient package will poll all the machines on your subnet for information, including their load averages. The machines must be running rstatd to serve up that information, and I would tcpwrapper it to only respond to your admin desktops. You can also specify individual machines to collect data from. With rstatd running on the ...


2

It doesn't make any changes on the EMC storage. It only scans the fibre channel bus to see what world-wide names are visible through each local port. If it sees the same WWN through multiple ports, then it sets that storage device up for multipath I/O. Changes are made to emcp.conf in /kernel/drv, and those changes in turn effect how device paths are ...


2

The kernel recognizes certain file formats that it can execute natively. This includes at least one binary format. Additionally, files that begin with #! (shebang) are considered scripts; for example, if a file is located at /path/to/script and begins with #!/bin/bash then the kernel executes /bin/bash /path/to/script arg1 arg2 when you invoke ...


2

ZFS is not a clustered file system, so what you talk about wouldn't actually work as described. In order to increase the capacity of a server's storage, you'd need to add more disks to the host, either in the chassis, or, if that's full, in a storage array. A host running ZFS can have up to 2^64 zpools, and each zpool can contain up to 2^64 vdevs, so you are ...


2

An approach that works fairly well for me... Connect one of those obsolete monitors you have lying around "just in case" to each of the small computers (RaspberryPi, etc.). Run a tiny, fast, RAM-based O/S like Puppy Linux (see how it works) on every computer. Setup passwordless (pre-shared password distribution) SSH between all computers. Install KVM ...


2

I believe you can do this using VMware, at least it appear to be the case according to this article titled: Fence Device and Agent Information for Red Hat Enterprise Linux. As to setting it up I did find these 2 resources which are too long to include in an answer here so I'm only going to reference them. They come from a ProxMox tutorial (another ...


2

I'm assuming that you're using GridEngine as the clustering software when you submit this script to run. Something like this: $ qsub myscript.sh You can include environment variables to qsub that you want the resulting shells that get spawned on the HPC cluster nodes like so: $ qsub -v DISPLAY=$(hostname):0.0 myscript.sh This should "inject" the ...


2

You can start the creation of the final tar file before all output files are created: Maybe that achieves the speed up you want. You can call tar this way: tar -cf foo.tar -T file-list file-list would be a FIFO. You need a script which detects new files in the source directory (inotifywatch) when each of these new files is finished (fuser) If a file ...


2

I don't have an answer to all your questions but I have some thoughts on some of them; perhaps this will allow you to narrow it down and get more specific. the same user accounts (and passwords, if that makes sense) In your context the easiest way to do that would be to just create user accounts with the same password. While there may be tools to sync ...


1

You don't specifically say which clustering software you're using, but based on the fact you're asking about qsub, I know that both GridEngine (and derivatives) along with PBS use that particular command, so let's start with those. I'm most familiar with GridEngine (and derivatives) so to submit a command using that package you'd do something like this. ...


1

You seem to be confusing what Solaris Cluster does vs. Oracle RAC. In Solaris Cluster (SunCluster) any application running is actually running on a single node in the cluster rather then on multiple ones. Which means that Veritas Volume Manager imports the associated Volume Group only on one of the nodes in the cluster. So there is no issue with it. If ...


1

It was a configuration mistake. There needs to be subnet and not ip address or subnet mask in interconnects


1

I believe this tutorial is exactly what you're looking for. The tutorial is titled: Setting Up A High-Availability Load Balancer (With Failover and Session Support) With HAProxy/Heartbeat On Debian Lenny. In general you're /etc/haproxy.cfg file will look similar to this: global log 127.0.0.1 local0 log 127.0.0.1 local1 notice ...


1

Heartbeat is deprecated . Use Pacemaker + Corosync for 1+1 , N+1, N+N clusters



Only top voted, non community-wiki answers of a minimum length are eligible