discusion related to cluster mecanism.

learn more… | top users | synonyms

33
votes
1answer
2k views

How do I distribute a large download over multiple computers?

I need to download a large file (1GB). I also have access to multiple computers running Linux, but each is limited to a 50kB/s download speed by an admin policy. How do I distribute downloading this ...
21
votes
1answer
26k views

How does tcp-keepalive work in ssh?

I am trying to code a shell-script that uses a ssh-connection for doing "heartbeats". I want to terminate the client- and server-side of that connection after a certain timeout (after the connection ...
9
votes
2answers
1k views

How to build a personnal cluster?

I have a debian server running some services (mail, web, git, mysql...). But it is alone, meaning that if it breaks, I have to repair it to access my services. I'd like to put a second server so that ...
9
votes
5answers
503 views

Directory sync with automatic peer discovery

I'm working with a cluster of Linux servers sitting behind the Amazon cloud autoscaling feature (documentation). There are files that accumulate on each server, they are cache files for the ...
8
votes
3answers
2k views

How to set up “poor man's” Linux desktop cluster?

There are several small Linux/Android devices on the market with 1 GHz ARM processor, 512 MByte RAM and cca. 4 Gbyte storage for $35-$40. If I buy 5 pcs, it's only around $200. Let's say, we have ...
7
votes
2answers
328 views

Straightforward Linux Clustering

We have many unused PC machines and we would like to use them to set up educational lab for high performance computing applications. Which Linux distribution is the most convenient to set up and easy ...
7
votes
2answers
2k views

Small Distributed Computing Cluster

I'm a high school student trying to build a linux cluster for a project (I have a bunch of decent computers slated for re-image this summer, so the tech department basically says as long as I don't ...
6
votes
3answers
404 views

Multiple tar processes writing to the same archive file at once

I am running many tasks on a Linux cluster. Each task creates many output files. When all tasks are finished, I run something like tar cf foo.tar output_files/to create a tar archive. This is a very ...
6
votes
1answer
595 views

Running parallel bash jobs on a HPC cluster using GNU parallel

On an HPC cluster I am trying to run multiple bash scripts (permute2.sh) from 1 bash script using GNU parallel, however it doesn't complete every job. It randomly completes one job, while it is stuck ...
4
votes
1answer
205 views

Multi-machine tool in the spirit of moreutils' `parallel`?

parallel from moreutils is a great tool for, among other things, distributing m independent tasks evenly over n CPUs. Does anybody know of a tool that accomplishes the same thing for multiple ...
4
votes
3answers
2k views

On Ubuntu Linux, is it normal for mandb to run continuously (apparently in the background)?

I have access to a Ubuntu Linux node at my institution. The nodes are shared among the group, but typically I am the only person who uses this particular node. I am running a calculation in parallel ...
4
votes
2answers
282 views

Setup linux cluster like Windows with CentOS or Ubuntu? [closed]

Along the lines of this MSDN article, how would you setup a pair of Ubuntu or CentOS (or RHEL) servers in a cluster so they appear to hosted applications as a single server, but will continue to work ...
3
votes
1answer
40 views

NFS file with same name but different content depending on host

I have cluster using NFS with a program myprogram which writes to a fixed location, e.g. /home/sharedfs/somedir/somefile.txt. This means that if you run myprogram from each node, they all write to the ...
3
votes
1answer
70 views

Configuration management, shared files, etc with a Laptop

I have a couple computers sitting around in my home network. They all run the same version of Ubuntu, and I would like to unify their configuration to some extent. At my university, you can log in to ...
3
votes
1answer
221 views

Fencing VM nodes

I'm using VMware Workstation as my hypervisor. I want to create a cluster with two nodes and a sharing storage with FreeNAS. My questions are around fencing. Is it possible to do it in this case? ...
3
votes
1answer
321 views

How does ZFS grow/scale?

I am wondering how would one design a ZFS system to build something that is massively scalable, along the lines of your favorite cloud storage, but with all the neat benefits the come with ZFS. A ...
3
votes
1answer
759 views

Recommendations for replacing a GFS cluster?

I have a couple of CentOS GFS-clusters (GFS as in Global File System) using a shared disk in a Fibre Channel SAN. They are mature now, and the time has come to start planning for their replacement. ...
3
votes
0answers
314 views

Corosync/Pacemaker cluster loss of synchronization

I'm facing a problem with the corosync and pacemaker cluster software. I've two virtual nodes (Scientific Linux 6.4) and corosync/pacemaker installed on them: corosync-1.4.1-17.el6.x86_64 ...
3
votes
0answers
227 views

Should I investigate Rocks or Ubuntu/Mosix for building a cluster?

I work at a college doing system administration. Someone in their lab is looking to build a cluster for running complex Octave and C++ projects. They have brand new modern machines. My question is ...
2
votes
1answer
69 views

SED and PDSH: adding a line to a file on a slave node in a cluster

I am trying to add a new line containing GATEWAY=10.0.10.1 to the /etc/sysconfig/network file on each of 32 slave nodes in a cluster. I read this post on how to insert a line with sed. This works for ...
2
votes
1answer
3k views

usr/bin/xterm Xt error: Can't open display: /usr/bin/xterm: DISPLAY is not set?

I'm trying to submit a job to a school server (HPC) with: #!/bin/bash #$ -S /bin/bash #$ -cwd #$ -o ./out_$JOB_ID.txt #$ -e ./err_$JOB_ID.txt #$ -notify #$ -pe orte 1 date pwd ...
2
votes
2answers
731 views

comm fails on bash variable input

I have a script that's supposed to get the list of files of two directories, get differences and execute some code for certain files. These are the commands to get the file lists: list_in=$(find ...
2
votes
1answer
42 views

How much space for /home on a hadoop cluster? [closed]

What is a reasonable size to provide for a /home partition for 100 users on a hadoop cluster? Assume that a landing zone has been provided to store files/data for ingestion into the cluster, so the ...
2
votes
2answers
709 views

How can I mount vxfs FS to two or more Solaris servers?

I want to have two Solaris servers share the same SAN vxfs Filesystem. Though at a time just one would be accessing the share. This is to allow for a quick failover in case the primary server ...
2
votes
1answer
6k views

How do I download and launch a JNLP file in CentOS?

I'm using Amazon's GPU cluster to experiment with some HPC java code on CentOS. The Java application is in JNLP format and has a GUI. I've followed Amazon's getting started video, but I'm not ...
2
votes
2answers
46 views

Gather job usage information on cluster

I am running simulations on a linux cluster shared by hundreds of other users. There is a job scheduler installed on the cluster that determines who runs where, and one of the scheduler commands ...
2
votes
0answers
62 views

Takeover delay in Heartbeat HA cluster

I have set up a heartbeat HA two node cluster. And both of server have same configuration of ha.cf #cat /etc/ha.cf logfile /var/log/ha-log autojoin none bcast eth1 warntime 20ms deadtime 40ms ...
2
votes
1answer
63 views

CentOS VM 3972 Segmentation Fault

I'm trying to install MATLAB 2012 on CentOS 5.8 running on a VM cluster. I was able to mount the .iso file, but when I run ./install, it was unable to write to /tmp giving the error, "no space on ...
2
votes
0answers
21 views

heartbeat stops after reviving primary node

I followed these steps correctly, everything worked well before I have revived heartbeat on node01 "at the last steps of the tutorial" when I hit /etc/init.d/heartbeat start it started correctly but ...
2
votes
1answer
121 views

mpd daemon prematurely ending jobs

I am a little out of my depth here so bear with me. I am trying to configure mpirun and mpiexec to run software called Materials Studio on a 1 node, 2 processor, 12 core cluster. The submission scheme ...
1
vote
4answers
2k views

How to find which machines are idle on a cluster of many nodes

I run measurements on a cluster consisting of 32 nodes/machines. I do not require all nodes, only 4, for example. The problem is most of the time, the nodes are busy with other people running their ...
1
vote
1answer
536 views

what is the difference between qsub and ./

Can anyone tell me the difference between the following ways of submitting a script: $ qsub script_name.sh and ./script_name.sh What are the differences between the above two ways of submitting ...
1
vote
1answer
46 views

Will having dozens of inactive logins hurt performance

I work on a cluster shared with ~20 users, many of them operating remotely. It's quite common to open the login node, type users and see ~100 users listed, mostly the same user logged in multiple ...
1
vote
3answers
5k views

NFS Automatic Fail-over or Load-balanced or Clustering?

I've been using NFS (and it is the only one i know well in Network Files) for a few years. But still can not able to find out .. How to make NFS Servers in Clustered Design (or Load-balanced?) Or at ...
1
vote
2answers
1k views

Clustering Servers to run one vm

I am looking at clustering about five machines and I wanted to get some opinions about what is the best way to go about it. I'm wanting to run one VM over the five machines, firstly I want to ask, 1: ...
1
vote
1answer
2k views

EMC storage + powercf command + powerpath

When I run powercf -q on my Solaris machine, what changes does this command do on the EMC storage, or on Solaris file system? From manual page: DESCRIPTION During system boot on Solaris hosts, ...
1
vote
1answer
24 views

Is there a way to achieve context-dependent path names (CDPN) on NFS?

In my GFS clusters I use the CDPN feature to have separate chrooted /dev/log directories on separate cluster nodes: /home/ftpuser/foo: lrwxrwxrwx 1 root root 18 Sep 26 2010 dev -> ...
1
vote
2answers
128 views

can I tell to useradd to execute a script? Maybe in /etc/default/useradd?

There are plenty of people that already answered to the question "how to execute useradd properly in a script?" My question is: "is it possible to tell to useradd to execute one script when it is ...
1
vote
1answer
38 views

Oracle 11gR2 grid silent installation: error for privateinterconnects

I am installing Oracle grid 11gR2 using silent install with response file. I am getting error after "Collecting interface information for node 1" in logs. I have a feeling that this could be due to ...
1
vote
1answer
202 views

HAProxy should not forward an incoming request to down instance

In my scenario, I have 2 servers, both are communicating with HAProxy. When a request comes in, HAProxy splits the request and sends it to both servers, following a round robin method. If in any case ...
1
vote
0answers
20 views

pacemaker hangs while stopping

I have recently installed pacemaker and corosync for managin a virtual IP. The thing is that when I want to stop a resource (Virtual IP) on all the nodes, the stop command hangs. [root@isis ~]# sudo ...
1
vote
0answers
13 views

How to replicate KVM for DR purpose?

I am looking to replicate multiple KVM servers from one data center to another data center. What steps with specific commands to be used to replicate one server so I can use that for others. Thanks ...
1
vote
1answer
32 views

Cluster Lock Manager without TCP/IP

I have 2 linux sharing same SAN disk LUN. There are many cluster files systems to manage lock/unlock. But all of them are using a seperate TCP/IP comminication. But I need a cluster lock manager ...
1
vote
0answers
37 views

About mem and vmem

I am working with a cluster machine running under linux. I have a shell script that uses mpirun to submit my jobs to the cluster machine. In that same script, I can choose the number of nodes that ...
1
vote
0answers
36 views

HA Cluster in Linux [closed]

My question is : Is there any other solution for High Availability Clustering in Linux like RedHat Cluster Suite, or not ? Thanks..
1
vote
1answer
134 views

Strategies for building distributed computing system with hybrid Mac and Linux systems

I want to build a distributed computing system to run Matlab, C and other programming languages for scientific computing. Now I've several old Mac machines with Lion Mac OS installed acted as web ...
1
vote
0answers
158 views

Getting from proficient to expert [closed]

I've got my vacation coming in and thought I might use that for something useful. Essentially, I've been a Linux user since around 1995 when I installed Slackware 2.0 on my computer and have been ...
0
votes
1answer
108 views

Running shell script on cluster in parralel

I thought dsh(Distributed Shel) will be a good option but when i ran my shell script across my nodes, I didnt get expected output, dsh -aM -c bash /home/cloudera/bash_script.sh server2@x.x.x.110: ...
0
votes
1answer
620 views

Script for running sequential job

I started recently to learn using Linux. I want to write a script in bash shell to run a serial job on a cluster. I have been searching for hints and instructions on how to write such a script. I ...
0
votes
3answers
1k views

Kill nohup command on cluster

Last night I launched a command via nohup on a cluster and then closed the connection. Today I would like to kill this job (I know it's still running because output is being generated in an output ...