discusion related to cluster mecanism.

learn more… | top users | synonyms

35
votes
1answer
2k views

How do I distribute a large download over multiple computers?

I need to download a large file (1GB). I also have access to multiple computers running Linux, but each is limited to a 50kB/s download speed by an admin policy. How do I distribute downloading this ...
32
votes
1answer
35k views

How does tcp-keepalive work in ssh?

I am trying to code a shell-script that uses a ssh-connection for doing "heartbeats". I want to terminate the client- and server-side of that connection after a certain timeout (after the connection ...
9
votes
2answers
1k views

How to build a personnal cluster?

I have a debian server running some services (mail, web, git, mysql...). But it is alone, meaning that if it breaks, I have to repair it to access my services. I'd like to put a second server so that ...
9
votes
3answers
3k views

How to set up “poor man's” Linux desktop cluster?

There are several small Linux/Android devices on the market with 1 GHz ARM processor, 512 MByte RAM and cca. 4 Gbyte storage for $35-$40. If I buy 5 pcs, it's only around $200. Let's say, we have ...
9
votes
5answers
525 views

Directory sync with automatic peer discovery

I'm working with a cluster of Linux servers sitting behind the Amazon cloud autoscaling feature (documentation). There are files that accumulate on each server, they are cache files for the ...
7
votes
2answers
349 views

Straightforward Linux Clustering

We have many unused PC machines and we would like to use them to set up educational lab for high performance computing applications. Which Linux distribution is the most convenient to set up and easy ...
7
votes
2answers
2k views

Small Distributed Computing Cluster

I'm a high school student trying to build a linux cluster for a project (I have a bunch of decent computers slated for re-image this summer, so the tech department basically says as long as I don't ...
6
votes
3answers
594 views

Multiple tar processes writing to the same archive file at once

I am running many tasks on a Linux cluster. Each task creates many output files. When all tasks are finished, I run something like tar cf foo.tar output_files/to create a tar archive. This is a very ...
6
votes
1answer
723 views

Running parallel bash jobs on a HPC cluster using GNU parallel

On an HPC cluster I am trying to run multiple bash scripts (permute2.sh) from 1 bash script using GNU parallel, however it doesn't complete every job. It randomly completes one job, while it is stuck ...
4
votes
1answer
231 views

Multi-machine tool in the spirit of moreutils' `parallel`?

parallel from moreutils is a great tool for, among other things, distributing m independent tasks evenly over n CPUs. Does anybody know of a tool that accomplishes the same thing for multiple ...
4
votes
3answers
3k views

On Ubuntu Linux, is it normal for mandb to run continuously (apparently in the background)?

I have access to a Ubuntu Linux node at my institution. The nodes are shared among the group, but typically I am the only person who uses this particular node. I am running a calculation in parallel ...
4
votes
2answers
424 views

Setup linux cluster like Windows with CentOS or Ubuntu? [closed]

Along the lines of this MSDN article, how would you setup a pair of Ubuntu or CentOS (or RHEL) servers in a cluster so they appear to hosted applications as a single server, but will continue to work ...
3
votes
1answer
54 views

NFS file with same name but different content depending on host

I have cluster using NFS with a program myprogram which writes to a fixed location, e.g. /home/sharedfs/somedir/somefile.txt. This means that if you run myprogram from each node, they all write to the ...
3
votes
1answer
8k views

How do I download and launch a JNLP file in CentOS?

I'm using Amazon's GPU cluster to experiment with some HPC java code on CentOS. The Java application is in JNLP format and has a GUI. I've followed Amazon's getting started video, but I'm not ...
3
votes
1answer
87 views

Configuration management, shared files, etc with a Laptop

I have a couple computers sitting around in my home network. They all run the same version of Ubuntu, and I would like to unify their configuration to some extent. At my university, you can log in to ...
3
votes
1answer
416 views

Fencing VM nodes

I'm using VMware Workstation as my hypervisor. I want to create a cluster with two nodes and a sharing storage with FreeNAS. My questions are around fencing. Is it possible to do it in this case? ...
3
votes
1answer
467 views

How does ZFS grow/scale?

I am wondering how would one design a ZFS system to build something that is massively scalable, along the lines of your favorite cloud storage, but with all the neat benefits the come with ZFS. A ...
3
votes
1answer
2k views

Recommendations for replacing a GFS cluster?

I have a couple of CentOS GFS-clusters (GFS as in Global File System) using a shared disk in a Fibre Channel SAN. They are mature now, and the time has come to start planning for their replacement. ...
3
votes
0answers
430 views

Corosync/Pacemaker cluster loss of synchronization

I'm facing a problem with the corosync and pacemaker cluster software. I've two virtual nodes (Scientific Linux 6.4) and corosync/pacemaker installed on them: corosync-1.4.1-17.el6.x86_64 ...
3
votes
0answers
249 views

Should I investigate Rocks or Ubuntu/Mosix for building a cluster?

I work at a college doing system administration. Someone in their lab is looking to build a cluster for running complex Octave and C++ projects. They have brand new modern machines. My question is ...
2
votes
5answers
11k views

NFS Automatic Fail-over or Load-balanced or Clustering?

I've been using NFS (and it is the only one i know well in Network Files) for a few years. But still can not able to find out .. How to make NFS Servers in Clustered Design (or Load-balanced?) Or at ...
2
votes
1answer
134 views

SED and PDSH: adding a line to a file on a slave node in a cluster

I am trying to add a new line containing GATEWAY=10.0.10.1 to the /etc/sysconfig/network file on each of 32 slave nodes in a cluster. I read this post on how to insert a line with sed. This works for ...
2
votes
1answer
5k views

usr/bin/xterm Xt error: Can't open display: /usr/bin/xterm: DISPLAY is not set?

I'm trying to submit a job to a school server (HPC) with: #!/bin/bash #$ -S /bin/bash #$ -cwd #$ -o ./out_$JOB_ID.txt #$ -e ./err_$JOB_ID.txt #$ -notify #$ -pe orte 1 date pwd ...
2
votes
2answers
929 views

comm fails on bash variable input

I have a script that's supposed to get the list of files of two directories, get differences and execute some code for certain files. These are the commands to get the file lists: list_in=$(find ...
2
votes
2answers
44 views

Code for submitting job on cluster

I don't know whether this is the right place for posting this question, if not kindly let me know where I can ask. My doubt: I use the following code to submit a job on a cluster, but I don't know ...
2
votes
2answers
66 views

mknfsmnt fails with “System call error number -1”

This system is part of a cluster and the other node mounts the nfs share successfully. This is currently the active node which means it has a secondary service ip address. I think this is related to ...
2
votes
1answer
54 views

How much space for /home on a hadoop cluster? [closed]

What is a reasonable size to provide for a /home partition for 100 users on a hadoop cluster? Assume that a landing zone has been provided to store files/data for ingestion into the cluster, so the ...
2
votes
2answers
1k views

How can I mount vxfs FS to two or more Solaris servers?

I want to have two Solaris servers share the same SAN vxfs Filesystem. Though at a time just one would be accessing the share. This is to allow for a quick failover in case the primary server ...
2
votes
1answer
76 views

Clustered NFS server reply ERR 24: Auth Bogus Credentials (seal broken)

I have 4 servers on the VirtualBox. Two of the servers are a CentOS 7 cluster with Pacemaker(corosync), and they have an NFSv4 server in Active/Passive mode. There are also 2 clients with CentOS 6, ...
2
votes
1answer
37 views

reduce bandwidth consumption from downloading yum updates

I have several machines (7) running fedora and would like to reduce the amount of external bandwidth I am consuming. I see that there were several attempts to setup mirrors, but those projects seem ...
2
votes
0answers
119 views

pacemaker hangs while stopping

I have recently installed pacemaker and corosync for managin a virtual IP. The thing is that when I want to stop a resource (Virtual IP) on all the nodes, the stop command hangs. [root@isis ~]# sudo ...
2
votes
2answers
81 views

Gather job usage information on cluster

I am running simulations on a linux cluster shared by hundreds of other users. There is a job scheduler installed on the cluster that determines who runs where, and one of the scheduler commands ...
2
votes
0answers
96 views

Takeover delay in Heartbeat HA cluster

I have set up a heartbeat HA two node cluster. And both of server have same configuration of ha.cf #cat /etc/ha.cf logfile /var/log/ha-log autojoin none bcast eth1 warntime 20ms deadtime 40ms ...
2
votes
1answer
78 views

CentOS VM 3972 Segmentation Fault

I'm trying to install MATLAB 2012 on CentOS 5.8 running on a VM cluster. I was able to mount the .iso file, but when I run ./install, it was unable to write to /tmp giving the error, "no space on ...
2
votes
0answers
31 views

heartbeat stops after reviving primary node

I followed these steps correctly, everything worked well before I have revived heartbeat on node01 "at the last steps of the tutorial" when I hit /etc/init.d/heartbeat start it started correctly but ...
2
votes
1answer
168 views

mpd daemon prematurely ending jobs

I am a little out of my depth here so bear with me. I am trying to configure mpirun and mpiexec to run software called Materials Studio on a 1 node, 2 processor, 12 core cluster. The submission scheme ...
1
vote
5answers
2k views

How to find which machines are idle on a cluster of many nodes

I run measurements on a cluster consisting of 32 nodes/machines. I do not require all nodes, only 4, for example. The problem is most of the time, the nodes are busy with other people running their ...
1
vote
1answer
1k views

what is the difference between qsub and ./

Can anyone tell me the difference between the following ways of submitting a script: $ qsub script_name.sh and ./script_name.sh What are the differences between the above two ways of submitting ...
1
vote
1answer
49 views

Will having dozens of inactive logins hurt performance

I work on a cluster shared with ~20 users, many of them operating remotely. It's quite common to open the login node, type users and see ~100 users listed, mostly the same user logged in multiple ...
1
vote
2answers
2k views

Clustering Servers to run one vm

I am looking at clustering about five machines and I wanted to get some opinions about what is the best way to go about it. I'm wanting to run one VM over the five machines, firstly I want to ask, 1: ...
1
vote
1answer
43 views

Linux run every program on cluster

I am playing around with Open MPI and I must say I love the concept of running programs on the cluster. The problem I am facing with Open MPI is that a specific program must support this way of ...
1
vote
1answer
2k views

EMC storage + powercf command + powerpath

When I run powercf -q on my Solaris machine, what changes does this command do on the EMC storage, or on Solaris file system? From manual page: DESCRIPTION During system boot on Solaris hosts, ...
1
vote
1answer
40 views

Saving power on a linux cluster

I would like to buy a cluster (30-60 nodes) for work and install linux on it. The cluster should have a queue system like PBS, for instance. It should also be capable of doing parallel computation. I ...
1
vote
1answer
59 views

Is there a way to achieve context-dependent path names (CDPN) on NFS?

In my GFS clusters I use the CDPN feature to have separate chrooted /dev/log directories on separate cluster nodes: /home/ftpuser/foo: lrwxrwxrwx 1 root root 18 Sep 26 2010 dev -> ...
1
vote
2answers
206 views

can I tell to useradd to execute a script? Maybe in /etc/default/useradd?

There are plenty of people that already answered to the question "how to execute useradd properly in a script?" My question is: "is it possible to tell to useradd to execute one script when it is ...
1
vote
1answer
49 views

Oracle 11gR2 grid silent installation: error for privateinterconnects

I am installing Oracle grid 11gR2 using silent install with response file. I am getting error after "Collecting interface information for node 1" in logs. I have a feeling that this could be due to ...
1
vote
1answer
275 views

HAProxy should not forward an incoming request to down instance

In my scenario, I have 2 servers, both are communicating with HAProxy. When a request comes in, HAProxy splits the request and sends it to both servers, following a round robin method. If in any case ...
1
vote
1answer
17 views

Trying to install a cluster, DHCP failing to start

Trying to install a linux cluster using this tutorial. I am running CentOS 6.3. When I try to start DHCP with service dhcpd start, it fails. I've used tail -f /var/log/messages and dhcpd -t, but I ...
1
vote
1answer
40 views

Running a systemd service once a day (on a CoreOS cluster)

I currently have a CoreOS cluster on AWS. The cluster runs multiples containers, mainly for Rails applications. However, one of this container is a pure Ruby app processing a lot of data from specific ...
1
vote
0answers
80 views

clvm (cman) has problems with dlm

I am trying to create a volume group which is accessible on two servers with clvm. I read about it and it looks like it is most easily to do this with cman. But after hours of testing I cannot achieve ...