3

I work at a company where third party contributors transfer files, typically by FTP, to our site.

Depending on which account the file is uploaded to, some scripts are triggered and the file is processed.

Usually the file is transferred via FTP to another machine, that does a specific job on the file, and then we monitor a "finished" folder on the remote machine and transfer it back.

The final stage of the job is to transfer the file to its long-term storage server.

Today we have solved this with Windows and a software called RoboTask. It has done the work okay for now, but it is not very user friendly and hides a lot of the logic.

I am looking for something that is:

  • Maintainable for many workflows
  • Modular, so that common jobs can be reused
  • Visual, so that our less tech savvy employees can understand what is going on

Any suggestions to this problem?

I tried asking this question to Superuser, but was told it was off topic since I am asking for a software recommendation.

If this is also the case in Unix & Linux, I am sorry, but could you tell me where it is appropriate to ask the question?

Improve this question
1
  • Have you considered using a networked revision control system. Such as mercurial (hg), git, or svn. It will involve co-operation of everybody. But is much better sorted for the job.
    – ctrl-alt-delor
    Jun 16, 2014 at 13:43

1 Answer 1

Reset to default
1

First off, upload by FTP isn't a good idea (FTP was invented before miscreants took over the Internet, so it's security provisions are next to non-existent). Provide a webpage (properly secured with TLS) for upload perhaps? I know little of web stuff, but I'm sure it's easy to trigger the "file arrived" logic there.

If the return isn't time-critical, I'd just stuff the file locally to the machine that is to munge it. As this is local, FTP or whatever should be fine, or use paswordless SSH (scp(1)). One problem is the target file names, but I gather you have that one solved already. Or use filename-yyyymmddhhmmss. On the munging machine use a cron job to process files regularly, and pick them up (by a cron job, or push them out from the munger) at leisure.

Need to run some cleanup jobs regularly to delete leftover files, alert of corrupted files, ..., which is also a job for cron.

[BTW, I'm very disapointed by the PTB, this is clearly a question for superuser, but I'm just a lowly participant on foot here.]

Improve this answer
1
  • Thanks for the answer. I should have been more explicit. We are using FTPS for the transfers. I am now writing my own workflow with Bash and Python, but I was hoping there was an almost "drag-and-drop" solution already existing.
    – mattis
    Jan 22, 2013 at 11:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .