2

I have a program (pre-compiled binary) that, when started manually on linux, runs fine. It has a CLI that I use to enter commands. This program is a software router, I can connect to it's CLI to adjust it's settings.

However if I want to start it up at boot time the program goes very slowly, unusably so. If there is information being printed to the screen for example, it will be printed out in chunks, as if there was a slow baudrate, that kind of thing. Every command I enter is like this, information is printed very slowly.

What could be happening, what is different from me running this manually after boot rather than from init.d with S99 or something? If I stop the process and relaunch it manually it works fine.

This python script is what is called from init.d:

import subprocess
application_pidfile = "/var/run/application.pid"
command_line=["start-stop-daemon", "-q", "-p", application_pidfile, "-S",
              "-m", "-b", "-x", "/bin/application"]
subprocess.call(command_line)

What I wonder is why would it still be going slow an hour after boot? Yet if I just stop it and start it manually it will run fine even a few seconds after boot? Even if I set a delay, such as start this process a few minutes after boot the same issue occurs.

Could it be starting in some slow "socket" mode or something?

  • 1
    You start it from init, yet it produces output to the screen? Processes started via init should be daemons and produce no output. Can you explain more about where the output goes, when, and why? And please include your init script. – Mikel Apr 1 '14 at 14:05
  • Obviously this is not the case for other init process, and since it is also not the case in general for others, you need to provide a reproducible example if you want an answer. @Mikel Note he used that "for example" -- even if the process does print to the screen, it should not be slowed down particular except to the extent that it is interleaved with other processes. – goldilocks Apr 1 '14 at 14:10
  • @TAFKA Sure. But how it prints to the screen could be an important piece of the puzzle. Does it print to /dev/console? Directly or via syslog for example? What does it try to print and when? We don't have any information. – Mikel Apr 1 '14 at 14:17
  • @Mikel The line in init.d calls the python script I have edited above. This runs in the background, I can connect to the application when it is running. The application is a software router. I connect to it's console. Here is where all the slow output is observed – Paul Apr 1 '14 at 14:20
  • 'If there is information being printed to the screen for example, it will be printed out in chunks.' This sounds like the normal buffering you get when a program doesn't output to a terminal. Perhaps you are just looking for the program to be line buffered so that each line is output immediately (or completely unbuffered). Maybe the reason you think it is slow is just because the output is left in buffer for a while. How is it that you see output when the program is started by an init script? – Graeme Apr 1 '14 at 14:55
1

Modern init systems such as systemd and upstart run multiple threads of execution, and even with the original init, things may be forked during the boot process (you've already said yours does, for example).

This means your process is running at the same time as other processes, and any output will be interleaved with their's:

   Process 1 says "I'm here"
   Process 2 says "I'm here"
   Process 1 says "Doing my thing..."
   Process 3 says "I'm here"
   Process 3 says "Doing my thing..."
   Process 1 says "Still busy..."
   Process 2 says "Doing my thing..."

Note the order here is random beyond start priority; an S05 will begin before a S70 process, but when they finish in relation to one another is indeterminate unless one completing is a specific prerequisite of the other.

If you've ever done any threaded programming, you'll be familiar with this horse race. Because the processes are running simultaneously, they do not interleave in an orderly fashion -- hence in my example, Process 3 gets to do several things before processes that started earlier get to do the equivalent. Etc. This is just happenstance and will not be the same every time.

Boot is a busy time for the system -- many things are happening actively at once. So if you compare the execution time to running the process alone subsequently, it will be slower, because in the latter case it does not have to compete with many (or, in significant terms, any) other active processes.

  • Thanks for the answer! I realise boot should slow things down all right, and that other things have to be done. I've edited in more information above, I'm starting a software router with start-stop-daemon from inid.d. This will run continuously -I can attach and detach from it to enter commands, so what I wonder is why would it still be going slow an hour after boot? Yet if I just stop it and start it manually it will run fine even a few seconds after boot? Even if I set a delay, such as start this process a few minutes after boot the same issue occurs. – Paul Apr 1 '14 at 14:27
  • 1
    That I can't say. You should add those details to the question too. I've added the python tag in case this is relevant (but I'm not a python user, so don't have any thoughts about that). – goldilocks Apr 1 '14 at 14:30
  • Okay. I'm gonna leave this answer up for posterity. – goldilocks Apr 1 '14 at 14:42
  • Good idea, it's informative. – Paul Apr 1 '14 at 15:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.