19

When I grep for a java process I get below output but it's limited to 4096 characters which results to actual process name(which is kafka.Kafka) not shown in grep output.

Is this a limitation of grep? Is there any way to print characters beyond 4096 limit?

ps -ef | grep java

java -Xmx6G -Xms6G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true -Xloggc:/x/kafka/data01/kafka-app-logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/x/kafka/data01/kafka-app-logs -Dlog4j.configuration=file:./../config/log4j.properties -cp :/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/aopalliance-repackaged-2.4.0-b34.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/argparse4j-0.5.0.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/connect-api-0.10.1.1.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/connect-file-0.10.1.1.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/connect-json-0.10.1.1.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/connect-runtime-0.10.1.1.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/guava-18.0.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/hk2-api-2.4.0-b34.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/hk2-locator-2.4.0-b34.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/hk2-utils-2.4.0-b34.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jackson-annotations-2.6.0.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jackson-core-2.6.3.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jackson-databind-2.6.3.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jackson-jaxrs-base-2.6.3.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jackson-jaxrs-json-provider-2.6.3.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jackson-module-jaxb-annotations-2.6.3.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/javassist-3.18.2-GA.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/javax.annotation-api-1.2.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/javax.inject-1.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/javax.inject-2.4.0-b34.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/javax.servlet-api-3.1.0.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/javax.ws.rs-api-2.0.1.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jersey-client-2.22.2.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jersey-common-2.22.2.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jersey-container-servlet-2.22.2.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jersey-container-servlet-core-2.22.2.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jersey-guava-2.22.2.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jersey-media-jaxb-2.22.2.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jersey-server-2.22.2.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jetty-continuation-9.2.15.v20160210.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jetty-http-9.2.15.v20160210.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jetty-io-9.2.15.v20160210.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jetty-security-9.2.15.v20160210.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jetty-server-9.2.15.v20160210.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jetty-servlet-9.2.15.v20160210.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jetty-servlets-9.2.15.v20160210.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jetty-util-9.2.15.v20160210.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/jopt-simple-4.9.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/kafka_2.10-0.10.1.1.jar:/x/home/bmcuser/kafka-paypal/kafka_2.10-0.10.1.1/bin/../libs/kafka_2.10-0.10.1.1-so

  • can you refer this post. unix.stackexchange.com/questions/101681/… – Kamaraj Feb 8 '17 at 3:36
  • That didn't solve the issue. I guess the limitation here is 4096 bytes. – zer0Id0l Feb 8 '17 at 3:41
  • 3
    It looks like probably you could make your classpath a lot shorter by using wildcard syntax, see stackoverflow.com/questions/219585/…, or you can definitely move it (entirely) off the commandline using env var CLASSPATH; either of those has a good chance of making the commandline short enough to see your classname. Or for Java processes only, if your UID has access and you have or get JDK, jps shows classname regardless of commandline truncation (at least on Linux). – dave_thompson_085 Feb 8 '17 at 10:11
  • 1
    @slebetman I'm sure OP is aware but the script which probably generated this isn't – cat Feb 8 '17 at 16:11
  • 1
    The title implies that ps -ef prints more, but grep is what's changing it. That seems... unfounded. – Charles Duffy Feb 8 '17 at 17:01
35

This is not a limitation of grep, but of /proc/PID/cmdline (technically, a design decision, not a limitation). /proc/PID/cmdline contains the complete command line of the process, with main command and arguments separated by ASCII NUL, and the file ends in NUL too. So, grep will print the whole file content if there is a match. (ps -ef gets the content of this file as CMD).

The maximum length is hardcoded in the (Linux) kernel to the PAGE_SIZE:

static int proc_pid_cmdline(struct task_struct *task, char * buffer)
{
        int res = 0;
        unsigned int len;
        struct mm_struct *mm = get_task_mm(task);
        if (!mm)
                goto out;
        if (!mm->arg_end)
                goto out_mm;    /* Shh! No looking before we're done */

        len = mm->arg_end - mm->arg_start;

        if (len > PAGE_SIZE)
                len = PAGE_SIZE;

hence 4096 bytes for such a system:

% getconf PAGE_SIZE
4096

Also, if you have multibyte character(s), the number of characters would be less than 4096, as you can imagine.

  • 7
    @zer0Id0l You can't change PAGE_SIZE: it's a property of the hardware. You'd have to design some new hardware with a larger page size, or (vastly easier…) remove the limitation that the content of cmdline is truncated to one page by implementing the proper memory mapping or copying. – Gilles Feb 8 '17 at 12:33
  • 4
    FWIW, when Giles say it's a property of hardware he means it's the property of your CPU's MMU. So you need to use a CPU with MMU of page size larger than 4k or design an architecture (and build a motherboard+BIOS for it) that use external MMU with page size larger than 4k (remember, in the past MMU used to be an external chip) or design your own CPU (plus compiler etc). The other option is to modify the linux kernel so that proc_pid_cmdline does not use PAGE_SIZE – slebetman Feb 8 '17 at 13:59
  • This design limitation doesn't need to exist. Feel free to write another proc_pid_cmdline that doesn't have it. – Joshua Feb 8 '17 at 20:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.