Argument list too long

From Helpful
Jump to: navigation, search

This error means that the shell-expanded argument list is too long for a chunk of kernel memory reserved for (MAX_ARG_PAGES, usually something like 128KB, and specifically for the environment + command line and probably some other details), hard-coded in the kernel.


This usually comes from having a * somewhere in your command, because most shells expand shell globs before it executes a command, so e.g.
cp * backup/
might be a very long command.

You can argue it's a design flaw, or that it's a nice safety against a self-DoS. Short version is that it's a fact of shell life.


There are various workable solutions:

  • if you meant 'everything in a directory', then you can often specify the directory, and to use recursion
  • if you're being selective, then
    find
    may be useful, and it allows doing things streaming-style, e.g.
find . -name '*.txt' -print0 | xargs -0 echo
(See also find and xargs)
  • Recompiling the kernel with a larger MAX_ARG_PAGES - of course, you dont' know how much you'll need, and this memory is permanently inaccessible for anything else so just throwing somehing huge at is is not ideal


Note

  • that most of these split the set of files into smaller sets, and execute something for each of these sets. : In some cases this significantly alters what the overall command does.
You may want to think about it, and read up on xargs, and its --replace.
  • for filename in `ls`; do echo $filename; done
    is not a solution, nor is it at all safe against special characters.
ls | while read filename ; do echo $filename; done
(specifically for bourne-type shells) works better, but I find it harder to remember why exactly so use find+xargs.