Argument list too long: Difference between revisions

From Helpful
Jump to navigation Jump to search
mNo edit summary
Tag: New redirect
 
Line 1: Line 1:
 
#redirect [[Some_explanation_to_some_errors_and_warnings#Argument_list_too_long]]
 
...in a linux shell, often happens when you used a * somewhere in your command.
 
 
The actual reason is a little lower level:
Shells will expand [[shell globs]] before it executes a command, so e.g. {{inlinecode|cp * /backup/}} actually might happen to expand to a long list of files.
 
Either way, it may create a very large string to be handed to the exec().
 
 
You get this error when that argument list is too long for the chunk of kernel memory reserved for passing such strings - which is hard-coded in the kernel {{comment|(MAX_ARG_PAGES, usually something like 128KB)}}.
 
You can argue it's a design flaw, or that it's a sensible guard against a self-DoS, but either way, that limit is in place.
 
 
 
There are various workable solutions:
 
* if you meant 'everything in a directory', then you can often specify the directory and a flag to use recursion
 
* if you're being selective, then {{inlinecode|find}} may be useful, and it allows doing things streaming-style, e.g.
: {{inlinecode|find . -name '*.txt' -print0 | xargs -0 echo}}  {{comment|(See also [[find and xargs]])}}
 
* Recompiling the kernel with a larger MAX_ARG_PAGES - of course, you don't know how much you'll need, and this memory is permanently inaccessible for anything else so just throwing a huge number at is is not ideal
 
 
Note
* that most of these split the set of files into smaller sets, and execute something for each of these sets. : In some cases this significantly alters what the overall command does. <!-- For example, if the command creates (as in replaces) an archive and does this a few times, most of the work will be overwritten and only the last set will be in the file you'll end up with.-->
: You may want to think about it, and read up on xargs, and its --replace.
 
* {{inlinecode|for filename in `ls`; do echo $filename; done}} is '''not''' a solution, nor is it at all safe against special characters.
: {{inlinecode|ls &#124; while read filename ; do echo $filename; done}} {{comment|(specifically for bourne-type shells)}} works better, but I find it harder to remember ''why'' exactly so use find+xargs.
 
 
 
 
 
 
[[Category:Unices]]
[[Category:Workaround]]
[[Category:Warnings and errors]]

Latest revision as of 15:12, 14 July 2023