subprocess research - max limit for piped output

[anatoly techtonik]

I am trying to figure out what is maximum size
for piped input in subprocess.check_output()

I've got limitation of about 500Mb after which
Python exits with MemoryError without any
additional details.

I have only 2.76Gb memory used out of 8Gb,
so what limit do I hit?

1. subprocess output read buffer
2. Python limit on size of variable
3. some OS limit on output pipes

Testcase attached.


C:\discovery\interface\subprocess>py dead.py
Testing size: 520Mb
..truncating to 545259520
..
Traceback (most recent call last):
File "dead.py", line 66, in <module>
backticks(r'type largefile')
File "dead.py", line 36, in backticks
output = subprocess.check_output(command, shell=True)
File "C:\Python27\lib\subprocess.py", line 567, in check_output
output, unused_err = process.communicate()
File "C:\Python27\lib\subprocess.py", line 791, in communicate
stdout = _eintr_retry_call(self.stdout.read)
File "C:\Python27\lib\subprocess.py", line 476, in _eintr_retry_call
return func(*args)
MemoryError
The process tried to write to a nonexistent pipe.