Redirecting process output in .NET

In .NET, you can use the System.Diagnostics.Process APIs to start a process and get it's output. It seems fairly simple, but there's a few interesting caveats beneath those APIs.

In this post I'll be looking at the simple case of starting a process, waiting for it to finish, and reading the contents of stdout.

To set the stage, here's the source code of the process we'll start:

using System;

namespace TextPrinter
{
	class Program
	{
		static void Main(string[] args)
		{
			var value = int.Parse(args[0]);

			for (var i = 0; i < value; i++)
			{
				Console.Out.Write('A');
			}
		}
	}
}

This Little Program^[1] accepts a single numeric command-line parameter and will print an A N times. For example, invoking TextPrinter.exe 4 will print AAAA.

Here's some code that will similarly accept a single command-line parameter, invoke the program above, and print out how many characters that program printed.

using System;
using System.Diagnostics;

namespace ConsoleRedirectExample
{
	class Program
	{
		static void Main(string[] args)
		{
			var count = int.Parse(args[0]);

			var psi = new ProcessStartInfo
			{
				FileName = "TextPrinter.exe",
				Arguments = count.ToString(),
				RedirectStandardOutput = true,
				UseShellExecute = false
			};

			using (var process = Process.Start(psi))
			{
				process.WaitForExit();

				var outputCount = process.StandardOutput.ReadToEnd().Length;
				Console.WriteLine("Got {0} characters of output.", outputCount);
			}
		}
	}
}

As with the example above, invoking ConsoleRedirectExample.exe 4 will print Got 4 characters of output..

Let's try a bigger number. What happens when we invoke ConsoleRedirectExample.exe 10000? Go on, try it.

You could be forgiven for thinking that with a bigger number it just takes a long time, but what actually happens here is that TextWriter.exe hangs entirely.

If we attach a debugger and pause execution, we get the following stack trace:

[Managed to Native Transition]
mscorlib.dll!System.IO.__ConsoleStream.WriteFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle hFile, byte[] bytes, int offset, int count, bool useFileAPIs) + 0x5a bytes
mscorlib.dll!System.IO.__ConsoleStream.Write(byte[] buffer, int offset, int count) + 0x4cbytes
mscorlib.dll!System.IO.StreamWriter.Flush(bool flushStream, bool flushEncoder) + 0x5e bytes
mscorlib.dll!System.IO.StreamWriter.Write(char value) + 0x5e bytes
mscorlib.dll!System.IO.TextWriter.SyncTextWriter.Write(char value) + 0x1e bytes
> TextPrinter.exe!TextPrinter.Program.Main(string[] args) Line 12 + 0x18 bytes	C#

What's actually happening here^[2] is that TextWriter.exe is writing to a pipe with a 4KB^[3] buffer size. When this fills, the console output stream waits until there's space to write the rest of the data.

At the same time, the parent process is waiting for the child to exit before it reads a single byte. This is a classic deadlock case - two processes are waiting for each-other to do something in order to continue.

We can make the parent process smarter, by performing the read operation asynchronously, like so:

using (var process = Process.Start(psi))
{
	var standardOutputReadTask = process.StandardOutput.ReadToEndAsync();

	process.WaitForExit();

	var outputCount = standardOutputReadTask.Result.Length;
	Console.WriteLine("Got {0} characters of output.", outputCount);
}

This will use .NET's asynchronous read API to read in the background as data comes in. After waiting for the process to exit, we'll get the result of the task, which will be the full text of the process's standard output stream.