1. Overview
In this tutorial, we’ll explore few common strategies to redirect the output of a process to a file and standard streams such as stdout and stderr simultaneously.
2. The tee Command
The tee command is one of the most popular Linux commands that we can use for redirecting a process’s output.
2.1. Redirect stdout
Let’s take a simple use case of redirecting the output of the ls command to stdout and a temporary file /tmp/out.log:
$ ls -C | tee /tmp/out.log
bin dev home lib32 libx32 mnt proc run srv tmp var
boot etc lib lib64 media opt root sbin sys usr
We can verify that the contents of the file are the same as the output generated from the executed command:
$ cat /tmp/out.log
bin dev home lib32 libx32 mnt proc run srv tmp var
boot etc lib lib64 media opt root sbin sys usr
Another important thing to note is that the default behavior of the tee command is to overwrite the contents of the file. However, if required, we can choose the -a option to append the new content after the existing content of a file.
2.2. Redirect stdout and stderr to the Same File
We need to understand that internally, the tee command is acting as a T-splitter for the incoming stdin so that data can be redirected to the stdout and one or more files. Let’s use this understanding to redirect stderr of a process to stdout and a file:
$ (ls -C; cmd_with_err) 2>&1 | tee /tmp/out.log
bin dev home lib32 libx32 mnt proc run srv tmp var
boot etc lib lib64 media opt root sbin sys usr
bash: cmd_with_err: command not found
We can notice that cmd_with_err is an unknown command, so it generates an error message. To make this available to the tee command, we redirect the stderr file descriptor (fd=2) to the stdout file descriptor (fd=1).
Alternatively, we can also use |& as a shorthand notation for 2>&1| to get the same result:
$ (ls -C; cmd_with_err) |& tee /tmp/out.log
bin dev home lib32 libx32 mnt proc run srv tmp var
boot etc lib lib64 media opt root sbin sys usr
bash: cmd_with_err: command not found
Now, let’s verify the content of the /tmp/out.log file:
$ cat /tmp/out.log
bin dev home lib32 libx32 mnt proc run srv tmp var
boot etc lib lib64 media opt root sbin sys usr
bash: cmd_with_err: command not found
2.3. Redirect stdout and stderr to Separate Files
In some scenarios, we might need to redirect the stdout and stderr of a process to separate files. We can do this by using process substitution while invoking the tee command. Before that, let’s take a look at a template code snippet that will enable the tee command to listen to a specific file descriptor and write back to the same file descriptor stream and a file:
fd> >(tee file_name fd>&fd)
We must note that fd is just a placeholder for a file descriptor, and the actual value will be 1 for stdout, 2 for stderr, and 0 for stdin.
Now, let’s use this understanding to redirect the stdout and stderr output of the process to /tmp/out.log and /tmp/err.log, respectively:
$ ((ls -C; cmd_with_err) 1> >(tee /tmp/out.log)) 2> >(tee /tmp/err.log 2>&2)
bin dev home lib32 libx32 mnt proc run srv tmp var
boot etc lib lib64 media opt root sbin sys usr
bash: cmd_with_err: command not found
We can verify that /tmp/out.log contains the valid stdout message, whereas /tmp/err.log contains the error message from stderr:
$ cat /tmp/out.log
bin dev home lib32 libx32 mnt proc run srv tmp var
boot etc lib lib64 media opt root sbin sys usr
$ cat /tmp/err.log
bash: cmd_with_err: command not found
3. Redirection Delays
In a few scenarios, invocation of the tee command to redirect the process’s output to a file and stdout can introduce delay. In this section, we’ll explore such a scenario and learn to mitigate it.
3.1. Scenario
Let’s see a simple python script that prints the current time every one second:
$ cat time.py
#!/usr/bin/python
from datetime import datetime
import time
import sys
from sys import stdout
while True:
sys.stdout.write(datetime.today().strftime("%H:%M:%S %p\n"))
time.sleep(1)
If we execute this script, we’ll observe a delay of one second between any two consecutive timestamps written on the stdout:
$ ./time.py
6:49:48 PM
6:49:49 PM
6:49:50 PM
6:49:51 PM
6:49:52 PM
6:49:53 PM
3.2. Delayed Redirection
Now, let’s use the tee command to redirect the output of this process to stdout and the time.out file:
$ ./time.py | tee time.out
Unlike earlier, we’ll notice that there’s no output written on stdout for a long time, after which a huge chunk of output will be dumped on stdout in a single go.
The delay is introduced due to Linux’s stdio buffering policy in the glibc, a system library used by python internally. The buffering policy causes the writes to stdout to pass through a 4096–byte buffer, thereby reducing the number of I/O calls required to write on the stream.
For interactive applications, such delays in redirection are not acceptable. So, let’s find ways to mitigate the redirection delay issue.
3.3. Mitigation
As the root cause of the issue is associated with the delay in flushing of data to stdout, one way to solve this issue is by ensuring timely flushing of data to stream in the application code:
$ cat time.py
#!/usr/bin/python
from datetime import datetime
import time
import sys
from sys import stdout
while True:
sys.stdout.write(datetime.today().strftime("%H:%M:%S %p\n"))
sys.stdout.flush()
time.sleep(1)
Let’s verify that the delay is indeed gone:
$ ./time.py | tee time.out
19:29:12 PM
19:29:13 PM
In this scenario, we had direct access to the application code, so we were able to modify it.
But, in many cases, the program could be an executable binary, and we might not have access to modify it. In such a scenario, we can use the unbuffer command in Linux to solve the delay caused by buffered writes to stdout.
Let’s remove the sys.stdout.flush() method call from our script and re-execute the redirection command using the unbuffer command:
$ unbuffer ./time.py | tee time.out
19:34:22 PM
19:34:23 PM
We can observe that there are no unexpected delays in the stdout writes now.
4. Conclusion
In this article, we explored several use cases of redirecting a program’s output to streams such as stdout and stderr simultaneously. Additionally, we learned few strategies to solve the issue of delayed redirection caused due to buffering.