1. Overview
In this tutorial, we’ll be looking into how to track child processes using the tool strace in Linux.
We’ll start with a script to create child processes. Then, we’ll track it with strace without any options. Finally, we’ll track it with strace with specific options useful for tracking child processes.
2. Script Prerequisites
In an empty directory, let’s create a Python script named create_child_processes.py:
#!/usr/bin/python3
import os
def fork_many_processes():
print(f"I'm the original process and my id is: {os.getpid()}")
n = os.fork()
if n == 0:
print(f"I'm the child process and my id is: {os.getpid()}")
n = os.fork()
if n == 0:
print(f"I'm the grandchild process and my id is: {os.getpid()}")
n = os.fork()
if n == 0:
print(f"I'm the great-grandchild process and my id is: {os.getpid()}")
fork_many_processes()
We use Python because it’s much easier compared to compiling C/C++ source code and executing a binary file. But the tutorial works with other languages as well.
2.1. Forking the Process
To fork a process means to create an identical and independent process. In Python, we use the fork method from the os module:
n = os.fork()
For the current process, the method returns the child process id. A new process is also born. This process inherits everything from the parent process and resumes the execution in the fork method. But this time, the method returns 0.
So, we can differentiate which process is the parent and which process is the child by checking the return value of the fork method. If the return value is 0, then the process of the script is the child process:
if n == 0:
Of course, the child process can also create another child process using the same mechanism.
2.2. Executing the Script
First, we need to make sure that the file is executable:
chmod +x create_child_processes.py
Then we can run it:
$ ./create_child_processes.py
I'm the original process and my id is: 912212
I'm the child process and my id is: 912213
I'm the grandchild process and my id is: 912214
I'm the great-grandchild process and my id is: 912215
The process created a child process. This child process created another child process that is the grandchild of the original process. Then this grandchild process created another child process that is the great-grandchild of the original process.
3. Using strace
We can use strace to observe system calls executed by the Python script:
$ strace ./create_child_processes.py
...(Preceding output truncated)
read(3, "#!/usr/bin/python3\nimport os\n\nde"..., 4096) = 504
read(3, "", 4096) = 0
close(3) = 0
getpid() = 993199
write(1, "I'm the original process and my "..., 46I'm the original process and my id is: 993199
) = 46
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f687ce1aa10) = 993200
rt_sigaction(SIGINT, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f687d02c210}, {sa_handler=0x629b90, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f687d02c210}, 8) = 0
I'm the child process and my id is: 993200
I'm the grandchild process and my id is: 993201
I'm the great-grandchild process and my id is: 993202
sigaltstack(NULL, {ss_sp=0x252c460, ss_flags=0, ss_size=16384}) = 0
sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}, NULL) = 0
exit_group(0) = ?
+++ exited with 0 +++
We can recognize some system call lines, like getpid() = 993199. The Python code os.getpid() invoked this system call. It’s easy to deduce that the print method in the Python script executed the write system call. Then, we finally see the clone system call.
As we know, the os.fork method in the Python script called this system call.
However, we don’t see the system calls invoked by the child processes. We only see the output produced by the child processes.
3.1. Tracking the Child Processes Using strace
strace has options that are useful for tracking child processes. We can use the –f option:
$ strace -f ./create_child_processes.py
...(Preceding output truncated)
getpid() = 994450
write(1, "I'm the original process and my "..., 46I'm the original process and my id is: 994450
) = 46
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLDstrace: Process 994451 attached
, child_tidptr=0x7f48eab43a10) = 994451
[pid 994451] set_robust_list(0x7f48eab43a20, 24) = 0
[pid 994450] rt_sigaction(SIGINT, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f48ead55210}, {sa_handler=0x629b90, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f48ead55210}, 8) = 0
[pid 994451] getpid() = 994451
[pid 994451] write(1, "I'm the child process and my id "..., 43I'm the child process and my id is: 994451
) = 43
[pid 994451] clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f48eab43a10) = 994452
strace: Process 994452 attached
[pid 994452] set_robust_list(0x7f48eab43a20, 24) = 0
[pid 994451] rt_sigaction(SIGINT, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f48ead55210}, {sa_handler=0x629b90, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f48ead55210}, 8) = 0
[pid 994452] getpid() = 994452
[pid 994452] write(1, "I'm the grandchild process and m"..., 48I'm the grandchild process and my id is: 994452
) = 48
...(Subsequent output truncated)
The system calls generated by the child processes are prepended with [pid …]. The number in the brackets is the process id, meaning that we can track which child process produced this system call. We can also track when this child process was born because the clone system call displayed the child process id it produced.
3.2. Tracking the Child Processes with a File using strace
It’s hard to analyze the child processes from the output in the terminal. Perhaps we should save it to a file for analysis later? No, we cannot use the redirecting technique:
$ strace -f ./create_child_processes.py > dump
$ cat dump
I'm the original process and my id is: 32086
I'm the original process and my id is: 32086
I'm the child process and my id is: 32087
I'm the original process and my id is: 32086
I'm the child process and my id is: 32087
I'm the grandchild process and my id is: 32088
I'm the original process and my id is: 32086
I'm the child process and my id is: 32087
I'm the grandchild process and my id is: 32088
I'm the great-grandchild process and my id is: 32089
Apparently, strace doesn’t display the output in the stdout. To save the output to a file, we can use the -o option and the name of the file as the argument for that option:
$ strace -o strace_log -f ./create_child_processes.py
I'm the original process and my id is: 32262
I'm the child process and my id is: 32263
I'm the grandchild process and my id is: 32264
I'm the great-grandchild process and my id is: 32265
$ tail strace_log
32265 sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}, <unfinished ...>
32264 sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}, <unfinished ...>
32265 <... sigaltstack resumed>NULL) = 0
32264 <... sigaltstack resumed>NULL) = 0
32265 exit_group(0 <unfinished ...>
32264 exit_group(0 <unfinished ...>
32265 <... exit_group resumed>) = ?
32264 <... exit_group resumed>) = ?
32265 +++ exited with 0 +++
32264 +++ exited with 0 +++
As we can see, the child process line starts with the process id, like 32265 or 32264 in this example.
3.3. Tracking the Child Processes with Separate Files using strace
It’s hard to track many child processes’ activities in one place like we did previously. Wouldn’t it be nice if we could look at each child process’ activities in separate places? We can do that by using strace with the -ff option and the -o option:
$ strace -o output -ff ./create_child_processes.py
I'm the original process and my id is: 31117
I'm the child process and my id is: 31118
I'm the grandchild process and my id is: 31119
I'm the great-grandchild process and my id is: 31120
The -o option accepts a filename pattern that strace will use to create the files. These files will contain the child processes’ activities:
$ ls output*
output.31117 output.31118 output.31119 output.31120
The number that’s appended as the file extension is the process id of the original process and the child processes. The dump file with the lowest id (output.31117 in this example) is the origin process’ dump file. We can confirm it:
$ grep origin output.31117
write(1, "I'm the original process and my "..., 45) = 45
We can open the other dump files and confirm that they are the child processes’ dump files:
$ cat output.31118
set_robust_list(0x7f5995cc9a20, 24) = 0
getpid() = 31118
write(1, "I'm the child process and my id "..., 42) = 42
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f5995cc9a10) = 31119
rt_sigaction(SIGINT, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f5995edb210}, {sa_handler=0x629b90, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f5995edb210}, 8) = 0
sigaltstack(NULL, {ss_sp=0x167d460, ss_flags=0, ss_size=16384}) = 0
sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}, NULL) = 0
exit_group(0) = ?
+++ exited with 0 +++
4. Conclusion
In this article, we’ve covered how to use strace to track child processes.
First, we used strace without any options to track a process, and we could see that the system calls generated by the child processes were missing.
Then, we used the -f option, and the system calls from the child processes were displayed. We also saved the output to a file using the -o option.
Finally, we used the -ff option and the -o option to dump the output of the child processes’ system calls into different files.