1. Overview
Listing files is a fundamental operation when working with the Linux command line.
In this tutorial, we’ll explore how to list files whose filenames don’t match a given pattern.
2. Introduction to the Problem
When it comes to listing files in the Linux command line, two commonly used commands tend to catch our attention: the ls command and the find command.
For example, we can list files under the current directory in a list view using the ls command with the well-known -l option:
$ ls -l
total 0
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 console.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:59 kent.file
-rw-r--r-- 1 kent kent 0 Mar 21 21:37 readme.txt
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 server.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 system_warning.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 user.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 warning.txt
Using ls with globbing gives us the flexibility to list only the files that are relevant to us, thereby removing any unnecessary clutter. For instance, we’d like to have a list of files whose names contain “warning“:
$ ls -l *warning*
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 system_warning.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 warning.txt
We can get the same file list using the find command:
$ find . -name '*warning*'
./warning.txt
./system_warning.log
However, sometimes we want to perform an “inverse filtering” operation. For example, we want to list files whose filenames don’t contain “warning“ under this directory.
Next, let’s see how to do this. Of course, we’ll discuss both the ls and the find commands.
3. Using the ls Command
As the ls command is probably the most commonly used file listing command, let’s first look at the inverse filtering with the ls command.
3.1. How About Piping the ls Output to grep -v?
We know the grep command is a great pattern-matching command-line tool. Further, with the -v option, grep allows us to do an inverted search.
So, why not pipe the output of ls -l to grep -v? First, let’s see if it works with our example:
$ ls -l | grep -v 'warning'
total 0
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 console.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:59 kent.file
-rw-r--r-- 1 kent kent 0 Mar 21 21:37 readme.txt
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 server.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 user.log
As we can see, all filenames containing “warning” has been filtered out. The idea works!
However, “grepping” on ls output is considered a bad practice, and we shouldn’t go for it.
A couple of examples can quickly explain the reason. Let’s say we want to get a list of files whose name contains “kent“. Under the current directory, as we can see, only the kent.file should be in the output. Now, let’s see what ls | grep produces:
$ ls -l | grep 'kent'
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 console.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:59 kent.file
-rw-r--r-- 1 kent kent 0 Mar 21 21:37 readme.txt
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 server.log
-rw-r--r-- 1 kent kent 0 Mar 21 23:17 some
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 system_warning.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 user.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 warning.txt
This time, the command list all files under the directory. This is because all files’ owners and groups are “kent” too. The grep command simply checks each output line and cannot tell whether a match is in the filename part.
Apart from that, many filesystems under Linux support special characters in filenames, such as space and newlines. So next, let’s create a new file under this directory:
$ touch "some
dquote> whatever warning.txt"
$ ls -l
total 0
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 console.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:59 kent.file
-rw-r--r-- 1 kent kent 0 Mar 21 21:37 readme.txt
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 server.log
-rw-r--r-- 1 kent kent 0 Mar 21 23:36 'some'$'\n''whatever warning.txt'
...
As the output above shows, we’ve created a file whose name contains newline and space characters. The ls -l output displays the filename as ‘some’$’\n”whatever warning.txt’. Now, let’s use the ls | grep approach to list all files whose names have “warning“:
$ ls -l | grep 'warning'
whatever warning.txt
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 system_warning.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 warning.txt
Obviously, the output above isn’t expected. The filename ‘some’$’\n”whatever warning.txt’ has been broken. This is because grep is a line-based pattern-matching tool. It cannot handle the filenames with newlines correctly.
Therefore, the ls | grep -v approach may work for some cases. However, it’s not a stable solution. So we should avoid using it.
3.2. Using the ls Command With the -I Option
Most modern Linux distributions use the GNU ls from the pre-installed GNU coreutils package. The GNU ls has provided the -I option to ignore glob patterns.
Next, let’s get a file list excluding “warning“s:
$ ls -l
total 0
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 console.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:59 kent.file
-rw-r--r-- 1 kent kent 0 Mar 21 21:37 readme.txt
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 server.log
-rw-r--r-- 1 kent kent 0 Mar 21 23:36 'some'$'\n''whatever warning.txt'
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 system_warning.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 user.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 warning.txt
$ ls -lI '*warning*'
total 0
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 console.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:59 kent.file
-rw-r--r-- 1 kent kent 0 Mar 21 21:37 readme.txt
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 server.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 user.log
As the output above shows, the command has produced the expected result, although one filename with “warning” contains a newline character.
3.3. Using Bash Shell’s Inverse Globbing
Most shells have the extended globbing option. If this option is enabled, we can use the inverse globbing feature. In this tutorial, we’ll take the popular Bash and Zsh as examples to address how to use their inverse globbing features.
Bash is probably one of the most widely-used shells. Let’s enable the extended globbing option:
$ shopt -s extglob
Further, we can check if this option has been enabled on a Bash shell:
$ shopt -s | grep 'extglob'
extglob on
With this option enabled, we can use “*!(pattern_to_exclude)*” to exclude filenames matching the pattern_to_exclude:
$ ls -l !(*warning*)
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 console.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:59 kent.file
-rw-r--r-- 1 kent kent 0 Mar 21 21:37 readme.txt
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 server.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 user.log
As we can see, the output above doesn’t contain any filename with “warning“. It’s worth mentioning that the command has correctly handled the file whose name contains a newline character.
3.4. Using Zsh Shell’s Inverse Globbing
Zsh is another popular shell program, and it supports the extended globbing option as well. Let’s enable Zsh’s extended globbing option:
zsh% setopt extendedglob
Also, we can verify whether the option has been enabled in the current Zsh:
zsh% set -o | grep extendedglob
extendedglob on
After ensuring the extendedglob option is enabled, we can use the inverse globbing feature. But we should note that Zsh’s inverse globbing syntax is a bit different from Bash’s: ^PATTERN_TO_EXCLUDE.
Next, let’s exclude all files whose filenames contain “warning” from our example directory:
zsh% ls -l ^*warning*
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 console.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:59 kent.file
-rw-r--r-- 1 kent kent 0 Mar 21 21:37 readme.txt
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 server.log
-rw-r--r-- 1 kent kent 0 Mar 21 21:24 user.log
As the output above shows, it works as expected.
4. Using the find Command
The find command is another common utility to find and list files in the Linux command line. For example, we’ve seen earlier that find with the -name test allows us to find files whose names match the given pattern: find -name “PATTERN”.
To negate a test in the find command, we can simply put a ‘*!*‘ character in front of a test, such as ! -name “PATTERN” to filter filenames not matching the given pattern.
Next, let’s apply this approach to listing files excluding the “warning” files:
$ find . ! -name '*warning*'
.
./kent.file
./readme.txt
./user.log
./console.log
./server.log
As the output shows, the find command does the job.
5. Conclusion
In this article, we’ve explored how to list files whose filenames don’t match a given pattern.
Some of us may have used the ls -l | grep -v approach to achieve it. Therefore, first, we highlighted its potential drawbacks through a couple of examples.
Then, we discussed effective ways to list files that don’t match a specified pattern using ls and find.
With the ls command, we can either use the “-I pattern” option to ignore a particular pattern or use the inverse globbing expression to achieve that when the shell’s extended globbing option is enabled.
Alternatively, the find command can list files that don’t match a given pattern. We can simply add ‘*!*‘ in front of a test to negate it. For example, ! -name ‘*.txt’ excludes all *.txt files.