Grep
From BITS wiki
filter huge files for relevant parts
You constantly get files of which only a small part is relevant! grep is the tool you need to squeeze millions of lines of data and keep only the essence.
# grep can be used in keep-mode to retain only what patches the filter $ grep "my-pet-name" huge_list_of_animals.txt # but also in trim-mode to exclude unwanted lines $ grep -v "dinosaurs" huge_list_of_animals.txt # and as always, the unix pipe is there to bless a perfect union # !! apply first the filter that will shrink your input the most for speedup $ grep "my-pet-name" huge_list_of_animals.txt | grep -v "dinosaurs" # which is the same as $ grep -v "dinosaurs" huge_list_of_animals.txt | grep "my-pet-name"
Note: grep can also count instead of returning results, just add -c to the command. Finally, zgrep will do on archives.
[ Main_Page ]