Linux File Manipulation Commands
I’ll keep updating this with various commands that I run into. If I find new ones I’ll update them from time to time. Please feel free to comment. It is starting as my personal cheat sheet storage place.
File Searching
List of file extensions.
awk + grep : find all the file extensions. It will also skip your “git” repository.
find . -type f -name "*.*" ! -path "./.git/*" | awk -F. '!a[$NF]++{print $NF}'
To get the count then simply add `wc` command.
find . -type f -name "*.*" ! -path "./.git/*" | awk -F. '!a[$NF]++{print $NF}' | wc -l
File Information
Print the longest line in a file
awk ‘length > m { m = length; a = $0 } END { print a }’ input-file
Get the line number of the longest line
awk ‘length > m { m = length; a = NR } END { print a }’ input-file
Vim Commands
To find non-ascii characters in your file:
/[^\x00-\x7F]
Logs
Apache Logs
Find count of unique IP addresses
Get the IP’s - usually the first item
$ awk '{ print $1 } ' access_log
Sort them
$ awk '{ print $1 } ' access_log | sort
They get only the unique items using the uniq command. You can count these if you want.
$ awk '{ print $1 } ' access_log | sort | uniq
To get the counts and sort them by totals.
$ awk '{ print $1 } ' access_log | sort | uniq -c | sort -n -k 1
If you only want to show the highest numbers.
$ awk '{ print $1 } ' access_log | sort | uniq -c | sort -n -k 1 | tail