Tom is the technical SEO manager at Blue Array, he mentions that in his role the standard SEO tasks can take up a huge amount of time when working with big data. He’s found that MAC OS Terminal is a really good way to speed this up.
What is Terminal?
Command Interface between you and the computer within the applications folder of MAC, when you open it up you don’t get anything much. It will take time to learn these commands.
What commands do I use?
Curl
Curl is a command that can be used to check response codes. This pulls the entire HTML reponse of what you’re testing, you can also just get the HTTP headers using curl – I. You can also follow redirects by adding in a second modifier. By putting in -O you can download the file directly to your home directory. Type man curl
Sort
This sorts things such as keyword data so you can import a csv and order this correctly. sort filename.csv. If a file is tens of thousands of rows maybe try to avoid these but you can get around this by pushing it to a new file.
Head/Tail
You can use this to see the first (head) or last (tail) rows helping you manage these large csv files and see if your file has worked.
Cat
Short for concatenate, used to combine create and organise files. This can also be used to sort keywords into CSV files. Make sure you use this in the right place as you don’t what to concatenate all of your CSV’s!
SED
Allows you to add text to rows. This can be used in conjunction with regex to add things like www or protocols to the start of URLs in your CSV’s.
Awk
This command is almost a programming language on its own which has many applications, Tom goes on to use log file analysis as an example
Case Study – Log File Analysis
Tom shows an example of log files he’s worked on with a client. He was able to extract all of the 404 errors from the log files along with all other status codes such as 404’s, 200’s and 301’s. He was also able to extract ALL search engine bot hits using the awk command which means you can review how crawlers are hitting your website.
Overall some really handy tips on how to use terminal as an SEO to speed up your workflow whilst making it look like you are coding!
