Log Analysis
SSH Logs
Example of an SSH log:
Oct 11 10:36:01 myraptor sshd[29981]: Connection from 56.13.188.38 port 55323
Oct 11 10:36:02 myraptor sshd[29983]: Failed password for harvey from 56.13.188.38 port 55323 ssh2
Oct 11 10:36:02 myraptor sshd[29985]: Received disconnect from 56.13.188.38: Bye Bye
Oct 11 10:36:04 myraptor sshd[29987]: Connection from 56.13.188.38 port 55324
Oct 11 10:36:05 myraptor sshd[29989]: Failed password for harvey from 56.13.188.38 port 55324 ssh2
Oct 11 10:36:05 myraptor sshd[29991]: Failed password for harvey from 56.13.188.38 port 55324 ssh2
Oct 11 10:36:05 myraptor sshd[29993]: Received disconnect from 56.13.188.38: 11: Bye Bye
Oct 11 10:36:55 myraptor sshd[29995]: Connection from 30.167.206.91 port 55325
Oct 11 10:36:57 myraptor sshd[29997]: Failed password for harvey from 30.167.206.91 port 55325 ssh2
Oct 11 10:36:57 myraptor sshd[29999]: Received disconnect from 30.167.206.91: 11: Bye Bye
Oct 11 10:36:58 myraptor sshd[30001]: Connection from 30.167.206.91 port 55326
Oct 11 10:36:59 myraptor sshd[30003]: Accepted password for harvey from 30.167.206.91 port 55326 ssh2
Oct 11 10:36:59 myraptor sshd[30005]: pam_unix(sshd:session): session opened for user harvey by (uid=0)
In this case, the compromised SSH server name would be myraptor. The attacker's IP address is 30.167.206.91 and the user being attacked is harvey. The attacker tried to brute force the password with the username harvey and it worked.
Nginx Access Logs
Example of an Nginx access log:
193.19.118.8 - - [30/Sep/2015:14:47:16 -0400] "GET /admin/ HTTP/1.0" 404 162 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:30.N) Gecko/20110302 Firefox/30.0"
84.51.79.188 - - [30/Sep/2015:15:20:39 -0400] "GET http://login.yahoo.com/config/isp_verify_user? HTTP/1.0" 404 162 "-" "-"
141.212.122.146 - - [30/Sep/2015:16:19:49 -0400] "CONNECT proxytest.zmap.io:80 HTTP/1.1" 400 166 "-" "-"
66.249.83.195 - - [30/Sep/2015:16:46:03 -0400] "GET / HTTP/1.1" 502 166 "-" "Google favicon"
169.50.3.171 - - [30/Sep/2015:17:13:04 -0400] "GET /xmlrpc.php HTTP/1.1" 404 162 "-" "-"
169.50.3.171 - - [30/Sep/2015:17:13:05 -0400] "" 400 0 "-" "-"
An easy way to analyze these without manually poring over all the data is with GoAccess. It can be installed locally with sudo apt install goaccess. The Common Log Format (CLF) is the default format for Nginx access logs.
Squid Proxy Logs
Example of a Squid proxy log:
1286536308.779 180 192.168.0.224 TCP_MISS/200 411 GET http://liveupdate.symantecliveupdate.com/minitri.flg - DIRECT/125.23.216.203 text/plain
1286536308.910 37 192.168.0.224 TCP_MISS/200 4083 GET http://liveupdate.symantecliveupdate.com/streaming/norton$202009$20streaming$20virus$20definitions_1.0_symalllanguages_livetri.zip - DIRECT/125.23.216.203 application/zip
1286536309.450 917 192.168.0.227 TCP_MISS/200 20670 GET http://www.youtube.com/watch? - DIRECT/209.85.231.136 text/html
1286536309.549 172 192.168.0.227 TCP_MISS/204 294 GET http://v15.lscache3.c.youtube.com/generate_204? - DIRECT/122.160.120.150 text/html
1286536309.586 921 192.168.0.68 TCP_MISS/200 507 POST http://rcv-srv37.inplay.tubemogul.co...eiver/services - DIRECT/174.129.41.128 application/xml
1286536309.608 829 192.168.0.68 TCP_MISS/200 507 POST http://rcv-srv37.inplay.tubemogul.co...eiver/services - DIRECT/174.129.41.128 application/xml
1286536309.660 785 192.168.0.68 TCP_MISS/200 507 POST http://rcv-srv37.inplay.tubemogul.co...eiver/services - DIRECT/174.129.41.128 application/xml
If you need to convert the timestamp into a human date, use the Epoch Converter.
To sort based on response time:
cat squid_access.log | awk '{print $2}' | sort -n
Find unique IP addresses:
cat squid_access.log | awk '{print $3}' | sort | uniq
To find the occurrences of HTTP methods:
cat squid_access.log | awk '{print $6}' | sort | uniq -c
Browser SQLite History
The browser history is stored in a SQLite database. It is easiest to parse through visually with sqlitebrowser, and most of the useful data will be in the moz_places table.