Graph Exchange

Picviz: Let's see uncommon URL (part 2/?)

Picviz: Let's see uncommon URL (part 2/?)

Today, I would like to see if the urls that are not common in the previous graph, In this graph, heatline rendering plugin is used to check with line coloration if an event is regular. In the fourth axis, you can see lines going at the bottom and red lines go there. So let's forget about this and filter to only display lines that appear above 50% of this axis.

The filter is between single quotes, just like what you'd do with tcpdump ( I actually took their code to handle this ;-) ).

This line was typed to get the graph you can see here:
pcv -Tpngcairo -Rheatline -Avirus access-wallinfire.net.pcv 'show plot > 50% on axis 4' -ra > picviz-uncommonurls.png

If we take a random IP, such as the one we clearly see on the second axis, 213.192.60.19, and googling about it, we find that this was an infected machine. The url here tells more about it.

As a conclusion for this graph, you can see that among all those lines of log, with a very empiric approach, we really discovered something. Not a very innovative attack I admit, but enough to keep searching (I will post ongoing researches here, keep following!).

Ah, and by the way Raffy, since you asked to only display lines every few times, I added the -L option, taking a number (N) as argument meaning every N lines you display the text.

Picviz graphing apache logs

Picviz graphing apache logs

This parallel coordinates graph shows 412429 lines of one of my wallinfire.net access log with generated with Picviz svn. This is the first of a set of graphs which will derivate from this one. The most complete one.

To generate such a graph, simply use the apache-access2picviz Perl script available from trunk/tools. Then, use the heatline plugin to see line frequencies: the more green the line is, the lowest it appears. When a line is in red, it means it comes often. This way you can easily see if an event is regular or not. To generate this image, you can type: pcv -Tpngcairo access.pcv -Rheatline -Avirus -rra > accesslogs.png

First axis = Time (24 hour) with 00:00 at the bottom and 23:59 on the very top.

Second axis = Source IP with 0.0.0.0 at the bottom and 255.255.255.255 on the very top.

Third axis = HTTP request type.

Fourth = Request

Tomorrow, I will post a filtered graph, on the request axis to see what are the IP addresses that are doing abnormal requests.

Picviz is available as free software on http://www.wallinfire.net/picviz

GEO Tagging Attacks

GEO Tagging Attacks

I am working on a Parser that transforms any kind of ASCII input log file into KML files used in Google Earth/Maps.

What does it do?

1. Reads the log
1a. Extracts an IP (column or Regex based)
1b. Extracts a node name (column or Regex based)
1c. Extracts a description (column or Regex based)
2. Retrieves a longitude and latitude to the given IP address
3. Writes a KML file for Google Earth

Why?

Well I was recently analyzing a big set of IDS events with abut 99% of false positives. Most of them originated from partners of the company located in England and France. Only a few of them - the real positives - originated from countries like Iran, Pakistan, Brazil, Russia and China.
That way it was really smooth to determine the real attacks and leave the false positives aside.

The picture is an animated GIF. Hope this format comes through. :)
It shows an analysis of a mod_security log file.

I'll provide a download link for the script soon. (the sticking point is the GeoIP database; most of them are commercial ones; currently I use TOR network and a GEO IP web service to determine the LAT and LON parameters. That should not be the final solution, right?!)

---- Update -----
I use a local database now which makes it very - Very - fast. It parses 8500 mod_security events in 10 secs. The resulting KML file has 5 MB. I generated a new picture set showing all events of 9 month placed in a world map. And I have to say ... "bad - bad bad Italy!". I am quite sure that this was the spreading of the MPack Malware Kit this spring.

--- Update 2 ---
I advanced the script with an option to define a source and target IP address to draw lines in the world map. Nice, isn't it? Now it is possible to parse log files with source and target to determine the connections between the nodes. Targets are marked with different icons.

INAV

INAV

INAV is a project that displays connection information in real time. It creates a dynamic interactive directed graph in real time. http://inav.scaparra.com

Analyzing Windows Eventlog Types

Analyzing Windows Eventlog Types

Windows Eventlog analysis with Nazar GUI using mouseover to determine the user accounts which caused the events.
*New version works web based Flash application with CSV input

Interesting Pattern in Storm Worm

Interesting Pattern in Storm Worm

Plotted above is the used Portrange of a Storm Worm Spambot with private IP. Interesting that it (almost) stops at about Port Number 33.789, very sparse above that... Verified with multiple binaries and by the analysis of a so-called Storm Gateway (supernode) with public IP, here as well sparse data above the mentioned port, while ports between 50000 and 51000 seem to be very dense again. More information and plots on
http://honeyblog.org/archives/196-Interesting-Pattern-in-Storm-Worm-Traffic.html

For these plots, I analyzed the binaries in NetFlow data, converted it to CSV Files and did some data mining on these files with the commercial tool 'SPSS Clementine'

Picviz iptables graph

Picviz iptables graph

Graph of ten minutes of iptables logs, showing 8000 events. It was generated with the not released yet Picviz (http://sourceforge.net/projects/picviz/) program.

More details on my blog

Visualized Storm fireworks for your 4th of July

Visualized Storm fireworks for your 4th of July

Turning old Storm news into a celebration of the 4th of July, we applied little AfterGlow magic to fireworks.pcap,
tcpdump -vttttnnelr /home/rmcree/pcap/fireworks.pcap | ./tcpdump2csv.pl "sip dip ttl" | perl ../graph/afterglow.pl -c /home/rmcree/afterglow/src/perl/graph/color.properties -p 2 | neato -Tgif -o fireworks.gif,
and the results look just like the fireworks we hoped they would.
For the analysis of this Storm variant, fireworks.exe, and the resulting fireworks.pcap that lead to this visualization, see http://holisticinfosec.blogspot.com/2008/07/visualized-storm-fireworks-for-your-4th.html.
Happy 4th of July!

API Calls and Imported Symbols of Nepenthes Download Binary Files

API Calls and Imported Symbols of Nepenthes Download Binary Files

API Calls and Imported Symbols of Nepenthes Download Binary Files

The goal of this graph is to show the api calls and the imported symbols used by malware files collected by Nepenthes.

To extrat this information I reutilize a file from Jan Goebel ´s Amun project.

I´ve added some regex to detect imported symbols.

Source Code:

"""
Jaime Blasco - jaime.blasco[at]aitsec.com
Thanks to Jan Goebel
[Amun - low interaction honeypot]

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, see
"""

import sys
import os
import re

def start(content, name):
### api
checksbin = {}
checksbin['listen'] = re.compile('\\xa4\\xad\\x2e\\xe9', re.S|re.I)
checksbin['bind'] = re.compile('\\xa4\\x1a\\x70\\xc7', re.S|re.I)
checksbin['closeSocket'] = re.compile('\\xe7\\x79\\xc6\\x79', re.S|re.I)
checksbin['accept'] = re.compile('\\xe5\\x49\\x86\\x49', re.S|re.I)
checksbin['LoadLibraryA'] = re.compile('\\x8e\\x4e\\x0e\\xec', re.S|re.I)
checksbin['WSASocketA'] = re.compile('\\xd9\\x09\\xf5\\xad', re.S|re.I)
checksbin['WSAStartup'] = re.compile('\\xCB\\xED\\xFC\\x3B', re.S|re.I)
checksbin['ExitProcess'] = re.compile('\\x7e\\xd8\\xe2\\x73', re.S|re.I)
checksbin['CreateProcessA'] = re.compile('\\x72\\xfe\\xb3\\x16', re.S|re.I)
checksbin['WaitForSingleObject'] = re.compile('\\xad\\xd9\\x05\\xce', re.S|re.I)
checksbin['system'] = re.compile('\\x44\\x80\\xc2\\x77', re.S|re.I)
checksbin['SetStdHandle'] = re.compile('\\x1d\\x20\\xe8\\x77', re.S|re.I)
checksbin['GetProcAddress'] = re.compile('\\xcc\\x10\\xbe\\x77', re.S|re.I)
checksbin['URLDownloadToFileA'] = re.compile('\\x36\\x1a\\x2f\\x70', re.S|re.I)
checksbin['connect'] = re.compile('\\xec\\xf9\\xaa\\x60', re.S|re.I)
checksbin['socket'] = re.compile('\\x6e\\x0b\\x2f\\x49', re.S|re.I)
checksbin['socket2'] = re.compile('\\x83\\x53\\x83\\x00', re.S|re.I)
checksbin['send'] = re.compile('\\xa4\\x19\\x70\\xe9', re.S|re.I)
checksbin['receive'] = re.compile('\\xb6\\x19\\x18\\xe7', re.S|re.I)
checksbin['WinExec'] = re.compile('\\x98\\xfe\\x8a\\x0e', re.S|re.I)
checksbin['WriteFile'] = re.compile('\\x1f\\x79\\x0a\\e8', re.S|re.I)
checksbin['Unknown (sign for correct decryption)'] = re.compile('\\x68\\x33\\x32\\x00\\x00\\x68\\x77\\x73\\x32\\x5F', re.S|re.I)

### plain
checksplain = {}
checksplain['possible windows cmd'] = re.compile('\\x63\\x6d\\x64', re.S|re.I)
checksplain['http address'] = re.compile('\\x68\\x74\\x74\\x70\\x3a\\x2f\\x2f', re.S|re.I)
checksplain['ftp address'] = re.compile('\\x66\\x74\\x70\\x3a\\x2f\\x2f', re.S|re.I)
checksplain['tftp.exe'] = re.compile('\\x74\\x66\\x74\\x70\\x2e\\x65\\x78\\x65', re.S|re.I)
checksplain['WSAStartup'] = re.compile('\\x57\\x53\\x41\\x53\\x74\\x61\\x72\\x74\\x75\\x70', re.S|re.I)
checksplain['WSASocketA'] = re.compile('\\x57\\x53\\x41\\x53\\x6f\\x63\\x6b\\x65\\x74\\x41', re.S|re.I)
checksplain['GetProcAddress'] = re.compile('\\x47\\x65\\x74\\x50\\x72\\x6f\\x63\\x41\\x64\\x64\\x72\\x65\\x73\\x73',re.S|re.I)
checksplain['CreateProcessA'] = re.compile('\\x43\\x72\\x65\\x61\\x74\\x65\\x50\\x72\\x6f\\x63\\x65\\x73\\x73\\x41', re.S|re.I)
checksplain['CreateFileA'] = re.compile('\\x43\\x72\\x65\\x61\\x74\\x65\\x46\\x69\\x6c\\x65\\x41', re.S|re.I)

### plain imported symbols
checksplainimport = {}
checksplainimport['kernel32'] = re.compile('\\x6b\\x65\\x72\\x6e\\x65\\x6c\\x33\\x32',re.S|re.I)
checksplainimport['USER32'] = re.compile('\\x55\\x53\\x45\\x52\\x33\\x32',re.S|re.I)
checksplainimport['MSVCR80'] = re.compile('\\x4d\\x53\\x56\\x43\\x52\\x38\\x30',re.S|re.I)
checksplainimport['ws2_32'] = re.compile('\\x77\\x73\\x32\\x5f\\x33\\x32',re.S|re.I)
checksplainimport['shell32'] = re.compile('\\x73\\x68\\x65\\x6c\\x6c\\x33\\x32',re.S|re.I)
checksplainimport['gdi32'] = re.compile('\\x67\\x64\\x69\\x33\\x32',re.S|re.I)
checksplainimport['oleaut32'] = re.compile('\\x6f\\x6c\\x65\\x61\\x75\\x74\\x33\\x32',re.S|re.I)
checksplainimport['advapi32'] = re.compile('\\x61\\x64\\x76\\x61\\x70\\x69\\x33\\x32',re.S|re.I)
checksplainimport['COMCTL32'] = re.compile('\\x43\\x4f\\x4d\\x43\\x54\\x4c\\x33\\x32',re.S|re.I)
checksplainimport['wsock32'] = re.compile('\\x77\\x73\\x6f\\x63\\x6b\\x33\\x32',re.S|re.I)
checksplainimport['URLMON'] = re.compile('\\x55\\x52\\x4c\\x4d\\x4f\\x4e',re.S|re.I)
checksplainimport['msvcrt'] = re.compile('\\x6d\\x73\\x76\\x63\\x72\\x74',re.S|re.I)
checksplainimport['CRTDLL'] = re.compile('\\x43\\x52\\x54\\x44\\x4c\\x4c',re.S|re.I)
checksplainimport['WININET'] = re.compile('\\x57\\x49\\x4e\\x49\\x4e\\x45\\x54',re.S|re.I)
checksplainimport['ntdll'] = re.compile('\\x6e\\x74\\x64\\x6c\\x6c',re.S|re.I)

keys = checksplain.keys()
for key in keys:
match = checksplain[key].search(content)
if match:
print name + "," + key + ",2"

keys = checksbin.keys()
for key in keys:
match = checksbin[key].search(content)
if match:
print name + "," + key + ",2"

keys = checksplainimport.keys()
for key in keys:
match = checksplainimport[key].search(content)
if match:
print name + "," + key + ",1"

if __name__ == '__main__':
list = os.listdir("binaries/")
for filename in list:
if os.path.exists("binaries/" + filename):
fp = open("binaries/" + filename, 'r')
content = "".join(fp.readlines())
fp.close()
start(content, filename)

The CSV file looks like:
...
50c0c0fa44ed9e09bbe9558c61e22006,http address,2
50c0c0fa44ed9e09bbe9558c61e22006,gdi32,1
50c0c0fa44ed9e09bbe9558c61e22006,kernel32,1
50c0c0fa44ed9e09bbe9558c61e22006,ws2_32,1
50c0c0fa44ed9e09bbe9558c61e22006,oleaut32,1
50c0c0fa44ed9e09bbe9558c61e22006,USER32,1
50c0c0fa44ed9e09bbe9558c61e22006,shell32,1
50c0c0fa44ed9e09bbe9558c61e22006,advapi32,1
849c5ae144ed43741d1c2eb4d0cd552a,possible windows cmd,2
849c5ae144ed43741d1c2eb4d0cd552a,CreateProcessA,2
849c5ae144ed43741d1c2eb4d0cd552a,kernel32,1
849c5ae144ed43741d1c2eb4d0cd552a,MSVCR80,1
...
...

1: Imported Symbol
2: Api call

And the color.properties file to generate the grah with afterglow:

color.target="lightblue" if ($fields[2]==2)
color.target="green" if ($fields[2]==1)
color.source="red"

Malware Files Collected By Nepenthes - Imported Symbols Relation

Malware Files Collected By Nepenthes - Imported Symbols Relation

With several binaries collected by nepenthes I have correlate the imported symbols with python module pefile and generate an interesting graph.

CSV:
...
...
b02a18d2dca59219b86354a442a95b0e,USER32.DLL
146d61fca77d748f5a5ecff53afd30e4,KERNEL32.DLL
146d61fca77d748f5a5ecff53afd30e4,COMCTL32.DLL
95a7a3e5ea764eed286b53623f9521ab,KERNEL32.DLL
2059abe419dfeca527b7cf5b53bbee6f,KERNEL32.DLL
005472c686a5f84ad8e2dea597f50e1d,KERNEL32.DLL
005472c686a5f84ad8e2dea597f50e1d,ADVAPI32.DLL
005472c686a5f84ad8e2dea597f50e1d,MPR.DLL
005472c686a5f84ad8e2dea597f50e1d,OLEAUT32.DLL
...
...

Regards