A directed graph of intranet SSH sessions as recorded by Argus, graphed using the "two node mode" of afterglow. Data collected with Argus V3.0 from multiple Cisco Netflow sources, graph generated using AfterGlow v1.5.7 and Neato v1.16, all running on OpenBSD.
The latest version of Argus can directly output CSV, so argus2csv is no longer needed. This particular graph was generated by the following:
racluster -r argus.cap -m saddr daddr dport -c, -s saddr daddr - 'tcp and dst port 22' | kevin-anonymize.pl | afterglow.pl -t -e 2 -c test.properties | neato -Tgif -o tcp22argus.gif
This directed graph reminds me of the social network you might see in a suburban high school, and revealed to us some interesting things, including the existence of a new network monitoring tool quietly installed by a rogue internal unix admin team... us and them, we're having a "come to Jesus" meeting tomorrow ;)
The DHS just released a solicitation for various security-related research projects among them TTA 4 - Network Data Visualization for Information Assurance. I am very pleased that the DHS puts visualization as one of their nine main concerns.
I am somewhat concerned with the solicitation however. They mention SiLK as one of the tool sets which the US-CERT uses a lot. And they would like to see visualization tools enhacing that suite. I am not sure that's the right thing to do. I think we need tools which do not just look at traffic flow information, but at all kinds of different data sources!
I am very curious what type of tools and solutions will be submitted for this and would love to see some advances and new approaches. Anyone going to submit?
So what are the benefits of visualization over other techniques? My favorite answer is this:
There are many more benefits to visualization. Here are just a few:
I am curious what other's think. Let's add to the list!
The National Science Foundation (NSF) has a challenge for science and engineering visualizations published. I am not sure if I have some visualizations that would qualify for the challenge. But maybe some of you have security data that could make the bar. I think it would be great to draw attention to visualization in the security space. So if you have something. Submit it!
The red pillar in the image shows the barrage of HTTP requests over the whole content space (Z axis, vertical) from a single IP address (Y axis, horizontal). The red color is due to 5xx status code of the response. My article
A New, Improved Visualization for Web Server Logs has more details. Raju Varghese (raju -at- intellisoft.ch).
On his blog, Anton started an entry about logging and gets into the topic of too many logs. I was suggesting visualization to analyze the vast amounts of logs in order to get a better handle/understanding of them. Anton countered with this:
Is this really the place to start a visualization fight? :-) You know what my issue with visualization are: - tools need really skilled analysts - often the resulting picture is no more insightful than the original log pile - I kinda prefer an analytic system which is smart to a visualization system which is... not so smart. Let's move this discussion to secviz :-)Here ya go. To answer Anton's objections:
Another excellent paper or in this case a survey. The authors do a great job of surveying the space of structured data visualization. They explain very well what graph layouting is, what the different algorithms are, where the problems are hidden, what the solutions are, how interaction plays into all of this, and also discuss three-dimensional views and what their benefit is. Awesome survey, really worth reading if you are interested in graph layouts.
I read a fantastic paper on visual perception. A must read for everyone designing visual systems. The paper is called Perception in Visualization, written by Christopher G. Healey. The paper is very very practical. It presents the theory behind perception very well and always gives examples. Some of the topics covered are:
It is fairly interesting to see how security prodcuts are maturing. In the last couple of years I have seen quite some progress in products using visualization. Let's look back a few years. Network-based IDSs, for example, logged events in a log file; text [and some still do!]. Over time, reporting was added; a way to summarize historical data. Drop a pie-chart on the report and you have something that you can hand to your collegues. Shortly after that dashboards came about. Finally we had something to show to our managers, not just our peers. Most products have a dashboard today. Not all of them are very useful, but at least they have one ;) The next evolutionary step was to link the dashboards with the data itself. Drill-down was added.
And this is where we are today. Most products are at this stage. Only a few products took this a steps further. They added for example dashboards that link to other dashboards, which show more specific information. Some products even offer customizable dashboards (not all do!). You have the capabilities to either build your own or change predefined ones.
There are only a handful of products in the security space which take visualization a bit more serious. Thos products offer visual interfaces which support dynamic queries [basically the capability to let you change/interact with the graphs on the fly.]. This is clearly how it should be. It gives the user the tools he needs to interact with the data.
I am very convinced that dynamic, interactive, visual interfaces are going to be added to more and more products. They are incredibly powerful and invalueable for data anlysis and representation!