Add Post   Gallery
This is a community portal. Sign up on the left and start posting about analytics and visualization of security data.



EEVi- Evaluating the Effectiveness of Visualisation in Cyber-Security

Hi all,

I am a PhD researcher at University of Southampton and my PhD topic is Visualisation in Cyber Security.

I have a questionnaire, for my thesis, aimed at people who have experience in Cyber Security, Visualization (or HCI) design or both. I would really appreciate if you can take some time out and fill out the questionnaire.

Please refer to the link below for more information or contact me. :)

Thank you.
Aneesha Sethi

Security Analytics - Visualization - Big Data Workshop Black Hat 2017


BlackHat 2017 - Las Vegas

Big Data is Getting Bigger - Visualization is Getting Easier - Learn How!
Dates: July 22-23 & 24-25
Location: Las Vegas, USA



Big data and security intelligence are the two very hot topics in security. We are collecting more and more information from both the infrastructure, but increasingly also directly from our applications. This vast amount of data gets increasingly hard to understand. Terms like map reduce, hadoop, spark, elasticsearch, data science, etc. are part of many discussions. But what are those technologies and techniques? And what do they have to do with security analytics/intelligence? We will see that none of these technologies are sufficient in our quest to defend our networks and information. Data visualization is the only approach that scales to the ever changing threat landscape and infrastructure configurations. Using big data visualization techniques, you uncover hidden patterns of data, identify emerging vulnerabilities and attacks, and respond decisively with countermeasures that are far more likely to succeed than conventional methods. Something that is increasingly referred to as hunting. The attendees will learn about log analysis, big data, information visualization, data sources for IT security, and learn how to generate visual representations of IT data. The training is filled with hands-on exercises utilizing the DAVIX live CD.

What's New?

The workshop is being heavily updated over the next months. Check back here to see a list of new topics:

  • Security Analytics - UEBA, Scoring, Anomaly Detection
  • Hunting
  • Data Science
  • 10 Challenges with SIEM and Big Data for Security
  • Big Data - How do you navigate the ever growing landscape of Hadoop and big data technologies? Tajo, Apache Arrow, Apache Drill, Druid, PrestoDB from Facebook, Kudu, etc. We'll sort you out.


The syllabus is not 100% fixed yet. Stay tuned for some updates.

Day 1:

Log Analysis

  • Data Sources Discussion - including PCAP, Firewall, IDS, Threat Intelligence (TI) Feeds, CloudTrail, CloudWatch, etc.
  • Data Analysis and Visualization Linux (DAVIX)
  • Log Data Processing (CSVKit, ...)

SIEM, and Big Data

  • Log Management and SIEM Overview
  • LogStash (Elastic Stack) and Moloch
  • Big Data - Hadoop, Spark, ElasticSearch, Hive, Impala

Data Science

  • Introduction to Data Science
  • Introduction to Data Science with R
  • Hunting

Day 2:


  • Information Visualization History
  • Visualization Theory
  • Data Visualization Tools and Libraries (e.g., Mondrian, Gephi, AfterGlow, Graphiti)
  • Visualization Resources

Security Visualization Use-Cases

  • Perimeter Threat
  • Network Flow Analysis
  • Firewall Visualization
  • IDS/IPS Signature Analysis
  • Vulnerability Scans
  • Proxy Data
  • User Activity
  • Host-based Data Analysis

Sample of Tools and Techniques

Tools to gather data:

  • argus, nfdump, nfsen, and silk to process traffic flows
  • snort, bro, suricata as intrusion detection systems
  • p0f, npad for passive network analysis
  • iptables, pf, pix as examples of firewalls
  • OSSEC, collectd, graphite for host data

We are also using a number of visualization tools to analyze example data in the labs:

  • graphviz, tulip, cytoscape, and gephi
  • afterglow
  • treemap
  • mondrian, ggobi

Under the log management section, we are going to discuss:

  • rsyslog, syslog-ng, nxlog
  • logstash as part of the elastic stack, moloch
  • commercial log management and SIEM solutions

The section on big data is covering the following:

  • hadoop (HDFS, map-reduce, HBase, Hive, Impala, Zookeper)
  • search engines like: elastic search, Solr
  • key-value stores like MongoDB, Cassandra, etc.
  • OLAP and OLTP
  • The Spark ecosystem



Raffael Marty is vice president of security analytics at Sophos, and is responsible for all strategic efforts around security analytics for the company and its products. He is based in San Francisco, Calif. Marty is one of the world's most recognized authorities on security data analytics, big data and visualization. His team at Sophos spans these domains to help build products that provide Internet security solutions to Sophos' vast global customer base.

Previously, Marty launched pixlcloud, a visual analytics platform, and Loggly, a cloud-based log management solution. With a track record at companies including IBM Research, ArcSight, and Splunk, he is thoroughly familiar with established practices and emerging trends in the big data and security analytics space. Marty is the author of Applied Security Visualization and a frequent speaker at academic and industry events. Zen meditation has become an important part of Raffy's life, sometimes leading to insights not in data but in life.

Secuirty Visualization Case Study

We recently posted a case study of how a Fortune 100 company is using Security Visualization as a front end to their various data collection systems. The Security Visualization allows the company's analysts to look at 100's of thousands of correlations each day and apply human pattern recognition to spot the "needles in the haystack". These are threats that are designed to avoid traditional intrusion and event management. Once the potential threat is identified and the log data is carved down to just the logs that are relevant, that subset of log data is then attached to a case study and delivered to case investigation for further evaluation. In addition to identifying and carving down to just the relevant logs, the security visualization also makes it easier to communicate the findings to the extended team.

In this situation data is imported from several sources. Those sources include intrusion detection systems (e.g., SourceFire), firewall protection (e.g., Palo Alto, SonicWALL), and virus scan / endpoint protection (e.g. Symantec) in addition to correlation systems (e.g., HP ESM, Splunk, etc.). Security Visualization allows the analysts to hunt for unknown and unexpected threats. Threats such as time staged attacks, diagonal attacks, cluster attacks, octal jump attacks, embedded activity attacks, etc.

This case study is recorded and can be viewed at The case study lasts 25 minutes, and is followed by a Q&A.

Doug Cogswell,
ADVIZOR Solutions, Inc.

Security Visualization Requirements- Phd Thesis Survey

Hi, I am a Phd candidate in Informatics Institude at Middle East Technical University. I prepared an online survey as a part of my phd thesis. However, since this subject is relatively new I can not find anybody who may fill this survey around me in Turkey.

The survey is in Google Forms, at link It is not very short :( It may take around 20 minutes but it is easy to fill, mostly composed of multi selection questions. Uncompleted survey results are not saved so the participants should complete the survey.

Although we ask questions related to security systems and security visualization systems used to understand the visualization requirements.The survey, in general, does not include questions that give personal discomfort. No tracking information such as email or organization name is asked during the survey. More descriptive information about how the survey results will be used exists in the starting page. So, please do not hesitate to fill, due to your privacy concerns.

I hope experts of this forum may help me by filling the survey during a coffee break. I need to take feedback soon, before my next thesis committee. I appreciate your help to a newbie security visualization researcher (me) :)

Many thanks,
Ferda Özdemir Sönmez

VizSec 2016 Call for Papers

The 13th IEEE Symposium on Visualization for Cyber Security (VizSec) is a forum that brings together researchers and practitioners from academia, government, and industry to address the needs of the cybersecurity community through new and insightful visualization and analysis techniques. VizSec provides an excellent venue for fostering greater exchange and new collaborations on a broad range of security- and privacy-related topics. VizSec will be held in Baltimore, MD, USA in conjunction with IEEE VIS.

The purpose of VizSec is to explore effective and scalable visual interfaces for security domains such as network security, computer forensics, reverse engineering, insider threat detection, cryptography, privacy, user assisted attacks prevention, compliance management, wireless security, secure coding, and penetration testing.

Technical Papers

Full papers describing novel contributions in security visualization are solicited. Papers may present techniques, applications, practical experience, theory, analysis, experiments, or evaluations. We encourage the submission of papers on technologies and methods that promise to improve cyber security practices, including, but not limited to:

- Situation awareness and/or understanding
- Incident handling including triage, exploration, correlation, and response
- Computer forensics
- Recording and reporting results of investigations
- Assisting proactive security configuration and deployment
- Reverse engineering and malware analysis
- Vulnerability management
- Multiple data source analysis
- Analyzing information requirements for computer network defense
- Evaluation and/or user testing of VizSec systems
- Criteria for assessing the effectiveness of cyber security visualizations
(whether from a security goal perspective or a human factors perspective)
- Modeling system and network behavior
- Modeling attacker and defender behavior
- Studying risk and impact of cyber attacks
- Predicting future attacks or targets
- Security metrics and education
- Software security
- Mobile application security
- Social networking privacy and security

When applicable, visualization and interaction techniques that effectively capture the insights of human analysts and/or allow analysts to collaborate efficiently are particularly desirable.

*** New for 2016! *** Case Studies

Short papers describing practical applications of security visualization are solicited. We encourage the submission of papers discussing the introduction of cyber security visualizations into operational context, including, but not limited to:

- Cases where visualization made positive contributions towards meeting
operational needs
- Gaps or negative outcomes from visualization deployments
- Situations where visualization was not utilized, but could have had a
positive impact
- Lessons learned from operational engagements
- Insights gained from the transition process

Cyber security practitioners from industry, as well as the research community, are encouraged to submit case studies.


Poster submissions may showcase late-breaking results, work in progress, preliminary results, or visual representations relevant to the VizSec community. The poster program will be a great opportunity for the authors to interact with the attendees and solicit feedback.


Submissions must be formated using the IEEE VGTC template that can be found at All submissions should be in PDF format.

Submit papers and poster abstracts using EasyChair:

Papers should be at most 8 pages including the bibliography and appendices. Papers will be peer-reviewed by at least 3 members of the program committee. Committee members are not required to read the appendices or any pages past the maximum. Submissions not meeting these guidelines will be rejected without consideration of their merit. Reviews are single-blind, so authors may include names and affiliations in their submissions. Submitted papers must not substantially overlap papers that have been published or that are simultaneously submitted to a journal or a conference with proceedings.

The VizSec proceedings will be published by IEEE. Authors of accepted papers must guarantee that their papers will be presented at the conference.
Case Studies

Case studies should be at most 4 pages including the bibliography and appendices. Case study submissions will be reviewed by the Paper Chair(s) and other members of the organizing committee to determine relevance to the VizSec community.

Accepted case study authors will have time to present their work at VizSec during the program.

Accepted case studies will be made available on this website.

Extended abstract for posters should be at most 2 pages including the bibliography. Poster abstracts will be reviewed by the Poster Chair(s) and other members of the organizing committee to determine relevance to the VizSec community.

Accepted authors must present a corresponding poster during the workshop. The poster authors can determine the layout by themselves, but the dimensions of the posters should not exceed the A0 space (841mm x 1189mm or 33.1" x 46.8"). Additionally, poster authors are requested to give a brief oral preview during a plenary "fast forward" session.

Accepted poster abstracts will be made available on VizSec website.

When applicable, submissions including tests and evaluations of the proposed tools and techniques are considered particularly desirable. If possible, making the data used for the tests available will also be considered positively. If you do not have real-world data to demonstrate your visualization, you may be interested in looking at the VAST Challenge data sets.

Important Dates

All deadlines are 5:00 PM PST

Papers and Case Studies

August 1, 2016
Submission for Papers and Case Studies
September 5, 2016
Author Notification for Papers and Case Studies
October 3, 2016
Camera Ready Submission and Copyright Forms for Papers


September 19, 2016
Abstract Submission for Posters
September 30, 2016
Author Notification for Posters

Visualizing Live Streams in 3D/VR

We've created a free tool for visualizing live streams of network traffic, using JMonkeyEngine (Java 3D gaming engine).

Please take a look at - we would very much appreciate feedback from this community.

Rather than focusing on mining of static datasets, this tool focuses on seeing activity over time, and controlling the timeline so that a human can connect the dots. Here's a link to information on the concept behind the visualization style.

As for the screenshot, this video explains what you're looking at.

screenshot of ip traffic in deep node

screenshot of ip traffic in deep node

screenshot of ip traffic in deep node

Youtube video using Afterglow, twopi and Nginx logs.

I attended Visual Analytics Workshop last year at BlackHat and have gotten endless use from afterglow, neato, etc to make interesting visualizations.

Here is a short youtube video I put together, with attack data taken from Nginx logs:

(Music is by a local San Francisco band: Vetiver)

DNS Mapping

DNS Mapping

Over at I created a DNS recon tool that generates a DNS map on the fly using 80+GB of DNS data from the project. This map is the domain.

MyDoom botnet

MyDoom botnet

This graph visualization shows the propagation of malware through a deliberately infected computer network. Twelve machines in the network were infected to see how the traffic spread to other machines. Over 7800 machines were included in the dataset.
All network in a single chart. Yellow links indicate benign traffic; red links indicate traffic with at least 1 infected packet. Nodes are sized by volume of traffic.
Data taken from the MyDoom-A.tar.gz, available here
Image generated with KeyLines.