Security Analytics - Visualization - Big Data Workshop Black Hat 2017

OVERVIEW

Big data and security intelligence are the two very hot topics in security. We are collecting more and more information from both the infrastructure, but increasingly also directly from our applications. This vast amount of data gets increasingly hard to understand. Terms like map reduce, hadoop, spark, elasticsearch, data science, etc. are part of many discussions. But what are those technologies and techniques? And what do they have to do with security analytics/intelligence? We will see that none of these technologies are sufficient in our quest to defend our networks and information. Data visualization is the only approach that scales to the ever changing threat landscape and infrastructure configurations. Using big data visualization techniques, you uncover hidden patterns of data, identify emerging vulnerabilities and attacks, and respond decisively with countermeasures that are far more likely to succeed than conventional methods. Something that is increasingly referred to as hunting. The attendees will learn about log analysis, big data, information visualization, data sources for IT security, and learn how to generate visual representations of IT data. The training is filled with hands-on exercises utilizing the DAVIX live CD.

What's New?

The workshop is being heavily updated over the next months. Check back here to see a list of new topics:

• Security Analytics - UEBA, Scoring, Anomaly Detection
• Hunting
• Data Science
• 10 Challenges with SIEM and Big Data for Security
• Big Data - How do you navigate the ever growing landscape of Hadoop and big data technologies? Tajo, Apache Arrow, Apache Drill, Druid, PrestoDB from Facebook, Kudu, etc. We'll sort you out.

SYLLABUS

The syllabus is not 100% fixed yet. Stay tuned for some updates.

Day 1:

Log Analysis

• Data Sources Discussion - including PCAP, Firewall, IDS, Threat Intelligence (TI) Feeds, CloudTrail, CloudWatch, etc.
• Data Analysis and Visualization Linux (DAVIX)
• Log Data Processing (CSVKit, ...)

SIEM, and Big Data

• Log Management and SIEM Overview
• LogStash (Elastic Stack) and Moloch
• Big Data - Hadoop, Spark, ElasticSearch, Hive, Impala

Data Science

• Introduction to Data Science
• Introduction to Data Science with R
• Hunting

Day 2:

Visualization

• Information Visualization History
• Visualization Theory
• Data Visualization Tools and Libraries (e.g., Mondrian, Gephi, AfterGlow, Graphiti)
• Visualization Resources

Security Visualization Use-Cases

• Perimeter Threat
• Network Flow Analysis
• Firewall Visualization
• IDS/IPS Signature Analysis
• Vulnerability Scans
• Proxy Data
• User Activity
• Host-based Data Analysis

Sample of Tools and Techniques

Tools to gather data:

• argus, nfdump, nfsen, and silk to process traffic flows
• snort, bro, suricata as intrusion detection systems
• p0f, npad for passive network analysis
• iptables, pf, pix as examples of firewalls
• OSSEC, collectd, graphite for host data

We are also using a number of visualization tools to analyze example data in the labs:

• graphviz, tulip, cytoscape, and gephi
• afterglow
• treemap
• mondrian, ggobi

Under the log management section, we are going to discuss:

• rsyslog, syslog-ng, nxlog
• logstash as part of the elastic stack, moloch
• commercial log management and SIEM solutions

The section on big data is covering the following:

• hadoop (HDFS, map-reduce, HBase, Hive, Impala, Zookeper)
• search engines like: elastic search, Solr
• key-value stores like MongoDB, Cassandra, etc.
• OLAP and OLTP
• The Spark ecosystem

TRAINER

Raffael Marty is vice president of security analytics at Sophos, and is responsible for all strategic efforts around security analytics for the company and its products. He is based in San Francisco, Calif. Marty is one of the world's most recognized authorities on security data analytics, big data and visualization. His team at Sophos spans these domains to help build products that provide Internet security solutions to Sophos' vast global customer base.

Previously, Marty launched pixlcloud, a visual analytics platform, and Loggly, a cloud-based log management solution. With a track record at companies including IBM Research, ArcSight, and Splunk, he is thoroughly familiar with established practices and emerging trends in the big data and security analytics space. Marty is the author of Applied Security Visualization and a frequent speaker at academic and industry events. Zen meditation has become an important part of Raffy's life, sometimes leading to insights not in data but in life.