All done on CentOS6
There are scripts to automate this, but I like installing everything from source to learn more about the packages.
Here’s an example script along with a gui for SiLK rw commands.
FlowBat
Download the following source code files from the netsa CERT project home page
Fixbuf, netsa-python, silk, yaf, super_mediator, analysis_pipeline.
Install the prerequisite packages.
yum –y install python-devel glib2-devel gnutls gnutls-devel lzo lzo-devel libpcap libpcap-devel zlib zlib-devel pcre-devel mysql mysql-server mysql-devel
See what you got.
ls
analysis-pipeline-4.4.tar.gz libfixbuf-1.6.0.tar.gz netsa-python-1.4.3.tar.gz silk-3.9.0.tar.gz super_mediator-0.4.0.tar.gz yaf-2.6.0.tar.gz
Build and install libfixbuf first:
tar zxvf libfixbuf-1.2.0.tar.gz
cd libfixbuf-1.2.0
./configure && make && make install
Next install netsa-python library:
tar zxvf netsa-python-1.3.tar.gz
cd netsa-python-1.3
python setup.py install
Build and install yaf:
tar -xvzf yaf-2.4.0.tar.gz
cd yaf-2.4.0
./configure –enable-applabel –enable-plugins
make && make install
Build super_mediator:
tar -xvzf super_mediator-0.4.0.tar.gz
cd super_mediator-0.4.0
./configure –with-mysql && make && make install
Build and install SiLK:
tar zxvf silk-3.9.0.tar.gz
cd silk-3.9.0
./configure –with-libfixbuf=/usr/local/lib/pkgconfig –enable-ipv6
make && make install
Start mysqld
service mysqld start
chkconfig mysqld on
Setup a password for the root user
/usr/bin/mysqladmin -u root password ‘<SuperSecretPassword>’
Login to the database (It will prompt you for the password you created in the last step):
mysql -u root –p
Create the database you intend to use for super_mediator:
mysql> create database smediator;
Create a user for super_mediator to access the database:
mysql> CREATE USER ‘mediator’@’localhost’ IDENTIFIED BY ‘<SuperSecretPassword>’;
Giver permissions to user to access only the smediator database:
mysql> GRANT ALL ON smediator.* TO mediator@’localhost’;
Configure SiLK:
We will using /data as the location of our SiLK repository:
mkdir -p /data
We will be using the default silk.conf file so copy it to the repo now:
cp site/twoway/silk.conf /data
cp src/rwflowpack/rwflowpack.conf /usr/local/etc/rwflowpack.conf
cp src/rwflowpack/rwflowpack.init.d /etc/init.d/rwflowpack
chmod +x /etc/init.d/rwflowpack
To configure rwflowpack, edit /usr/local/etc/rwflowpack.conf
#/usr/local/etc/rwflowpack.conf
ENABLED=1
statedirectory=/var/lib/rwflowpack
CREATE_DIRECTORIES=yes
BIN_DIR=/usr/local/sbin
SENSOR_CONFIG=/data/sensor.conf
DATA_ROOTDIR=/data
SITE_CONFIG=/data/silk.conf
PACKING_LOGIC=
INPUT_MODE=stream
INCOMING_DIR=${statedirectory}/incoming
ARCHIVE_DIR=${statedirectory}/archive
FLAT_ARCHIVE=0
ERROR_DIR= #${statedirectory}/error
OUTPUT_MODE=local
SENDER_DIR=${statedirectory}/sender-incoming
INCREMENTAL_DIR=${statedirectory}/incremental
COMPRESSION_TYPE=
POLLING_INTERVAL=
FLUSH_TIMEOUT=
FILE_CACHE_SIZE=
FILE_LOCKING=1
PACK_INTERFACES=0
LOG_TYPE=syslog
LOG_LEVEL=info
LOG_DIR=${statedirectory}/log
PID_DIR=${LOG_DIR}
USER=root
EXTRA_OPTIONS=
We will need to create the Sensor configuration file sensor.conf to setup the listening probe. Change the internal-ipblocks to match your network
probe S0 ipfix
listen-on-port 18001
protocol tcp
end probe
sensor S0
ipfix-probes S0
internal-ipblocks 192.168.1.0/24 10.10.10.0/24
external-ipblocks remainder
end sensor
Move the sensor.conf to the repository:
mv sensor.conf /data
Start rwflowpack:
service rwflowpack start
chkconfig rwflowpack on
Verify that rwflowpack is listening on port 18001:
netstat –vnatpl
To use the SiLK command line tools, you need to set the SILK_DATA_ROOTDIR variable:
export SILK_DATA_ROOTDIR=/data
Using super_mediator
Create the file directories that super_mediator will use to write files that will eventually get imported into the MySQL Database.
mkdir -p /data/smediator/dpi
mkdir -p /data/smediator/dns
Use super_table_creator to create all the tables in your database:
/usr/local/bin/super_table_creator –name mediator –pass=<SuperSecretPassword> –database=smediator
/usr/local/bin/super_table_creator –name mediator –pass=<SuperSecretPassword> –database=smediator –dns-dedup
Create your super_mediator.conf file. One is installed by default into /usr/local/etc. The following one will get you started:
COLLECTOR TCP
PORT 18000
COLLECTOR END
#rwflowpack
EXPORTER TCP
PORT 18001
HOST localhost
FLOW_ONLY
EXPORTER END
#dedup process
EXPORTER TEXT
PATH “/data/smediator/dns/yaf2dns”
DELIMITER “|”
ROTATE 1200
DNS_DEDUP_ONLY
LOCK
MYSQL_USER “mediator”
MYSQL_PASSWORD “<SuperSecretPassword>”
MYSQL_TABLE “dns-dedup”
MYSQL_DATABASE “smediator”
EXPORTER END
#dpi 2 database
EXPORTER TEXT
PATH “/data/smediator/dpi”
ROTATE 1200
MULTI_FILES
DPI_ONLY
LOCK
MYSQL_USER “mediator”
MYSQL_PASSWORD “<SuperSecretPassword>”
MYSQL_DATABASE “smediator”
EXPORTER END
DNS_DEDUP
MAX_HIT_COUNT 5000
DNS_DEDUP END
LOGLEVEL DEBUG
LOG “/var/log/super_mediator.log”
PIDFILE “/data/super_mediator.pid”
Start super_mediator:
/usr/local/bin/super_mediator -c /usr/local/etc/super_mediator.conf –daemonize
Also add this to /etc/rc.local.
Confirm super_mediator is running:
ps -ef | grep super
If super_mediator is not running, check for any errors:
cat /var/log/super_mediator.log
Configure and Start YAF
mkdir -p /var/log/yaf/{log,run}
export LTDL_LIBRARY_PATH=/usr/local/lib/yaf
Edit /usr/local/etc/yaf.conf:
ENABLED=1
YAF_CAP_TYPE=pcap
YAF_CAP_IF=eth1
YAF_IPFIX_PROTO=tcp
YAF_IPFIX_HOST=localhost
YAF_IPFIX_PORT=18000
YAF_STATEDIR=/var/log/yaf
YAF_EXTRAFLAGS=”–silk –applabel –max-payload=2048 –plugin-name=/usr/local/lib/yaf/dpacketplugin.la”
Start yaf
Copy the yaf init script and enable.
cd ~/Downloads/yaf-2.6.0/etc/init.d
cp yaf /etc/init.d/
chmod +x /etc/init.d/yaf
chkconfig yaf on
service yaf start
Confirm Install and Sample Analysis
Confirm MySQL database contains data:
mysql -u root –p
use smediator;
select table_name, table_rows from information_schema.tables where table_schema = DATABASE();
+————+————+
| table_name | table_rows |
+————+————+
| dhcp | 0 |
| dnp3 | 0 |
| dns | 12915 |
| dns_dedup | 0 |
| dpi_id | 143 |
| enip | 0 |
| flow | 6637 |
| ftp | 0 |
| http | 57373 |
| imap | 0 |
| irc | 0 |
| modbus | 0 |
| mysql | 0 |
| nntp | 0 |
| p0f | 0 |
| pop3 | 0 |
| rtp | 0 |
| rtsp | 0 |
| sip | 0 |
| slp | 0 |
| smtp | 0 |
| ssh | 1 |
| tftp | 0 |
| tls | 6389 |
+————+————+
Confirm SiLK is creating flow records:
rwfilter –proto=0- –type=all –pass=stdout | rwcut | head
0 Comments