ulogd2, advanced firewall logging2009.rmll.info/img/pdf/ulogd2.pdf · entry output filters eric...
TRANSCRIPT
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Ulogd2, Advanced firewall logging
Eric Leblond
INL172 rue de Charonne75011 Paris, France
RMLL 2009, July 8, Nantes
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 1/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Some words about me
NuFW main developper
INL co-founderNetfilter hacker
some kernel stuffuserspace libraryulogd2organizer of Netfilter Workshop 2008
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 2/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Netfilter logging history
At the beginning was syslog
Pre Netfilter days
Flat packet loggingOne line per packet
A lot of informationNon searchable
INPUT DROP IN=eth0 OUT= MAC=00:1a:92:05:ee:68:00:b0:8e:83:3b:f0:08:00 SRC=62.212.121.211 \DST=91.121.73.151 LEN=60 TOS=0x00 PREC=0x00 TTL=58 ID=35342 DF PROTO=TCP SPT=59261 \DPT=113 WINDOW=5440 RES=0x00 SYN URGP=0
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 3/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Netfilter logging history
Ulogd days
Netfilter introduces ULOG targetiptables -A INPUT -p tcp -j ULOG --ulog-prefix "bad packet"
Communication via a netlink socketSpecial type of socketused for kernel userspace bidirectionnal communication
Ulogd, a logging daemonSyslog and file outputSQL output: PGSQL, MySQL, SQLite
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 4/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Netfilter logging history
Linux 2.6.14: Netfilter userspace reloaded
Netfilter introduces NFnetlinkRewrote userspace interactionFor logging, queueing and connection trackingMultiple communication on a single netlink socket
Three new librarieslibnetfilter_queue: userspace decisionlibnetfilter_log: logginglibnetfilter_conntrack: connection tracking handling
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 5/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Netfilter logging history
Ulogd2: an ulogd generalisation
Interact with the new libraries
Rewrite of ulogdlibnetfilter_log
Packet loggingIPv6 readyFew structural modification
libnetfilter_conntrackConnection tracking loggingAccounting, loggingCompletely new
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 6/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
1 Introduction
2 Connection tracking
3 Ulogd2 Architecture
4 Using Ulogd2
5 Conclusion
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 7/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Some words about connection tracking
Stateful filtering
Original IP packet filter:Filter only on IP header fieldsHave no idea of the packet history
Stateful filtering is:follow the history of connection
Is packet part of an existing connection ?Is packet correct relatively to the protocol ?
to determine the validity of a packet
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 8/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Some words about connection tracking
Netfilter connection tracking
Netfilter maintains a connection tableValid for "all" protocols
For flow-oriented protocol: TCP, SCTPFor protocol without state: UDP
Support both IPv4 and IPv6
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 9/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Some words about connection tracking
Network Address Translation
Private Network can’t go to internet
Firewall has to modify packet to show its addressTwo way of seeing a connection
From insideFrom outside
Conntrack keep track of the correspondance
tcp 6 431996 ESTABLISHED src=192.168.1.131 dst=91.121.73.151 sport=52964 dport=22\packets=13 bytes=772 src=91.121.73.151 dst=192.168.1.131 sport=22 dport=52964 \packets=11 bytes=7548 [ASSURED] mark=0 secmark=0 use=1 \
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 10/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Some words about connection tracking
libnetfilter_conntrack: Connection tracking handling library
Interrogation:Connections listingRetrieve information about a connection
IP informationAccounting statistics
Modification:Create new entryChange or fix timeoutChange mark
Destruction
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 11/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Some words about connection tracking
Connection tracking events
Send all significative connection related events touserspace :
NEW: connection creationESTABLISHED: Switch from NEW to ESTABLISHEDconnectionDESTROY: connection destruction
Make possible to maintain a connection history inuserspace
Accounting information
NAT decision history
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 12/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Netfilter logging
Ulogd2, a modular daemon
Able to use multiple entriesSupport for packet loggingSupport for flow logging
And multiple outputText basedDB based
Plugin based architectureEntryOutputFilters
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 13/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Netfilter logging
Ulogd2, schema of architecture
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 14/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Netfilter logging
Packet logging
Compatible with old kernelIPv4 support:
ULOGNFLOG
IPv6 support:NFLOG only
Hardware information:Network interfacesHardware header
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 15/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Netfilter logging
Connection tracking event logging
libnetfilter_conntrack based
IPv4 and IPv6
Listen to eventsContains the two IP tuples
Orig IP headerReply IP header
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 16/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
From input to output
The stack concept
Workflow based configuration: stackChoose an inputDescribe treatment and transformation to applyChoose an output
Based on key value propagation trough the stack
stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMUstack=ct1:NFCT,mark1:MARK,ip2str1:IP2STR,pgsql2:PGSQL
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 17/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
From input to output
The stack concept: plugin
Each plugin has :Input keysOutput keys
Plugin structure# /opt/ulogd2/sbin/ulogd --info /opt/ulogd2/lib/ulogd/ulogd_filter_IP2STR.soName: IP2STRInput keys:
Key: oob.family (unsigned int 8)Key: oob.protocol (unsigned int 16)Key: ip.saddr (IP addr)Key: ip.daddr (IP addr)[...]
Output keys:Key: ip.saddr.str (string)Key: ip.daddr.str (string)[...]
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 18/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
From input to output
Ulogd2, the stack concept
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 19/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
From input to output
Ulogd2, the stack concept
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 20/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
From input to output
Various output plugin
File-basedSyslogFilePCAP
DatabasesPGSQLMySQLSqlite (TODO)
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 21/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
From input to output
Treatment and filtering
Treatment plugins:Decoding plugins: BASE, IFINDEXConversion plugins: IP2STR, IP2BIN, MAC2STR
Filtering:Decide if treatment has to be continuedMARK plugin: stop propagation through stack if there is nomatch on mark
Multiplexing:Reusing INPUT dataMultiple logging
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 22/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
SQL output
Really use databases
Let dababase work to the databaseUse database capability
Procedure for insertionExtensible schemas
Optimize schemaAvoid empty fieldsIndex on most frequent request
Autoconfigurationulogd calls a procedureparams are taken from field name in a tableno need to recompile ulogd if we change the DB
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 23/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
SQL output
Easy modification
Procedure can do different things with dataProvided procedure
Insertion of all available data in DBFor connection trackingFor packet logging
Possible extensionArbitrary accountingStatistics
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 24/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
SQL output
Extensible database schemas
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 25/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
SQL output
Extensible database schemas
Easy to extendAdd table with your custom fieldlink ID of the new table with ulog2 ID.
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 26/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
SQL output
Use VIEW for usage ease
VIEW can be built for common task
TCP quad viewCREATE OR REPLACE VIEW view_tcp_quad ASSELECT ulog2._id,ulog2.ip_saddr_str,tcp.tcp_sport,
ulog2.ip_daddr_str,tcp.tcp_dportFROM ulog2 INNER JOIN tcp ON ulog2._id = tcp._tcp_id;
and provide easy select
TCP quad selectulog2=> SELECT ip_saddr_str,tcp_dport FROM view_tcp_quad;ip_saddr_str | tcp_dport---------------+-----------148.60.18.179 | 1194148.60.18.179 | 1194
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 27/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Packet logging
Security interest
Analysed dropped trafficAttack attemptScansWorms or trojan traffic
Analyse authorized trafficKeep a trace of access to critical dataForensic on succesful attackWork with other security subsystem
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 28/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Packet logging
Production interest
A firewall block some packetsPacket necessary for network servicesLogging is critical when setting up a new firewall
and detect misconfigurationPacket necessary for network servicesPackets revealing improper configuration of service
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 29/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Packet logging
Nulog: displaying packet data
Display SQL data (Ulogd1 format)Ulogd2 support in progress
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 30/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Connection event logging
Security interest
Advantages of logging flow over logging packetStart timeEnd timeVolume information
Better view of severity of the eventDuration informationData volumeNAT information
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 31/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Connection event logging
Recover internal IP from external data
Connection logging containsOrig IP tupleReply IP tuple
Someone from outside asks you information about anattack:
Extern world only knows the Reply tupleConnection logging lead you to the IP at the origin of anattack
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 32/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Connection event logging
Accounting
Each connection logging contains:bytes usagepacket usage
Summing usage lead you to global statisticUsing any IP criteria (per port or per IP bandwith)Or using external information (per user bandwith)
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 33/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
NF3D
NF3D
Data visualisation tryout
Represent both packet and connection on a graph
Link packet to their corresponding connection
Connections are displayed in a GANTT fashion
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 34/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
NF3D
NF3D: SSH brute force
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 35/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
NF3D
NF3D: Demonstration
Let’s pray Murphy.
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 36/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
A complete logging system
ulogd2 is a complete logging tool for NetfilterPacket loggingConnection logging
Easy to extendVia pluginVia database modification
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 37/ 38
Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion
Questions ?
Contacts:Directly: [email protected] List: [email protected]
References:Ulogd2:http://netfilter.org/projects/ulogd/index.htmlUlogd2 documentation:http://software.inl.fr/trac/wiki/ulogd2/userNulog:http://software.inl.fr/trac/wiki/EdenWall/NuLogNF3D: http://software.inl.fr/trac/wiki/nf3d
Eric Leblond INL 172 rue de Charonne 75011 Paris, France
Ulogd2, Netfilter logging reloaded 38/ 38