Defining Flows

Flows can be defined using REST API /flow/ or using JavaScript setFlow() function.

See RESTflow for an overview and examples.

Flow Definition Attributes
AttributeDefaultExampleComment
namepairName used to identify flow specification
keysipsource,ipdestinationList of Flow Keys or Key Functions
valuebytesNumeric Flow Key, e.g. bytes, frames, requests, duration
valuesframesList of additional values
filteripsource=10.0.0.1Boolean expression filtering Flow Keys or Key Functions using comparison operators: equals(=), not equals(!=), match(~), not match(!~), brackets ( ), and logical operators and(&), or(|). The left hand side of a comparision operation references a flow key or key functions applied to flow key(s). The right hand side is a comma separated list of values to test. Match operations are tested against a list of Pattern strings.
n510Number of largest flows to maintain (i.e. the n in "top n"). Values < 5 or > 20 will be adjusted to fall within the allowed range
t210Smoothing factor (in seconds). Values < 0.2 or > 600 will be adjusted to fall within the allowed range.
droppedfalsetrueAnalyze dropped packet notifications, otherwise packet samples
fs,_SEP_String used to separate flow record fields
logfalsetrueLog flows for Script and REST API access
flowStartfalsetrueLog start of flow, otherwise record end of flow
activeTimeout6010Number seconds before flushing active flow
ipfixCollectors10.0.0.1Send flows as IPFIX messages to specified list of collectors - only the following subset of keys is allowed, macsource, macdestination, ethernetprotocol, vlan, priority, ipprotocol, ipsource, ipdestination, ip6source, ip6destination, ip6nexthdr, tcpsourceport, tcpdestinationport, udpsourceport, udpdestinationport, inputifindex, outputifindex and the following values, bytes, frames.
Key Functions

Functions of the form <funcname>:<arg1>:<arg2>... can be applied used to define flow keys or filter. Functions can be nested using square brackets [] as a delimeter, e.g. prefix:[uri:httpurl:path]:/:2 applied to the URL http://localhost:8008/metrics/json would return /metrics.

FunctionExampleArgumentsComment
groupgroup:ipsource:defaultgroup:<flowkey>:<group1>:<group2>One or more group lookups are applied in order until there is a match
countrycountry:ipsourcecountry:<flowkey>:<attr>Returns country code for address where attr is code, name, or both
Note: Set System Property geo.country
asnasn:ipsourceasn:<flowkey>:<attr>Returns Autonomous System Number (ASN) for address where attr is number, descr, or both
Note: Set System Property geo.asn
ouioui:macsourceoui:<flowkey>:<attr>Returns Organizationally Unique Identifier (OUI) for MAC address where attr is number, or name
Note: Set System Property oui.names
dnsdns:ipsourcedns:<flowkey>Returns FQDN for address
Note: Set System Property dns.servers
hosthost:macsource:uuidhost:<flowkey>:<attr>Where attr is host_name, machine_type, os_name, uuid, os_release, vir_host_name, vir_machine_type, vir_os_name, vir_uuid, or vir_os_release
prefixprefix:ipsource:.:2prefix:<flowkey>:<delim>:<n>
suffixsuffix:ipsource:.:2suffix:<flowkey>:<delim>:<n>
concatconcat:_:ipsource:tcpsourceportconcat:<delim>:<flowkey1>:<flowkey2>..Joins keys using delimiter
maskmask:ipsource:24mask:<flowkey>:<bits>Appy CIDR mask to address and return network/bits
nullnull:vlan:undefinednull:<flowkey>:<val>Allow null values as keys
oror:ipsource:ip6sourceor:<flowkey1>:<flowkey2>Return first non-null key
eqeq:ipsource:ipdestinationeq:<flowkey1>:<flowkey2>Return true if keys are equal
rangerange:tcpsourceport:0:1023range:<flowkey>:<lower>:<upper>Returns true if value in range
mapmap:vlan:vlan_namesmap:<flowkey>:<map1>:<map2>One or more map lookups are applied in order until there is a match
hashhash:ipsource:ipdestinationhash:<flowkey>:<flowkey>...Computes a hash across the set of keys
uriuri:httpurl:pathuri:<flowkey>:<attr>Where attr is normalized, scheme, user, authority, host, port, path, file, extension, query, fragment, isabsolute, isopaque. See URI
attrattr:appattributes:locattr:<flowkey>:<name>:<case>Where name is a field in an HTTP query string, e.g. "cc=visa&loc=mobile", and setting the case flag to true indicates a case sensitve lookup.
lclc:appnamelc:<flowkey>Convert value to lowercase
ucuc:appnameuc:<flowkey>Convert value to uppercase
nodenode:inputifindexnode:<inputifindex|outputifindex>Returns node name. Requires a topology.
linklink:inputifindexlink:<inputifindex|outputifindex>Returns link name. Requires a topology.
ifnameifname:inputifindexifname:<inputifindex|outputifindex>Returns port name.
ifif:tcpdirection:received:ipsource:ipdestinationif:<flowkey_cmp>:<value>:<flowkey_true>:<flowkey_false>Selects key based on condition.
firstfirst:stack:.:ip:ip6first:<flowkey>:<delim>:<val1>:<val2>...Return the value that occurs first in the delimited list
bitsbits:ip6flowlabel:1032703bits:<flowkey>:<mask>:<shift>Mask and shift bits in numeric flowkey
Value Functions

The following prefixes can be used to modify the way that the value field is computed:

FunctionExampleArgumentsComment
raterate:bytesrate:<flowkey>Compute per second rate for value, e.g. bytes/second
avgavg:durationavg:<flowkey>Compute time average for value
minmin:transitdelaymin:<flowkey>Compute minimum value over interval
maxmax:queuedepthmax:<flowkey>Compute maximum value over interval
countcount:ipsourcecount:<flowkey>Compute number of discrete values of key

Note: If no value function is specified then the default rate: function is applied. The smoothing factor, t, controls the interval over which the value function is applied.

Flow Keys

The following flow keys are supported by sFlow-RT. However, the subset of available keys will depend on the sources of sFlow data and can be queried using REST API /flowkeys/json or JavaScript flowKeys() function. Additional flow keys can be defined using Custom metrics.

Field names may have a numeric suffix indicating that one or more instances of the same attribute exist within packets. For example, ipsource.1, indicates that an inner IP source address exists within a tunnel, see Down the rabbit hole.

Note: A special flow key stack captures the layers decoded from the packet. For example, a stack value of eth.ip.udp.vxlan.eth.ip.tcp indicates that the packet was sampled from a VxLAN tunnel and has inner and outer Ethernet and IP layers.

Forwarding Information

Forwarding information associated with sampled packet, see sFlow Version 5

NameExampleComment
inputifindex12SNMP ifIndex of ingress port, or "local" if packet originated from device
outputifindex34SNMP ifIndex of egress port, or "multiple" if packet copied to more than one egress port, or "internal" if packet delivered to device, or "discard" if packet discarded
outputifcount3Indicates number of egress ports when outputifindex=multiple
outputdiscardreasonttl_exceededIndicates reason for packet discard if outputifindex=discard (see sFlow Version 5 page 27)
directioningressPacket direction with respect to port that sampled it
vlansource4VLAN on ingress
vlandestination1000VLAN on egress
prioritysource3Layer 2 priority on ingress
prioritydestination5Layer 2 priority on egress
ipnexthoprouter10.0.0.254Next hop router
ip6nexthoprouterFD03::102Next hop IPv6 router
ipsourcemaskbits24Mask bits for source subnet
ipdestinationmaskbits24Mask bits for destination subnet
bgpnexthop10.0.0.254BGP next hop
bgpnexthop6FE03::102BGP next hop IPv6
bgpas123Autonomous System Number (ASN) of reporting router
bgpsourceas123ASN associated with source address
bgpsourcepeeras123ASN associated with peer from which packet was received
bgpdestinationas123ASN associated with destination address
bgpdestinationpeeras123ASN associated with next hop peer
bgpdestinationaspath123-345-456Sequence of AS numbers for selected route
bgpcommunities234:32-234:666Communities associated with selected route
bgplocalpref3Local preference associated with selected route
natsrc10.0.0.254Translated source address
natsrc6FE03::102Translated IPv6 source address
natsrcport234Translated source TCP/UDP port
natdst10.0.0.254Translated destination address
natdst6FE03::102Translated IPv6 destination address
natdstport234Translated destination TCP/UDP port
usersrcpeterUser ID associated with packet source
userdstpeterUser ID associated with packet destination
url/index.htmlHTTP request line
urldirectionsourceDirection of connection
urlhostsflow.orgThe Host field from the HTTP header
mplslabelstackin1-2-4Label stack of received packet
mplslabelstackout1-2-4Label stack for transmitted packet
mplsnexthoprouter10.0.0.254MPLS next hop
mplsnexthoprouter6FE03::102MPLS IPv6 next hop
mplstunnellspnametun1Tunnel name
mplstunnlid123Tunnel ID
mplstunnelcos3Tunnel COS value
mplsvcinstancenamevc1VC instance name
mplsvcvllid3VLL/VC instance ID
mplsvclabelcos3VC label COS value
mplsftndescrftn1See MPLS-FTN-STD-MIB mplsFTNTable
mplsftnmask10See MPLS-FTN-STD-MIB mplsFTNTable
mplsfecaddrprefixlen10See MPLS-LDP-STD-MIB mplsFecTable
vlantunnelstack2-3-4
Dropped Packets

See sFlow Dropped Packet Notification Structures

NameExampleComment
reasonttl_exceededReason for dropping packet
queueindex7Eqress queue number selected for sampled packet
aclnumber3Access list number
aclnameddosAccess list name
acldirectioningressingress, egress, or unknown
functiontcp_v4_rcvName of the function in software network stack that discarded the packet
function_fulltcp_v4_rcv+0x7c/0xef0Name and address of the function in software network stack that discarded the packet
linux_drop_reasonNO_SOCKETLinux dropreason for discarding packet
hw_trap_groupl3_dropsLinux Devlink Trap group
hw_trap_nameblackhole_routeLinux Devlink Trap name
Transit Delay

See sFlow Transit Delay Structures

NameExampleComment
transitdelay839660224Delay for sampled packet to traverse switch (nanoseconds)
queuedepth11354112Depth seen by sampled packet in egress queue identified by queueindex (bytes)
WiFi Transmit / Receive

See sFlow 802.11 Structures

NameExampleComment
ciphersuite1
ciphersuiteoui000fac
ciphersuitetype2
ssidSSID string
bssidBSSID
wifiversiongversion
wifichannel3channel number
rsnireceived signal to noise ratio
rcpireceived channel power
speed
duration
transmissions2number of transmissions for sampled packet
retransduration
power
occupancy% radio occupancy
Tunnel Encap/Decap

See sFlow Tunnel Structures

NameExampleComment
outputmacdestinationdestination MAC of encapsulation
outputmacsourcesource MAC of encapsulation
outputethernetprotocolEthernet protocol of encapsulation
inputmacdestinationdestination MAC of encapsulation
inputmacsourcesource MAC of encapsulation
inputethernetprotocolEthernet protocol of encapsulation
inputiptos
inputipecn
inputipdscp
inputipdscpname
inputipsource
inputipdestination
inputip6source
inputip6destination
inputtcpsourceport
inputtcpdestinationport
inputtcpflags
inputudpsourceport
inputudpdestinationport
outputiptos
outputipecn
outputipdscp
outputipdscpname
outputipsource
outputipdestination
outputip6source
outputip6destination
outputtcpsourceport
outputtcpdestinationport
outputtcpflags
outputudpsourceport
outputudpdestinationport
outputheaderoffset
inputheaderoffset
outputvni
inputvni
Ethernet
NameExampleComment
eth_offset0Ethernet header offset from start of packet
macsource003EE1C6DCCAsource address
macdestination984BE1034A61destination address
isbroadcastfalse
ismulticastfalse
isunicasttrue
vlan200
priority0
llcbytes50
llcssap170
llcdsap170
llcctl3
llcsnapoui0000C
llcsnapprotocol267
ethernetprotocol2048
WiFi
NameExampleComment
wifi_offset0WiFi header offset from start of packet
wifitype
wifisubtype
wififlags
macreceiver
mactransmitter
bssid
PBB
NameExampleComment
pbb_offsetPBB header offset from start of packet
pbbflags
pbbisid
VN-TAG
NameExampleComment
vntag_offsetVN-TAG header offset from start of packet
vntagdir
vntagvifsrc
vntagvifdst
vntaglooped
vntagreserved
vntagversion
TRILL
NameExampleComment
trill_offset18TRILL header offset from start of packet
trilloptions0
trillhops62
trillbridgeout3
trillbridgein6
MPLS
NameExampleComment
mpls_offsetMPLS header offset from start of packet
mplslabels
LLDP
NameExampleComment
lldp_offsetLLDP header offset from start of packet
lldpchassisphysalias
lldpchassisifalias
lldpchassisport
lldpchassismac
lldpchassisifname
lldpchassislocal
lldpportifalias
lldpportphysalias
lldpportmac
lldpportifname
lldpportcircuitid
lldpportlocal
lldpttl
lldpportdescr
lldpsysname
lldpsysdescr
lldpcapabilities
lldpcapabilitiesenabled
ATA over Ethernet
NameExampleComment
aoe_offsetAoE header offset from start of packet
aoeversion
aoetargete1.4
aoeatacmdread
aoeoperationquery
requests1number of requests
Fiber Channel over Ethernet
NameExampleComment
fc_offsetFCoE header offset from start of packet
fcsource
fcdestination
fctype
scsiop8
scsiopnameREAD6
requests1number of requests
Audo Video Bridging - Transport Protocol (AVTP)
NameExampleComment
avtp_offsetAVTP header offset from start of packet
avtpsubtype
avtpversion
avtpstreadid
avtpcontroldatalen
avtpseqno
avtpstreamdatalen
avtptimestamp
avtpgateway
IEC 61883 over AVTP
NameExampleComment
iec61883_offsetIEC61883 header offset from start of packet
iec61883tag
iec61883channel
iec61883sid
iec61883dbs
iec61883fmt
InfiniBand Global Routing
NameExampleComment
ibgr_offsetIBGR header offset from start of packet
ibgripver6IP Version
ibgrtclass0traffic class
ibgrlabel0flow label
ibgrpaylen4112payload length
ibgrnxthdr27next header
ibgrhoplimit64hop limit
ibgrsgid0000:0000:0000:0000:0000:ffff:0a0a:0216source GID
ibgrdgid0000:0000:0000:0000:0000:ffff:0a0a:0216destination GID
InfiniBand Base Transport
NameExampleComment
ibbt_offsetIBBT header offset from start of packet
ibbtopcode1opcode
ibbtver0transport Header Version
ibbtdestqp345destination QP
ibbtackfalseacknowledge Request
ibbtoptransportRCtransport Type
ibbtopnameRDMA_WRITEoperation Name
ARP
NameExampleComment
arp_offset18ARP header offset from start of packet
arphardwaretype1
arpprotocoltype2048
arpoperation1
arpmacsenderAC87A30F1323
arpipsender10.0.0.1
arpmactarget0026BB6C1EB0
arpiptarget10.0.0.2
ICMP

See ICMP unreachable and Exporting events using syslog

NameExampleComment
icmp_offset34ICMP header offset from start of packet
icmptype3message type, e.g. Destination Unreachable (3)
icmpcode2message code, e.g. Protocol Unreachable (2)
icmpseqno13sequence number
icmpunreachablenet10.0.0.1IP address in network unreachable response
icmpunreachablehost10.0.0.1IP address in host unreachable response
icmpunreachableprotocol41protocol in protocol unreachable response
icmpunreachableportudp_30000port in port unreachable response
IP
NameExampleComment
ip_offset14IP header offset from start of packet
iptos00000000type of service bits
ipecn11explicit congestion notification bits
ipdscp0differentiated services code point
ipdscpnamebe(0)differentiated services code point name
ipttl63time to live
ipprotocol17protocol
ipbytes54payload bytes
ipid5210identification
ipflags010flags
ipfragoffset0fragment offset
ipsource10.0.0.1source address
ipdestination10.0.0.2destination address
ICMP version 6
NameExampleComment
icmp6_offset62ICMP6 header offset from start of packet
icmp6type3message type, e.g. Destination Unreachable (3)
icmp6code2message code, e.g. Protocol Unreachable (2)
icmp6seqno13sequence number
icmp6mldmaxrespdelay
icmp6mldaddress
icmp6racurhoplimit
icmp6ramanagedconfig
icmp6raotherconfig
icmp6ralifetime
icmp6rareachabletime
icmp6raetranstimer
icmp6nstarget
icmp6narouter
icmp6nasolicit
icmp6naoverride
icmp6natarget
icmp6redirecttarget
icmp6redirectdestination
icmp6unreachablenoroute
icmp6unreachableprohibited
icmp6unreachablebeyondscope
icmp6unreachableaddress
icmp6unreachablesourcepolicy
icmp6unreachablerejectroute
icmp6unreachableportudp_30000port in port unreachable response
IP version 6
NameExampleComment
ip6_offset14IPv6 header offset from start of packet
ip6tos01100000type of service bits
ip6ecn00explicit congestion notification bits
ip6dscp0differentiated services code point
ip6dscpnamebe(0)differentiated services code point name
ip6flowlabel501244flow label
ip6ttl63time to live
ip6sourceFE80::104C:51DF:4458:E00Asource address
ip6destinationFE80::A00:27FF:FEB8:326Ddestination address
ip6bytes60payload bytes
ip6extensions0list of next header values for extension headers
ip6fragoffset0fragment offset
ip6fragmfalsefragment m flag
ip6nexthdr17next header
TCP
NameExampleComment
tcp_offset34TCP header offset from start of packet
tcpsourceport80source port
tcpdestinationport26955destination port
tcpseqno1971494866sequence number
tcpackno3138709947acknowledgement number
tcpflags000010000flag bits
tcpoffset5offset
tcpwindow512window
tcpopts1-1-8options
tcpoptbytes12option bytes
tcppayloadbytes161payload bytes
TCP Info

See Network performance monitoring

NameExampleComment
tcpdirectionreceivedsampled packet direction, sent or received
tcpmsssnd1448cached effective mss, not including SACKS
tcpmssrcv1336max. recv. segment size
tcpunacked10packets which are "in flight"
tcplost0lost packets
tcpretrans0retransmitted packets
tcppmtu1500last pmtu seen by socket
tcpcwndsnd22sending congestion window
tcpreordering3reordering
tcprtt668smoothed RTT (microseconds)
tcprttvar153RTT variance (microseconds)
tcprttsdev14.387RTT standard deviation
tcprttserr0.012RTT standard error
tcprttmin439minimum RTT (microseconds)
tcprttwait143tcprtt - tcprttmin
UDP
NameExampleComment
udp_offset34UDP header offset from start of packet
udpsourceport
udpdestinationport
udpbytes
DHCP
NameExampleComment
dhcp_offsetDHCP header offset from start of packet
dhcpopcode
dhcphtype
dhcphlen
dhcphops
dhcpxid
dhcpsecs
dhcpflags
dhcpciaddr
dhcpyiaddr
dhcpsiaddr
dhcpgiaddr
dhcpchaddr
dhcpsname
QUIC
NameExampleComment
quic_offset42QUIC header offset from start of packet
quicheaderformlongshort/long header
quictype0-rttpacket type
quicversion1version
quicspinbittruespin bit
ESP
NameExampleComment
esp_offset34ESP header offset from start of packet
espspi3395130038
espseqno7633681
GRE
NameExampleComment
gre_offset34GRE header offset from start of packet
greflowid
grevsid
greversion0
greprotocoltype2048
VxLAN
NameExampleComment
vxlan_offset42VxLAN header offset from start of packet
vxlanvni5000virtual network identifier
VxLAN GPE
NameExampleComment
vxlangpe_offsetVxLAN GPE header offset from start of packet
vxlangpeversion
vxlangpeflags
vxlangpevnivirtual network identifier
vxlangpenextprotocolnext protocol
NSH
NameExampleComment
nsh_offsetNSH header offset from start of packet
nshversion
nshflags
nshmdtype
nshspi
nshsi
nshnextprotocolnext protocol
GPRS Tunneling Protocol (GTP)
NameExampleComment
gtp_offset42GTP header offset from start of packet
gtpversion1
gtpmsgtype255
gtpmsglength1400
gtptied1634tunnel endpoint identifier
Geneve
NameExampleComment
geneve_offsetGeneve header offset from start of packet
geneveprotocoltype
genevevnivirtual network identifier
DNS

See DNS amplification attacks for an example

NameExampleComment
dns_offset42DNS header offset from start of packet
dnsqrfalserequest=false, response=true
dnsopcode0op code
dnsaafalseauthoritative answer
dnstcfalsetruncated
dnsrdfalserecursion desired
dnsratruerecursion available
dnsz0reserved
dnsrcode0response code
dnsqdcount1number of entries in question
dnsancount0number of entries in answer
dnsnscount0number of entries in name server section
dnsarcount0number of entries in resources section
dnsqnameyahoo.com.domain name in query
dnsqtype15query type code
dnsid9409query ID
dnsqtypenameMX(15)query type name
dnsqclass1query class
requests1number of requests
SNMP
NameExampleComment
snmp_offset42SNMP header offset from start of packet
snmpversion2cversion
snmpcommunitypubliccommunity
snmppduget_reqmessage type
snmpbytes446size of SNMP payload
requests1number of requests
NTP
NameExampleComment
ntp_offset46NTP header offset from start of packet
ntpversion2version
ntpmode7
ntpctlresponse
ntpctloperation
ntpctloffset
ntpctlbytes
ntppvtresponse
ntppvtimpl
ntppvtreq
ntppvtbytes
ntpstratum
ntpreferenceid
requests1number of requests
RTP
NameExampleComment
rtp_offsetRTP header offset from start of packet
rtppayloadG.722
rtpssrc8f02
RTCP
NameExampleComment
rtcp_offset42RTCP header offset from start of packet
rtcplostfraction0.2
rtcpjitter3
Chargen
NameExampleComment
chargen_offset42Chargen header offset from start of packet
chargen!"#$%&'()*+,-./01234567payload
requests1number of requests
SSDP
NameExampleComment
ssdp_offset42SSDP header offset from start of packet
ssdpstartlineNOTIFY
requests1number of requests
HTTP
NameExampleComment
http_offset54HTTP header offset from start of packet
httpmethodGETmethod
httpurl/index.htmlURI as it came from client
httpstatus200status code
requests1number of requests
SIP
NameExampleComment
sip_offset42SIP header offset from start of packet
sipmethodINVITE
siptarget100@10.0.0.1
requests1number of requests
Application Sockets

See sFlow Host Structures

NameExampleComment
protocol
serveraddress
serveraddress6
serverport
clientaddress
clientaddress6
clientport
proxyprotocol
proxyserveraddress
proxyserveraddress6
proxyserverport
proxyclientaddress
proxyclientaddress6
proxyclientport
HTTP Operation

See sFlow Blog for articles on HTTP sFlow

NameExampleComment
httpmethodGETmethod
httpprotocol1.1protocol version
httphostsflow-rt.comHost value from request header
httpuseragentMozilla/5.0User-Agent value from request header
httpxff10.0.0.1X-Forwarded-For value from request header
httpauthuseradminRFC 1413 identify of user
httpmimetypeapplication/jsonMime-Type of response
httpurl/index.htmlURI exactly as it came from client
httprefererhttps://sflow-rt.com/index.phpReferer value from request header
httpstatus200status code
bytes4222request + response bytes
reqbytes1210request bytes
respbytes3012response bytes
duration23110duration of operation (in microseconds)
requests1number of requests
Memcache Operation

See sFlow Blog for articles on Memcache sFlow

NameExampleComment
memcachecommandGET
memcacheprotocol
memcachestatusOK
memcachenumkeys
bytes
duration
requests1number of requests
Generic Application Operation

See Scripting languages for article describing how to send generic application transactions

NameExampleComment
appname
appoperation
appattributes
appstatus2
appstatusdescrTIMEOUT
duration
bytesrequest + response bytes
reqbytesrequest bytes
respbytesresponse bytes
requests1number of requests