Network Error Detection & Analysis

Kubeshark is useful for detecting various network-related errors, such as:

In addition to offering details about detected errors, Kubeshark allows you to record raw traffic for in-depth analysis using tools such as Wireshark

Learn more in the Traffic Recorder section.

TCP (Connection) Errors

Connection errors, identified as part of the TCP protocol, include:

ErrorDescription
SynSentThe client attempted to establish a connection, but the connection was refused. SYN sent, but ACK not received.
CloseWaitOne side closed the connection with FIN, but the final ACK confirmation is pending.
LastAckOne side received a FIN from the peer, sent an ACK, sent a FIN, and is waiting for the final ACK from the peer.
ResetA RST packet is seen, indicating that one side will neither accept nor send more data.

TCP Errors

Timeout Errors

Kubeshark’s L4 stream capture times out after 10 seconds. If an L4 stream capture doesn’t complete within this timeframe, the stream is marked stale and dropped. THe timeout time is configurable and can be set using this configuration value:

tap:
    tcpStreamChannelTimeoutMs: 10000 # in milliseconds

or use: --set tap.tcpStreamChannelTimeoutMs=10000

Packet Loss

Packet loss can cause Kubeshark L4 streams to timeout and parser errors. Additionally, lost packets might result in no symptoms and no signals.

Potential causes of packet loss:

CauseDescription
Insufficient CPU resources for KubesharkPacket loss can occur when Kubeshark is unable to read packets quickly enough, often due to a lack of CPU resources.
AF-PACKET vs PF-RINGAF-PACKET is prone to packet loss, whereas PF-RING is less likely to cause it.

ICMP

Kubeshark intercepts, dissects, and presents ICMP messages. ICMP is an L4 protocol used for error reporting and network diagnostics. For instance, if a router cannot forward a packet because the destination is unreachable, an ICMP message is sent back to the sender indicating the problem.

Partial list of errors reported by ICMP messages:

Destination Unreachable: Sent when a packet cannot be delivered to its destination for various reasons, with subcodes including:

  • Network Unreachable
  • Host Unreachable
  • Protocol Unreachable
  • Port Unreachable
  • Fragmentation Needed and DF (Don’t Fragment) set
  • Source Route Failed
  • Destination Network Unknown
  • Destination Host Unknown
  • Source Host Isolated
  • Network Administratively Prohibited
  • Host Administratively Prohibited
  • Network Unreachable for Type of Service
  • Host Unreachable for Type of Service

Time Exceeded: Indicates that the Time to Live (TTL) field of the packet has reached zero, necessitating discarding the packet. This is commonly used in network route tracing.

Source Quench: A deprecated message requesting the sender to decrease the message sending rate due to router or host congestion.

Redirect Message: Sent by routers to indicate a more efficient packet routing path.

Parameter Problem: Indicates an error in the IP packet header fields, preventing processing.

Router Advertisement and Router Solicitation: Used for routing information discovery and announcement.

ICMP reported errors

Dissection Errors

Dissection errors are reported by Kubeshark protocol parsers:

ErrorDescription
unexpected EOFTCP connection closed unexpectedly.
Parser errorKubeshark’s application layer protocol parser reports an invalid payload according to the protocol definition.

L4 Stream Capture Timeout

Kubeshark will drop entire Layer 4 (L4) streams if not all packets are captured before the process times out.

The default timeout is set to 10 seconds. This can be modified by setting the tap.tcpStreamChannelTimeoutMs parameter.

Half Connections

Half connections represent incomplete transactions where either a request or a response is missing.

Half Connection Example

Kubeshark aims to identify the reasons for these incomplete transactions.

Error Filter

Use Kubeshark Filter Language (KFL) to selectively display or filter out various errors. For example:

!error      # Filters out any half-connections
!icmp       # Filters out any ICMP entries
!tcp        # Filters out any TCP level errors

Tip: Error Dashboard

Enter the following KFL in the KFL box: response.status > 400 or error or icmp or tcp, apply, copy and enter to favorite. The link should look like this:

http://<kubeshark-domain>:<port>/?q=response.status%20%3E%20400%20or%20error%20or%20icmp%20or%20tcp

It will look even better on a 60” screen.