DOCSIS troubleshooting: pinging and sniffing

A customer DOCSIS network had some strange issues with modem and MTA performance, and here are some simple scripts that were used for troubleshooting.

A monitoring session was set up on a distribution switch, and all CMTS traffic was sent to eth1 on the sniffing host.

The file addrlist contains all monitored IP addresses (CM and MTA), one per line.

runsniffer.sh, the sniffer launch script, creates hourly captures for all the monitored addresses:

tcpdump -i eth1 -s0 -G 3600 \
 -w /srv/capture/cmts8-%F--%H-%M-%S.pcap \
`perl -e 'open(IN, "addrlist"); print join(" or ",  map {chomp; "ip host " . $_} ())'`

runfping.sh runs fping every 5 seconds, and fping sends only one ICMP Echo packet and reports all the IP addresses which responded longer than 500ms:

while true; do
  DATE=`date`
  RESULT=`fping -f addrlist -t500 -r0 -u`

  if test -n "${RESULT}"; then
    /bin/echo -e \[$DATE\]   $RESULT >>fpinglog
  fi
  sleep 5
done

As soon as there’s some content in the log, checkandemail.sh sends it in email. It does it so every hour:

while true; do
  if test -s fpinglog; then
     cat fpinglog | mail -s "Modem test log" \
        xx@domain.com yy@domain.com zz@domain.com
  fi
  sleep 3600
done

The scripts are launched with nohup, so that they can run for as long as needed:

nohup sh runsniffer.sh &
nohup sh runfping.sh &
nohup sh checkandemail.sh &
Advertisements

, , , ,

  1. Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: