Parsing tcpdump/pcap files in bash, without Wireshark.

Sometimes I work in an environment in which software access is restricted.  Things like Wireshark are DEFINITELY not allowed.

However, I'm still expected to troubleshoot network connectivity issues.  Luckily, tcpdump exists in the EPEL Repo, but leaves much to be desired when trying to read the PCAP files.  I decided to read up on the PCAP spec, and create my own parser in bash.  Crazy? Maybe.  But, it does work.

Another nice capability here is that using something like vim, you can easily tweak this script to dump application data for further debugging.  For example, switch your HTTPd to run in Non-SSL, run a capture, and dump the application data.  Of course, you may have to work out the correct byte offset for your packets. As this is a bare minimal implementation of a pcap parser and does not nearly support the full scope of capability provided by libpcap.

The real issue here is that I have not taken the time to parse some flags which consist of single bits.  There is also much work to be done in regard to writing the conditionals for various Ethernet, IP Routing and TCP frames.  But this is a good baseline to start.

To use this script just run 'tcpdump -w file.pcap -X' to run a capture.  Once done, pass the file as an argument to the pcap-analyzer bash script, and you will be able to read your frames on the CLI!  This should work out of the box with Centos/RHEL 6 Minimal, with the addition of the tcpdump RPM.

Here is a screenshot.

As you can see frame 2 starts to break because of the parsing.  Some flags actually mean, the header is going to be 28 bytes instead of 26.  As a result of the byte variations, the byte iterator ends up on some obscure part of the stream I don't handle, and things get start to break.  Git forks are welcome if you like this little utility and are interested in making it more robust.

https://github.com/charlescva/common/blob/master/pcap-analyzer.sh

One final important characteristic about Ethernet wire data is that it is ALWAYS Big Endian.  This is confusing because most CPUs are little endian.  To work around this issue, I use both 'hexdump' and 'od' because one enforces big endian while the other is CPU dependent.

Other resources Used:
http://www.tcpdump.org/manpages/pcap-savefile.5.html
http://www.tcpdump.org/linktypes.html
https://en.wikipedia.org/wiki/Ethernet_frame#Ethernet_II
https://en.wikipedia.org/wiki/IPv4#Packet_structure
https://en.wikipedia.org/wiki/Transmission_Control_Protocol#TCP_segment_structure

QuickTip: Tree Alias

Sometimes directory structures are a bit of a nest.  You often want to see the full structure, but may not have access to packages like 'tree'.  I found a great expression and decided to wrap it into an aliased command in ~/.bash_profile

alias tree="ls -R | grep \":$\" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/   /' -e 's/-/|/'"

Now, when you want to see a complex directory structure in the shell, just change to the working directory, and type `tree`

Custom `tree` command

Just Install Docker on Ubuntu 14.04 64-bit...

It has been a while since I have posted, so I'm making this one short.  Ever want a clean and concise script written in Bash that will install the latest version of Docker (And stay up to date)?  Well, here. Note that Docker wants you to pick the DNS.  So I've used 8.8.8.8 (Google-DNS) for this example.

Note: This will also install the kernal extras to enable AUFS support.

#!/bin/bash

#Update our local package index

sudo apt-get update

#Make sure Apt supports HTTPS

[ -e /usr/lib/apt/methods/https ] || {

  apt-get install -y apt-transport-https

}

# Get server key for repo

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9

# Update Repo Sources...

[ -e /etc/apt/sources.list.d/docker.list ] || {

  rm /etc/apt/sources.list.d/docker.list

}

sudo echo "deb https://get.docker.io/ubuntu docker main" > /etc/apt/sources.list.d/docker.list

sudo apt-get update

# ensure  Linux kernel extra modules are installed which support cpu cgroups, etc...

sudo apt-get install -y linux-image-extra-$(uname -r)

# install docker

sudo apt-get install -y lxc-docker

# update docker DNS and listen on localhost tcp

sudo echo 'DOCKER_OPTS="-dns 8.8.8.8 -H unix:///var/run/docker.sock -H tcp://127.0.0.1:5555"' > /etc/default/docker

Elaborate Linux Script for determining IP

Sometimes the ordering of your Ethernet adapters are not the same on a specific machine vs the others in your cluster.  So simply grepping the IP based on adapter name can be a little tricky.  So I just made this script to do it for you.

#! /usr/bin/env bash
for network in $(cat /proc/net/dev | grep ':' | cut -d: -f1 | awk '{ print $1 }'); do
if [ "$network" != "lo" ]
then
echo $(/sbin/ifconfig $network | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}')
fi
done

Clearly you could modify the if condition to fit your needs.