Inconsistent Timestamps Over Long Wireshark Captures
I’ve recently been running Wireshark around the clock on a couple of servers at work to troubleshoot a problem that pops up whenever it feels inclined. One late night while perusing trace files with sleepy eyes, I happened to realize that packets were arriving at server B two minutes and forty seconds after they were sent from server A, according to the timestamps on the packets. I thought that was very odd, of course. It seemed rather impossible that packets were taking almost three minutes to reach their destination on our high capacity network–and where were the errors? I checked the system time on both servers, and they were both correct and within a second or two of each other. I restarted both captures and double checked the system times and the timestamps on the packet and they were correct at that time. A few days later, I see the same issue arise. Then I notice that the current packets that are coming in are stamped a few minutes behind the system time. What the heck?
Well it turns out that Wireshark’s reliance on the WinPcap library in Windows causes this issue. When a capture is started, WinPcap checks the system time for a reference point and then tracks the time on its own by referencing the KeQueryPerformanceCounter routine (that’s processor mumbo-jumbo to me, maybe you are more well-versed). It looks like there are two options to handling this problem, and number one is to deal with the issue with an understanding that the time is going to lag more and more behind the system time the longer the capture runs. You might be able to alleviate the problem by restarting the capture session frequently, although you will probably lose packets during that process. You can use Wireshark’s timestamp offsetting feature to help get things on the same track if you are referencing two trace files taken at the same time, but there might still be some discrepancies.
Option two is to change a registry key setting in Windows which will force WinPcap to use the system time for its packet timestamps. The drawback with this option is that it is less precise compared to the KeQueryPerformanceCounter value. Your packets will be stamped with a time in 10-15 millisecond increments, rather than microseconds. That means you are likely to have packets with the same timestamps pretty often, like in the picture below.
According to a Wireshark mailing list exchange, the registry value to change is:
HKLM\System\CurrentControlSet\Services\NPF\TimestampMode
And the possible values are:
“0 (default) -> Timestamps generated through KeQueryPerformanceCounter, less reliable on SMP/HyperThreading machines, precision = some microseconds
2 -> Timestamps generated through KeQuerySystemTime, more reliable on SMP/HyperThreading machines, precision = scheduling quantum (10/15 ms)
3 -> Timestamps generated through the i386 instruction RDTSC, less reliable on SMP/HyperThreading/SpeedStep machines, precision = some microseconds”
And you’ll need to restart the NetGroup Packet Filter Driver service with these commands before kicking off your capture:
net stop npf net start npf
I hope this is useful information for you! Good luck!