A few months ago during the first stage of our 1-to-1 iPad deployment, I ran into an issue where I could connect to our wireless network just fine and send and receive traffic to my heart’s content, but sometimes when I strayed to another location within the school I would lose connectivity for several minutes. I checked out the configuration on the Wireless Access Points and it looked fine–the SSIDs were all set up appropriately, the radios were beaconing and everything looked great from the management interface. Why couldn’t I roam? We had this same style of configuration at the previous 7 schools and this issue was not present. To make things a little more confusing, the issue wasn’t 100% reproducible. Sometimes it happened, sometimes it didn’t.
Almost luckily, we hadn’t finished properly documenting and naming the access points at this facility, and before attempting to begin the troubleshooting of what I thought was a wireless issue, I was physically locating and documenting each AP via their MAC addresses by bouncing the switch interfaces they were connected to (a crude method for sure, but these APs lacked any location features, and it was after-hours). I noticed that a few of the access points had “Static” as their learning method in the MAC address table, where I expected to see “Dynamic”. I checked the configuration on the switches to make sure no static entries were actually entered, then by chance ran across some port-security settings in some of the interface configurations. After a bit of reading, I remembered that port-security settings can result in a static entry in the MAC address table–it doesn’t have to result from a manual entry typed by hand. Aha!
Unfortunately I didn’t grab the configuration before making changes to it, and this was months ago, but if memory serves here’s what the port-security configurations looked like, with my comments following the double slashes:
switchport mode access //configure the port as an access port, meaning only allow the port to be a member of one VLAN–required for port-security settings
switchport port-security //enable port security on this interface
switchport port-security maximum 50 //allow a maximum of 50 MAC addresses to be learned on this interface
switchport port-security violation protect //if more than the maximum specified number of MAC addresses are seen on this port, don’t allow them to be added to the MAC address table but leave the interface up rather than shutting it down
switchport port-security aging static //allow the MAC addresses to age, otherwise the first 50 will stay in the MAC address table forever and no new addresses will be learned
switchport port-security aging time 5 //remove the MAC addresses from the MAC address table after 5 minutes when…
switchport port-security aging type inactivity //…no frames from that MAC address are seen on the port during that time
After conversing with the Director of Technology, it seems that these settings were an attempt to limit the number of clients associating with each access point, as they were rated from the vendor as capable of supporting up to about 50 devices at a time. Unfortunately, this implementation was a bit ill-fated. Any time a client would connect to an AP that was connected to a port with the settings above, that client would have problems when attempting to roam. The association with the next access point would go off without a hitch, but the switch itself wouldn’t allow the client’s MAC address to be learned on that new port for 5 minutes. Some ports had the configuration and some didn’t, which led to the inconsistencies.
So as it turns out, the problem that I originally thought was a wireless issue had nothing to do with wireless. It also turns out that our brand of access points (AeroHive) allows you to configure a maximum number of connections in the access point configuration itself, which I think is a much better way to go. I think this scenario presents solid evidence of the need for troubleshooting skills. I had nothing to do with entering those commands, and had no idea they were there, but nevertheless I found myself troubleshooting the issue. This also prompted a new phrase of which I need to constantly remind myself: “The wireless network is connected to the wired network!”