IT Help Needed

Hi,

I work as an IT for a local private school. As of right now I can't determine why the clients on our network get slow internet as well as file transferring speed.

The outline of the infrastructure is Modem -> Firewall -> Branching Fiber Optic Media Converters -> Switch -> Work Stations.

I've spoken with the ISP and no problems on their end. Seems to be enough bandwidth and nothing to cause the behavior. I've tested the line by connecting to the firewall and that also seems to be fine. When I get to the clients it is extremely slow, about 1/3 the speed at the Firewall level. Firewall is set to 100 and Full Duplex, it also handles DHCP fine.

Our DNS server is however at the same level as the workstations except for it is on a specific branch. This means all workstations must traverse up the their branch, then back down the branch leading tothe DNS server. The DNS server as well another machine next to it connect to the same switch gets a fine connection... DNS server has 2 ISP provided IP's as the first forwards and then the two public forwards provided by google.

This tends to make me think that its traversing up the branches and then down to the DNS server which is causing the latency. I believe this would also effect file transfer speeds as well? But I'm wondering can this really cause a 1/3 loss in bandwidth? The distance of the branches are no more than 50 feet or so. I'm wondering if maybe the converters are causing this or maybe the fire wall is a bottle neck. Or maybe fiber optic converters in general slow down a connection? Again all client, server, and firewall are running at 100 Mbs and full duplex on machines, I manually made sure. I'll list the following types of hardware we use.

Firewall - Cisco ASA 5505
Switches - Linksys EZXS16W
Media Converters - Intellinet 515337
Clients - Windows XP with Broadcom NIC's
Server - Windows 2003 Server, 4gb RAM, 2 2.33ghz CPU's with 2 Cores Each.

Hardware might be a little off, doing this from memory but I'll get the exact specifics tomorrow if someone thinks they got something. I've spent days googling it and I can't figure it out...

 

i had a problem not to long ago with my internet where i wasnt getting my speed and then found out it was the modem it self its could not handle all the traffic it was getting so that could be your problem.

 

Which is why I checked with the ISP that we weren't maxing out our bandwidth...

 

This might seem like a no-brainer, but did you try bypassing the firewall?

 

I tried the connection right after the firewall and its fine. If the connection after the firewall is good then it's not the culprit right?

 

Was going to suggest the same thing. Start pealing back layers until the problem goes away.

 

mhmm, other than the firewall, you might try using pathping from the effected computes, that might show you where a long delay is occuring and/or where your loosing packets. its a slightly fancier version of traceroute, it runs the regular traceroute, then 250 secs later it gives detailed information. naturally, low level devices like switches dont appear on these... but using two or three computers from each branch should allow you to ascertain where the problem is by process of elimination, failing that, if you hook up a laptop at places where you think the issue might be, you can pathping from there directly, might have to wait untill after hours to start pulling cables though.

otherwise... id suggest looking up the hardware... some products have reputations for passing much slower than they are rated for... and a network is only as fast as its slowest link.

lastly, and this is reaching abit, but you might consider testing your cables for faults, i know fiber can be really finicky about how its bent, and even ethernet cables are sometimes just bad. but since the problem is widespread, looking at your trunks couldnt hurt.

 

Thanks Aral. Sounds like I'll try pathping. None of those switches will show up... maybe the media converters will?

The hardware is sorta old, and the guy who installed it had no idea what he was doing... problem is I wouldn't know what to look for.

The cable fault seems like its reaching a bit. Odds of it being cables that go to the client PC's... Would have to be 2 cables where the path on the branches narrow...

In all honesty I believe it to either be the firewall (it's sorta old) or the media converters... I'll check what I can today and post it. The problem is I can't exactly ask the school to spend money unless I know it's going to fix our problem... Doesn't have to be 100% but if it were strong he would trust me and let me purchase...

 

Ok so I just ran pathping from a client PC.

From the client to the firewall there was only 1 direct hop. The RTT was 0 ms, lost 1/100 packets.

From the client to the DNS server was 1 direct hp, rtt = 0 and lost 0/100 packets.

From the client to google.com didn't make it. rtt = --- and lost 100/100 packets...

From the DNS server to firewall was 0 ms, 0/100 packets lost.

DNS Server to google did not go through with same stats as the client.

DNS Server to Client at 0 ms and 0/100 packets lost.

So my guess is the switches and media converters don't count and that the firewall is blocking my pathping... That was all I could do from home, guess I'm going to get a laptop and try and plugin at strands along the branches see if that brings up anything...

 

If a direct connection to the modem is working fine, then that means your ISP and modem are working well, but anything after that, could be a culprit... that includes your firewall. Then it's just a matter of breaking it down to what, starting with a direct connection to the firewall.

I would be using a personal laptop to test all of this, as any workstation could be compromised.

If you find the firewall to be the culprit, it isn't necessarily a hardware issue. It's always possible that critical ports are being blocked, or any other kind of setting conflict.

 

you could also use traceroute to find the slow down.

Look for
* Latency > 1ms INSIDE your network. Healthy networks stay under 4ms for fairly short runs.

Here is the output from my desktop to our firewall and core router which is on the other side of the campus.

1 0.522 ms 0.487 ms 0.455 ms
2 1.293 ms 1.616 ms 1.946 ms

It sounds like a FW issue, either a port is blocked or there is a rule in place that's causing grief.

Do you have any switches, hubs or routers between clients and the firewall?

 

I have done a direct connection to the firewall with a laptop. That what I meant by "I tried the connection right after the firewall and its fine".

And I find it strange that the DNS server, which is at the same level as the workstations has a good connection but the workstations do not. Which is why I thought it had something to do with the work stations having to go up their branch, hit the firewall, then go down the branch leading to the DNS server, than back up to the firewall, then finally out to the modem... whereas the DNS server just points to itself, up its branch to the firewall, and then out.

 

There you f*ckin go! I get a 4ms run just from a workstation up to the firewall...You the fuckin man bro.

Except the part about Do you have any switches, hubs or routers... I listed that already lol!

F*cking making progress now!

Ok so from Workstation -> Firewall -> DNS server is all over the place, 5ms to 16ms to just a * which I assume means timed out...

From DNS -> Firewall is somewhere around 16ms or sometimes higher.

From DNS -> Firewall -> same Workstation was about the same if not a little more...

WOAH! From DNS -> Client connected by only 1 switch its about 16 to 17 seconds...

WTF... You know my numbers come bck like 16ms >1ms >1ms in that exact order... the 2nd two are always >1ms

Why is the first attempt always way higher than the 2nd and 3rd?

 

Hm... makes me think its the cable going directly from switch to DNS server... But then why would the DNS have a good speed... Maybe cause it traverses that cable only once where as the other machines must do it twice?

 

Yah,

I would start looking at cables and maybe EMI or other things that could be causing issues. We had an issue in one of our class rooms similar to yours, ended up being the cable lost its shielding due to being to close to a heating duct. Replaced the cable and all was good again.

Another thought, do you have anyway on your network gear to see what type of connection (duplex, half-duplex, 10/100/100MB etc) your clients are connecting with? Could be another data point to look at.

I would also invest in a Network Cable Tester, could save you a boat load of time tracking down wires etc.

 

what's EMI?

I manually set the computers, and firewall to Full Duplex and 100MB... Made sure with the ISP that the modem ran the same.

A cable tester would be nice, does the jack just plugin to a slot on the tester or something?

 

As Trice pointed out - as far as EMI goes, I recall the cable installation team I dealt with stating that Cat6 cable has to be so many feet (5?) from any florescent lighting, and run perpendicular (not parallel) to any electrical wiring.

I think you mentioned the guy who did the wiring wasn't an expert - if he wasn't, you might want to check. It would be very "easy" to want to throw a plastic tie onto the electrical wire conduit, and run your cables that way.

HTH

 

EMI = ElectroMagnetic Interference

It's really interesting all the things that can cause interference with each other.

 

I must be stalking your post - just saw you reply;

Testers are really neat - they come in a variety of abilities. Some just test if you have a connection, others will simulate a network running and give you performance results.

The unit I used had two ways of working, either alone or with one of the terminators. It had two network ports on the top. If you were making a patch cable, you just used the two ports at the top. If you had a in-the-wall cable you were testing, you'd attach the terminator at one end, walk to the other end and use the tester.

It would give you whether all the wires were wired correctly (pin 1 to pin 1, etc), if there were any breaks, or any shorts (wires hitting each other, perhaps from melting). It could also tell you how long a cable run is. I believe 300M or 100Feet is the maximum specification for copper cabling in networking.

HTH