The web debugging proxy Fiddler is a terrific tool.
On top of being über useful in capturing all of the web traffic coming in and out of your computer, it can also be used to analyze, manipulate, and redirect that traffic.
For web developers, the type of functionality that Fiddler offers is invaluable, but it can be just as useful for the amateur tinkerer. With only a little knowledge, it’s possible to reverse engineer web APIs, sites, and online services. You can easily see malware launching communications from your device, be it your desktop PC, laptop, or mobile, and you can fiddle with these communications (hence the name) down to very fine details.
I have to give a shout-out to my favourite software in the same category, Charles, but Fiddler’s price tag (free) makes it a good, albeit not quite as powerful alternative.
It’s good enough, in fact, to even capture and analyze HTTPS – an encrypted layer of communication that sits on top of standard web traffic that’s used from anything from accessing your GMail account to your online bank account. In fact, Fiddler’s ability to capture and analyze such traffic is at once very useful and simultaneously disturbing and somewhat scary. After all, if Fiddler can do it so easily, why not some other piece of software that’s not quite as visible … perhaps not even installed by you?
I’ll have to save that discussion for some other time, though, because in this post I wanted to cover what happens when Fiddler stops being able to deal with HTTPS, or encrypted web traffic. In other words, what if it stops being able to do what you set it up to legitimately do?
For example, I have Fiddler running pretty much all the time as part of my day job, and most of the time I don’t have to think twice — it negotiates standard HTTP and encrypted HTTPS without any issues and, in the web browser, everything loads fine.
Then this happens:
Here was have Google’s Chrome complaining about a security certificate issued, apparently, from Google itself!
Other times the site would simply fail to load altogether, though if I replaced “https://” with “http://” in the address bar, everything would work fine. In other words, secure browser connections, or HTTPS, were failing — but only when Fiddler was attempting to capture them. Turning off Fiddler fixed the problem right away.
At this point I’m still not certain why exactly this happened. All I know for certain is that at some point, the security certificates stored on my computer were no longer recognized and the browser wouldn’t allow any HTTPS connections with Fiddler. And since Fiddler intercepts traffic at the operating system level, and not the browser level, any HTTPS connections from any browser would fail.
Oh, and my current project required me to analyze HTTPS traffic. Of course.
I searched a variety of knowledge bases and found information that was both woefully out of date and that didn’t work anyways. I’m using Fiddler 2 (v22.214.171.124 beta), which is not terribly new but seems quite exotic in terms of the internet’s help sites, so I was left to my own devices.
Luckily the solution was quite simple. This can be used the first time you’re setting up Fiddler for use with HTTPS, or if it stops working for whatever reason.
- Open up Fiddler’s HTTPS options panel by selecting (from the main menu):
Tools -> Fiddler Options… -> HTTPS (tab)
- Clear any installed interception certificates. Ensure that “Capture HTTPS CONNECTs” is selected (checked), but other options such as “Decrypt HTTPS traffic” are unselected (unchecked). This will enable the “Remove Interception Certificates” button — click it.
- Accept any pop-up dialogs that are displayed until the certificates are removed.
- Fully close and restart Fiddler (you’ll probably be prompted to do so anyway).
- Repeat step 1 to open the HTTPS tab.
- This time ensure that you select ”Capture HTTPS CONNECTs” as well as ”Decrypt HTTPS traffic“. As soon as you select this second option you will receive a dialog box that looks something like this:
Click on “Yes” (including on any subsequent dialogs), to install Fiddler’s certificate in the operating system. This will allow Fiddler to sit in the middle of encrypted web traffic, capture it, and analyze it.
- Click on “OK” in the HTTPS tab you’ve now returned to.
- You should now be able to seamlessly connect to HTTPS and see the traffic in Fiddler. You may also have to accept an untrusted certificate in your browser next time you try to connect, but since you know that you’re legitimately only going to be viewing your own traffic, you can accept it. Here’s an example from Firefox (here you would click “Add Exception…” to force the browser to accept this connection:
Now you should be able to see secure web connections right along side standard HTTP traffic. The only real differences are the initial HTTPS “tunnel” that’s established as s first step in a secure connection, and the fact that the “Protocol” column for subsequent communications with that address will show “HTTPS“:
Doing in-depth analysis of this information is a topic in itself so that’d best saved for a future post, but if you’ve spent enough time online and recognize web addresses and URLs, this should be more than enough to get you started, and it should be sufficient to allow you to flag suspicious activity.