Quickie: Extracting HTTP URLs With tshark
After I posted diary entry "Quickie: tshark & Malware Analysis", someone asked me how to extract HTTP URLs from capture files with tshark.
Use option -r to read a capture file, and options -T fields and -e http.request.full_uri to let tshark print the full URL of HTTP requests. Problem is that tshark will also output an empty line for each packet. I filter these out with findstr or grep:
Please post a comment if you know how you can avoid these empty lines with a tshark option.
It's also possible to print the full protocol packet tree with packet details, and search this for URLs with my re-search.py tool. The difference here, is that you will find all kinds op URLs, not only for HTTP requests.
For example, many of the URLs seen in this screenshot, are found inside certificates.
Didier Stevens
Senior handler
Microsoft MVP
blog.DidierStevens.com DidierStevensLabs.com
Comments
Anonymous
Dec 3rd 2022
9 months ago
Anonymous
Dec 3rd 2022
9 months ago
<a hreaf="https://technolytical.com/">the social network</a> is described as follows because they respect your privacy and keep your data secure. The social networks are not interested in collecting data about you. They don't care about what you're doing, or what you like. They don't want to know who you talk to, or where you go.
<a hreaf="https://technolytical.com/">the social network</a> is not interested in collecting data about you. They don't care about what you're doing, or what you like. They don't want to know who you talk to, or where you go. The social networks only collect the minimum amount of information required for the service that they provide. Your personal information is kept private, and is never shared with other companies without your permission
Anonymous
Dec 26th 2022
9 months ago
Anonymous
Dec 26th 2022
9 months ago
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> nearest public toilet to me</a>
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> public bathroom near me</a>
Anonymous
Dec 26th 2022
9 months ago
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> nearest public toilet to me</a>
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> public bathroom near me</a>
Anonymous
Dec 26th 2022
9 months ago
Anonymous
Dec 26th 2022
9 months ago
https://defineprogramming.com/
Dec 26th 2022
9 months ago
distribute malware. Even if the URL listed on the ad shows a legitimate website, subsequent ad traffic can easily lead to a fake page. Different types of malware are distributed in this manner. I've seen IcedID (Bokbot), Gozi/ISFB, and various information stealers distributed through fake software websites that were provided through Google ad traffic. I submitted malicious files from this example to VirusTotal and found a low rate of detection, with some files not showing as malware at all. Additionally, domains associated with this infection frequently change. That might make it hard to detect.
https://clickercounter.org/
https://defineprogramming.com/
Dec 26th 2022
9 months ago
rthrth
Jan 2nd 2023
8 months ago