My next class:
Network Monitoring and Threat Detection In-DepthSingaporeNov 18th - Nov 23rd 2024

Updated PFSense Client

Published: 2016-04-13. Last Updated: 2016-04-13 22:04:20 UTC
by Johannes Ullrich (Version: 1)
10 comment(s)

Earlier this week, PFSense 2.3 was released. The new release changed the name of a function I use in our pfsense log submission client, and the client will fail to parse the logs. I just released a new version of the script, that you can download here:

https://isc.sans.edu/clients/dshieldpfsense.txt (GPG Signature: https://isc.sans.edu/clients/dshieldpfsense.txt.asc ).

If you rather just apply the change to your existing file, find the line (should be line 65):

$flent = parse_filter_line(trim($line));

and replace "filter" with "firewall_log":

$flent = parse_firewall_log_line(trim($line));

This should fix the issue. The new client checks what version you are running, so it will work with 2.2 and 2.3. (but only tested with 2.3 right now).

Please let me know if you have any problems! And thanks to those who reported the issue.

 

---
Johannes B. Ullrich, Ph.D.
STI|Twitter|LinkedIn

Keywords:
10 comment(s)
My next class:
Network Monitoring and Threat Detection In-DepthSingaporeNov 18th - Nov 23rd 2024

Comments

Just wondering what the consensus is regarding pfSense. Currently we're using CP 12000 UTM & we're not too happy with it, especially in regards to VPN. Does pfSense provide any web content filtering or DLP functionalities?

Anonymous

Apr 14th 2016
8 years ago

The first time I ran it, it showed me in the console that is sent the log file information. Since then, it does not show anything being sent AND I only on occasion get an email. Does it only send information at a certain threshold? I am also seeing nothing in "My Reports" since enabling this and yes, my uid and authkey are there. I also have email notification set up and they are working.

Anonymous

Apr 24th 2016
8 years ago

I got this working with edit, when copy pasted it was putting a return in how it parsing the logs so they were not just 1 line with <tab> for dshield to read. I was seeing that logs went up but nothing was being read.. Not sure why that happened, but now it seems to be sending logs correctly, and they are reported by dshield.

But something is not right with time and reading the logs?

Cron is running when it should, putting entries in the log that no new lines since last run. But clearly if you look in the logs there was new blocks that are just S to the wan interface that are there. So why did it not send them?

Running every 30 minutes, it clearly should send a few entries every 30 minutes.. But that does not seem to be the case..

Anonymous

Sep 5th 2016
8 years ago

Newer versions of PFSense are using the circular log buffer file format - which doesn't play too nicely with the script as it stands.

Only one line to change:


diff dshieldpfsense.php-orig dshieldpfsense-new.php

61c61
< $log=fopen("/var/log/filter.log","r");
---
> $log=popen("/usr/local/sbin/clog /var/log/filter.log","r");


This opens a child process, "clog" which turns the circular log into a regular linear log, output to stdout. $log is still a regular file descriptor, so the rest of the script continues unchanged.

Works well for me - now I'm seeing regular submissions every half hour as expected. YMMV of course !

This approach is still not perfect as the clog format can result in an incomplete first line. I did have this wrapped in a shell script including "tail -n +2" to drop the first line - but Johannes' original script is robust enough to handle the junk first line (if there is one).

Keep up the great work ISC team !

-- Update --

If you are interested why... the infrequent submission is because the last line in the circular log file is older than the last submission date that the dshield sensor records - until the whole clog file is overwritten.

This then causes the following test to fail until the whole file has wrapped around again. (The test assumes the last entry in the file is the latest log entry, which is not true for a circular log file)

# dealing with errors
if ( $lasttime>=$time ) {
log_error("no new lines added to log since last run OK");
exit();
}

Negative impact: Events that occur in the time between the circular log wrapping around and the time of the cron job triggering overwrite entries that have not been submitted yet.

Neutral impact: Submissions only occur when the log file has been completely overwritten, so log submissions are less frequent unless you have a noisy firewall !

Anonymous

Dec 8th 2016
7 years ago

Hi Johannes,
I just installed a pfSense version 2.3.3-RELEASE-p1 (amd64) firewall at my house. I modified and uploaded the dshieldpfsense.php script to it and tried to run it. It is located at: /root/bin is owned by root:wheel, and has these permissions: -rwxr-xr-x. I keep getting this error when I try to run it: could not send 2602 lines to DShield server does not require authentication, it probably requires starting TLS

Any ideas what I did wrong and what I should do to fix them?

Thanks!
Jon

Anonymous

Mar 17th 2017
7 years ago

Working as intended. 2.3.3-RELEASE-p1. Thanks.

Anonymous

Apr 25th 2017
7 years ago

If anyone else is having this issue, the solution is to change your pfSense notifications from StartTLS to SMTP over SSL / TLS. Check for ports with your specific email provider, but you'll probably want port 465.

Anonymous

Jun 26th 2017
7 years ago

I was able to resolve startTls issues (using Office365) with the following:

$smtp->ssl = (isset($config['notifications']['smtp']['ssl'])) ? 1 : 0; # existing line

#If your email provider only allows startTls connections, turn on start_tls
$smtp->start_tls = 1;

$smtp->tls = (isset($config['notifications']['smtp']['tls'])) ? 1 : 0; # existing line

Anonymous

Aug 19th 2017
7 years ago

HI,

pfSense 2.4 was released. Unfortunately the script silently stops working on 2.4 . Anyone else seeing this?

René

Anonymous

Oct 16th 2017
7 years ago

The script updated in a different repo and wasn't updated at the prior location. The new location is below.

https://github.com/jullrich/dshieldpfsense

If you are interested, I also updated the how to I did some time ago on configuring it.

https://www.linuxincluded.com/sending-pfsense-logs-to-the-dshield-project/

Anonymous

Nov 30th 2017
6 years ago

Login here to join the discussion.


Top of page
Diary Archives