Open Source/Cheap: Internet Monitor & Website Blocker

[demonic]

Hey there,

I have a firewall at work and its a proprietary device. Watchguard Firebox, runs Linux.

Does a good job, but when you want to run its own extras, it costs alot and its a per year licence.

Is there anything open source or cheap that I can install on a workstation and it can sniff the packets destined for the firewall and do 2 things.

1) Monitor who is trying to access a website
2) Block the website.

So I can have a report if John at sales is trying to access www.underagegirls.com

Cheers

 

[Zaphod]

You might be able to work something out using Privoxy. Its blocking and management functions are quite powerful, but you'd probably have to work out your own system for log parsing and sending admin alerts.

 

[demonic]

Thanks will take a look.

I've been looking at Squid (http://www.squid-cache.org/) and Dansguardian (http://dansguardian.org/).

The biggest hurdles I can see, is getting them to work in a mode that doesnt need the traffic to pass through them. Kinda like snort that can work in promiscous mode.

But if the traffic does need to pass through them, then no biggie.

If anyone has experience with squid or dansguardian, or knows other open source solutions. Then please let me know

 

[the maddman]

Or you could run Squid as transparent proxy, with either SquidGuard or Dan's Guardian. You'd also get the benefit of being able to cache static data at the proxy. I see about a 50% bandwidth reduction with 100 users here.

EDIT: Whoops to slow. To get squid to work you do need to pass the traffic through it, but in transparent mode you can have your firewall forward all port 80 traffic to the proxy to be handled. It's pretty slick, but you can't proxy SSL traffic that way.

 

[the maddman]

Thanks will take a look.
Kinda like snort that can work in promiscous mode.

You can also use Snort for things like this, but it is mainly a network monitor, and doesn't really block anything. I've made a passive ethernet tap, which is great, since you can't detect the Snort box in anyway on the network, so far I've only played with it though, I haven't had time to get a fully working system setup.

 

[demonic]

Think I have found what I need : http://www.kathmannlabs.net/mediawiki/index.php/Open_Source_Web_Content_Filtering_Project

Except the link for the vmware appliance isnt working.

If anyone knows of a good link or has this, let me know.

Cheers

 

[Frank]

Don't get your hopes up too high.

I often work at a company where they have a terribly restrictive proxy server, and while I don't browse for porn and stuff at work, I do value unrestricted internet access greatly. So I use an anonymizer.

Normally, the free ones work great, but if I really wanted unrestricted and secret access to anything out there, I would spend a few bucks each month on a good one, that tunnels through anything, encrypted if needed, and is essentially unblockable by any convential measure.


Then again, your average Joe wouldn't even know those things exist.

 

[the maddman]

Don't get your hopes up too high.

I often work at a company where they have a terribly restrictive proxy server, and while I don't browse for porn and stuff at work, I do value unrestricted internet access greatly. So I use an anonymizer.

Normally, the free ones work great, but if I really wanted unrestricted and secret access to anything out there, I would spend a few bucks each month on a good one, that tunnels through anything, encrypted if needed, and is essentially unblockable by any convential measure.


Then again, your average Joe wouldn't even know those things exist.

If the admin really wants to be a real pain, you can even stop that. Snort can trigger on any unencrypted web traffic, and on machines the admin controls, you just install your own SSL root cert and you can proxy (aka play man in the middle) on SSL connections. Then just ban anything else out to the internet. It's just a lot more work for hardly any gain. Here where I work, you use the local proxy, or you don't get internet access.

 

[Frank]

If the admin really wants to be a real pain, you can even stop that. Snort can trigger on any unencrypted web traffic, and on machines the admin controls, you just install your own SSL root cert and you can proxy (aka play man in the middle) on SSL connections. Then just ban anything else out to the internet. It's just a lot more work for hardly any gain.

Agreed. You (the admin) lose in the end, unless you simply turn of internet access altogether for those cases.

Here where I work, you use the local proxy, or you don't get internet access.

That is the case with many companies. But simply using a free anonymizer will get around that. And SSL (https) is normally not blocked, because many important sites (banks, for example!) use those. Playing man in the middle is not going to help there, if at all possible (depends). So you can pretty much tunnel anything you fancy through that in almost all cases.

 

[the maddman]

That is the case with many companies. But simply using a free anonymizer will get around that. And SSL (https) is normally not blocked, because many important sites (banks, for example!) use those. Playing man in the middle is not going to help there, if at all possible (depends). So you can pretty much tunnel anything you fancy through that in almost all cases.

Thanks to all the data control laws now, it's being more and more common to terminate SSL connections at the firewall/proxy so they can be logged. It really comes down to how much personal use of the Internet is alright on company time.

 

[Frank]

Thanks to all the data control laws now, it's being more and more common to terminate SSL connections at the firewall/proxy so they can be logged. It really comes down to how much personal use of the Internet is alright on company time.

As a programmer, I often need very random info. That can be from things indirectly related to the process I'm automating, to things that you don't want normal users to know about. And when I'm in a blind, I'll relax and browse some random forum, or other interesting things to get my mind off matters.

If you try and allow only the things that someone deems acceptable, who is the person you have in mind while making that list? Do you want to block anything that is low-level and related to computer security? I do have the need to know about all that stuff. Do you want to block Google image searches or things used to customize your computer? I do need those to be able to make that program. Do you block online stores? I might be developing just that, and would need to see how others do it. Forums and newsgroups? Half the info I need is found there, often at very unlikely ones. Etc.

For example, if you would block anything related to games, because that's not something you want "your" users (Joe Sixpack?) to visit during work hours, I wouldn't be able to find a lot of relevant info. Or browse here.

 

[the maddman]

As a programmer, I often need very random info. That can be from things indirectly related to the process I'm automating, to things that you don't want normal users to know about. And when I'm in a blind, I'll relax and browse some random forum, or other interesting things to get my mind off matters.

If you try and allow only the things that someone deems acceptable, who is the person you have in mind while making that list? Do you want to block anything that is low-level and related to computer security? I do have the need to know about all that stuff. Do you want to block Google image searches or things used to customize your computer? I do need those to be able to make that program. Do you block online stores? I might be developing just that, and would need to see how others do it. Forums and newsgroups? Half the info I need is found there, often at very unlikely ones. Etc.

For example, if you would block anything related to games, because that's not something you want "your" users (Joe Sixpack?) to visit during work hours, I wouldn't be able to find a lot of relevant info. Or browse here.

I consider developers a special case. I'd just setup Squid with a bucket system to start throttling the connection after the first gig of data. Personally, I think the best way to monitor internet usage is watch the hourly usage of people. Someone that heavily browses the web for an hour a day isn't the problem, it's the person that's always on it when they have work to do.

 

[Frank]

I consider developers a special case. I'd just setup Squid with a bucket system to start throttling the connection after the first gig of data.

I could live with that.

Personally, I think the best way to monitor internet usage is watch the hourly usage of people. Someone that heavily browses the web for an hour a day isn't the problem, it's the person that's always on it when they have work to do.

Well, I can understand that. But it's a heck of an amount of work. And that person that is online for most of the day, while having work to do, would be me. And most other IT people.

Then again, I think most white collar workers really need broad internet acces to be able to do their job, and are in the same category. Google is your friend, whatever you do. As long as it requires independent thought and computer access.

Curbing that would limit their productivity severely. And if it doesn't, it's most likely simply because they don't use all those resources, while they should.

 

[Frank]

Btw.

I don't think it's feasible to try and solve this problem with a purely technical solution, if there even is one. Instead, I like the pragmatic approach: simply monitor excesses, and tell their boss it will cost him money if it continues. You can probably think of many ways to fill that in. The simplest being, that he has to pay you money for the actions you have to take the next time it happens. Or the BSA, or whomever.

That works equally well against illegal software, regular fixes because someone was doing things he/she shouldn't and any other problems you encounter.

 

[the maddman]

I could live with that.


Well, I can understand that. But it's a heck of an amount of work. And that person that is online for most of the day, while having work to do, would be me. And most other IT people.

Then again, I think most white collar workers really need broad internet acces to be able to do their job, and are in the same category. Google is your friend, whatever you do. As long as it requires independent thought and computer access.

Curbing that would limit their productivity severely. And if it doesn't, it's most likely simply because they don't use all those resources, while they should.

I've found it's pretty easy to tell who's doing research and who isn't from the logs on the proxy machine. I've had one user pull down 4 gigabytes a day just from myspace + ebay. The person that is downloading gigs from autodesk.com isn't a problem. I have a bunch of Python scripts to look for trends, and then I'll modify them to ignore what's really business related.

Of course, I'm not trying to dig through logs for thousands of users either, that would add a lot of trouble to it.

 

[nkathman]

Think I have found what I need : http://www.kathmannlabs.net/mediawiki/index.php/Open_Source_Web_Content_Filtering_Project

Except the link for the vmware appliance isnt working.

If anyone knows of a good link or has this, let me know.

Cheers

kathmannlabs.net is my site, and I've put the VMWare virtual appliances back up. When the site hit the homepage of digg and del.icio.us, I had to take it down as it would have quickly overwhelmed the bandwidth. It's now on a dedicated server. Let me know if you have any problems pulling it down.

 

[demonic]

kathmannlabs.net is my site, and I've put the VMWare virtual appliances back up. When the site hit the homepage of digg and del.icio.us, I had to take it down as it would have quickly overwhelmed the bandwidth. It's now on a dedicated server. Let me know if you have any problems pulling it down.

Hey,

I recently went to the site and found its updated. I downloaded the latest VMWare release and it rocks. I wrote you an email, saying thank you. So check your mail.

I was going to deploy it in my production environment and then to evaluate it for myself and then to roll out to a small group of users from there on. This will be deployed to a maximum of 100 users. But my I dont have my IPOD cable which had the VMware instance of it.

There were a few niggles with it, which I wrote you in the email.

Im looking at this, for my first real project of getting to grips with Linux. I'll be installing IMspector as well, as I want full logging activity of my network.

So definately, check your email as I wrote about making a setup guide for this and as really, if this is as good as some of the corporate ones out there, like corporate guardian. This really does need to be shouted about.

Will be giving thoughts about it in the next week or so. As will be configuring it, deploying and testing it, etc, etc.. My boss definately wants reports

 

[the maddman]

Hey,

I recently went to the site and found its updated. I downloaded the latest VMWare release and it rocks. I wrote you an email, saying thank you. So check your mail.

I was going to deploy it in my production environment and then to evaluate it for myself and then to roll out to a small group of users from there on. This will be deployed to a maximum of 100 users. But my I dont have my IPOD cable which had the VMware instance of it.

There were a few niggles with it, which I wrote you in the email.

Im looking at this, for my first real project of getting to grips with Linux. I'll be installing IMspector as well, as I want full logging activity of my network.

So definately, check your email as I wrote about making a setup guide for this and as really, if this is as good as some of the corporate ones out there, like corporate guardian. This really does need to be shouted about.

Will be giving thoughts about it in the next week or so. As will be configuring it, deploying and testing it, etc, etc.. My boss definately wants reports

If you're already fiddling with VMWare, setup another virtual machine with which ever Linux you've picked to run, take a snapshot of it, and go and get your hands dirty! VMWare is a great way to just try things without any worries about doing any real damage. Don't worry if it doesn't seem to be sinking in at first, it took me 3 tries to get started with Linux, back when I got a hold of RedHat 4.2. Best of luck with everything.

 

[demonic]

If you're already fiddling with VMWare, setup another virtual machine with which ever Linux you've picked to run, take a snapshot of it, and go and get your hands dirty! VMWare is a great way to just try things without any worries about doing any real damage. Don't worry if it doesn't seem to be sinking in at first, it took me 3 tries to get started with Linux, back when I got a hold of RedHat 4.2. Best of luck with everything.

Well unfortunately, I have time constraints. Theres alot to do on this network, until I have time to investigate stuff.

Eventually, I'll have time, but not yet.

 

[the maddman]

Well unfortunately, I have time constraints. Theres alot to do on this network, until I have time to investigate stuff.

Eventually, I'll have time, but not yet.

Understandable, I just don't like to rely on systems I don't understand.