The hidden capabilities of windows firewall

Windows, beginning with XP SP2 contains a decent firewall. It doesn't have leak prevention or outbound connection filtering. However it does have: inbound connection filtering, ICMP filtering, a default deny policy, GUI and command line interface, configuration using group policy and something I discovered only recently: limiting a certain rule with multiple IP / netmasks (up to around 10). It also comes preinstalled with the OS.

What this means that you can open up a given service only to the people who need to access it. This provides an additional security layer. Why make it possible for the entire world to brute-force your SSH account? Drop them at the firewall! This doesn't replace a good password, but is an additional layer of security

Some more random firewall related advice:

Port based rules are preferable to application based rules, because with them you enable one port with one protocol (TCP or UDP) for a given set of IP addresses, while if you use application based rules, every port for that given application will be enabled. This is important for application which open up multiple ports (for example an FTP server which has also a web administration interface or Apache, which opens up a port for HTTP and one for HTTPS).

Configure the binding address / listening address for servers correctly. For example:

• If your computer is multi-homed (it has multiple network interfaces), by default most of the servers will listen on all the interfaces. If you need them to listen on only one of the interfaces, specify it as the binding address. Also limit the access from the firewall to that particular subnet (remember, multiple layers of security!)
• Usually you can specify the listening address in the form of <ip address>:<port>. If you use 0.0.0.0 for the ip address, the server will listen on all the interfaces (not recommended!). If you specify 127.0.0.1, it will listen only on the loopback interface. This is a special virtual interface which you can connect to only from the local computer. This is the recommended configuration for administration interfaces. database servers and any other service you only want to connect to locally (or using an SSH tunnel, which from the programs point of view is a local connection).

Here are some further references:

• Customizing Windows Firewall
• Netsh, the Network Services Shell
• Troubleshooting Windows Firewall settings

And remember, if you want to change the listening port for remote desktop for an additional layer of security on a machine you don't have physical access to and you don't want to call technical support, the correct order of steps is (which I learned the hard way :)):

1. Open the new port you selected on the firewall for TCP connections.
2. Verify that you can access that port using a tool like netcat of TTCP
3. Change the registry
4. Reboot the computer and pray that it comes up :)
5. Connect on the new port to remote desktop
6. Remove the rule for the old port

Graceful degradation people!

I went to check out Less Accounting (because I saw it at the Web 2.0 show). Here are two screenshots, one with javascript enabled and the other with it disabled:

Do I really need javascript to submit a simple form? Really? As for the server running the given website, its version number isn't too reassuring either.

The Problem with Programming

Via Raganworld comes the following interview with Bjarne Stroustrup (you know, the C++ guy :)):

The Problem with Programming

My favorite quote:

There are just two kinds of languages: the ones everybody complains about and the ones nobody uses.

(the article has also a nice threaded discussion possibility added. almost as nice as the one at codeproject)

How to create a well behaved bot?

From the ever watchful master of bots and scrapers incrediBILL (:-)) comes the following tip, which you might find useful if you yourself plan / need to implement a web crawler as part of your product / service.

Nmap online

Some brave people have put online a service where you can scan an arbitrary computer with Nmap using their server to do the scanning on your behalf. One can say that this is the grownup version of Shields Up!.

I was worried that their service could be abused in an attack-by-proxy (or in this case reconnaissance-by-proxy) scenario, so I contacted them. They were kind enough to reply to me very fast, and here are the things I learned:

• They are confident that the limits set in their Terms of Service are enough. At the moment of writing this post these are the following (but they can change of course):
• There can be only one active (running) scan from one IP
• Maximum 5 scan requests from one IP per 24 hours
• Maximum 20 scan reqests from one IP per 7 days
• Scan timeout is 60 min
• An other safety measure is the fact that the logs are kept and are correlated with the requesting IP.
• When I saw the possibility to enter an arbitrary command line, I got very nervous and asked them if they considered command line injection? Thankfully they said yes and reassured me that they taken the necessary security measures (and also that their logs were full of attempts to verify the existence of an exploit)

Given all this, I can only say: scan away! :)

Threaded vs. Flat discussions

This post from Coding Horror talks about a subject near and dear to my heart: the usability of the discussion boards. Here is my take on what constitutes an easy to use environment:

• Flat discussion works best if few replies are expected for a given topic. You can make sure of this by splitting up the conversation by topic.
• Threaded discussion can be usable, but you must make sure that the user can keep track of the context. Two negative examples: the comments for Security Focus articles. More precisely: the way that the expand all link works (it shows a only a little bit of each post instead of showing the full text, making it unusable). An other problematic site: TechRepublic. Their biggest problem is that the post is placed at the top, without regard for its position in the tree. While this may eliminate some design problem, it completely breaks the association between the post and its context. Now a good example (IMHO): CodeProject. It uses javascript to load the message you request and display it in-line, keeping the context. When javascript is disabled, it uses server side scripting to imitate the same behavior (you can test this easily if you are using Firefox with the NoScript extension).
• For returning members it is very important to quickly find out what's new. While the method of color-highlighting commonly used kinda works, the best way IMHO is to create an RSS feed for your forum.

And remember: software is not the solution, it's just a tool. You can find articles and podcasts (part 2) about the human element which the most important factor in the community.

Things I read / listen to - part2

While going through the motions of checking for new items on Google Reader, I realized that many great blogs / podcasts were left out from the last list I published. For example Casting from the server room which is a (great) podcast similar to In The Trenches (and not surprisingly they are both members of Friends in Tech), so I decided to use the sharing feature of Google Reader to make my reading list available (unfortunately you'll need javascript enabled for this to work):

If you want to subscribe to it (yeah, right like anyone is interested in what I read :) ), you can do it here: . I will also put this widget in the side bar.

If you dot' succeed at first...

try harder.

I was trying to get Ubuntu (6.10) to hibernate, without much success (remember, I'm a Linux newbie). One of the side effects of the experiment was that Ubuntu did not recognize my swap partition any more (probably because during the hibernation-attempt it was overwritten with some random data from memory).

If you find yourself in a similar situation, you can run the Gnome Partition Editor, by typing in a console sudo gparted or pressing Alt-F2 and typing gksudo gparted. You should see an unknown type partition in the list. Delete it and (re)create a partition with type linux-swap in the free space. Now reboot your computer. This should solve the problem.

Warning! Using the partition editor, you can seriously damage your computer and lose data! Only use it if you know what you are doing!

Update: please read the followup on this advice.

Things I read / listen to

After getting mentioned on a great security/forensics blog, I thought I list of the feeds I watch. This is not a comprehensive list (I have at least twice as much in my Google Reader), but these are the ones with which I try to keep up:

• 2600.com - Off the hook radio show. While sometimes more political than technical, always fun.
• A Day in the Life of an Information Security Investigator - interesting real life examples
• A List Apart - the bleeding edge of web design
• Andy, ITGuy - interesting blog of a security professional
• anti-virus rants - while there are some errors here and there, a useful read
• Anton Chuvakin Personal Blog
• Assa’s blog
• Authentium Virus Blog - the voice of the people who will be kicked in the rear end by MS (for bypassing PatchGuard)
• Binary Revolution Radio - a great show about computer related stuff
• Bits from Bill
• Ctrl-Alt-Del Online - a great webcomic. Don't start from the first one, unless you have some serious free time and someone to call the emergency room when you can't start laughing :)
• Cyber Speak podcast - a great podcast about forensics
• eEye research - these are a smart bunch and I want to know their every move :)
• F-Secure weblog - mostly marketing free material
• Gimme Hardware/Software Interface
• In the trenches - a great podcast for IT pros and sysadmins. Part of the Friends in Tech network.
• IncrediBILL's Random Rants - a guy obsessed (in a good way) with stopping uninvited bots from crawling his site.
• Intellectual Hedonism - a nice podcast about .NET
• Invisible Things - Joanna Rutkowska - do I need to say anything else?
• Larry Osterman's WebLog
• LugRadio - Linux User Group radio - very nice
• McAfee Avert Labs Log - somewhat marketing oritented. Don't always believe what it tells you, but at least look at it.
• Mondays
• Network Security Blog
• Offensive Computing
• PaulDotCom - one of the best security podcast out there for the knowledgeable people. Hy Twitchy!
• Pro::PHP podcast - I was a PHP developer and I still like to follow the happenings in the community
• The Practical Webdesign Podcast - a great podcast about webdesign with Paul Boag (see also boagworld)
• RaDaJo - a security blog
• Ruby On Rails podcast - I like to listen to it and imagine what it would like to use Ruby on Rails :)
• SANS - Internet Storm Center
• SecuriTeam Blogs
• Security Cadets
• Security Garden - while there is little original information, it is a good gathering place for interesting (security related) stuff from around the 'net
• Security Now! - the podcast which delivers the material for many of my rants :)
• Securosis.com - a very good info-sec blog. The articles are long but they are worth the time reading them
• SploitCast - nice podcast. Also check out their forum.
• StillSecure, After All These Years
• SunbeltBlog
• Tenable Network Security
• The Cold Fusion Podcast - while I never used CF, I like to take a look at how others do it :)
• The Old New Thing - as Raymond Chen says: not actually a .NET blog, or as I put it: hardcore Windows API stuff
• The Oracle at Delphi - if you are a Delphi programmer, a must read
• The Security Catalyst - nice podcast, a little hyping from time to time
• The Security Roundtable - it seems it will die, but the past discussions are still nice
• virtualdub.org - VirtualDub - the best video editor out there (and it's free software!)
• WebAxe - a web accessibility podcast
• Website Management Advice - a very, very good podcast about all that is related to websites.
• Websense blog
• Wikipedia Weekly
• Windows Weekly
• Rootkit.com

Enjoy! (And don't blame me if you won't have time for your job/spouse with this many great stuff out there :))

You can make a difference!

Just a quick post: if you find something on the global 'net that you don't like, blog about it! It is amazing how quickly (a) you find out that you were wrong at the thing you criticized should be actually so or (b) get a response from the person that they are actually working on it.

Version B is actually what happened to me today. I was looking through my mail and was amazed that somebody from Websense actually responded to my don't reinvent the wheel post stating that they know about the problems and they plan to fix it very soon. Thank you.

You can make a difference!

Cutting off user-mode

With every release Microsoft tries to separate user-mode and kernel mode more and more. Some say that this is a temporary solution, however it is still important. These hacks were probably done in the name of efficiency back in the days, but this is largely irrelevant today with as much computing power as we have and should be rapidly eliminated, otherwise they undermine the security of the system.

Now for the fun part of it: it is possible to change the IOPL level of a user mode process from user mode. What this means that you can directly control all the hardware from user mode (think DMA controller, HD controller, etc). For those of you who don't want to fiddle with the policy editor, download PsTools and use the -s command line switch - don't forget to give the full path name to the executable. And yes, it works with Vista too if you are Administrator (I've tested using a pre-RTM build, but I don't think they changed anything in the RTM build).

Be safe out there and remember: don't run as root!

PS: If you don't have access to a compiler, you can grab the exe here. Just remember, it will restart your computer without a warning!

• File size: 36864 bytes
• MD5: 19cd8a70f199df4182eb198818e6c782
• SHA1: 7c5dc5ab1c36b3876bedcb3641513a06b75bf453

Hack the Gibson - Episode #67

This will be a short post, because the podcast (sorry, the netcast) was very on the spot. Also this was the first time I've heard that Steve included an errata. Things are definitely improving. I hope that I made a small contribution to this.

One fun fact: the GRC.com website is running on Windows with IIS. While the fact that the normal pages return GRC Custom Hybrid NanoProbe Engine/1.57 (experimental) as the server software (captured with ServerSpy) proofs that (a) Steve is a geek and (b) he knows how to configure his box, the error pages reveal the truth behind the curtain:

The page cannot be found

The page you are looking for might have been removed, had its name changed, or is temporarily unavailable.

Please try the following: If you typed the page address in the Address bar, make sure that it is spelled correctly. Open the home page, and then look for links to the information you want. Click the Back button to try another link. HTTP 404 - File not found Internet Information Services Technical Information (for support personnel) More information: Microsoft Support

Stopping waves

I came across a very nice article over at the SploitCast forums about stopping waves by being calm and not by trying to create a counter wave. My interpretation of it: loose the hype! You won't create any long-lasting effect with it. If you truly are out to improve security, be calm and explain over and over again. You can make a difference!

Fixing Open Source

Before you all jump over me: I won't be talking here about a silver bullet which would take the open source movement in the right direction, ensure its competitiveness or something like that. What I'll talk about is the joy that you can have by fixing open source scripting libraries.

To be specific: in one of my Perl scripts I wanted to send mail. The problem was that I didn't have access to a SMTP server on that machine. After toying around with the example code of Net::SMTP::Server I concluded that it didn't work and went off to find some lightweight solution. One of the more promising solutions was RunMail, however I was still uncomfortable about running binary code of unknown origin on the server. As a results I fired up telnet and the Perl debugger and stepped into the library source code. There I found the following part:

    # Loop through the recipient list.
    foreach $target (@{$self->{TO}}) {
 my $rr;
 my $domain = /@(.*)/;
 my $res = new Net::DNS::Resolver;
 my @mx = mx($res, defined($1) ? $1 : hostdomain);

I promptly replaced the line so that $1 would be set, and voila, it works!

    # Loop through the recipient list.
    foreach $target (@{$self->{TO}}) {
 my $rr;
 $target =~ /@(.*)/;
 my $res = new Net::DNS::Resolver;
 my @mx = mx($res, defined($1) ? $1 : hostdomain);

Now, if this would have been a binary library, I would had to: (a) get the source code (b) set up a build environment (c) compile from source (to get the debugging symbols) and (c) figure out how to integrate the debugging version with the release version such that I get a breakpoint where I need to and I get the debugging symbols. And I didn't even speak about the rich introspection capabilities of scripting language debuggers!

Update: the actual problem was in Net::SMTP::Server::Relay, not in Net::SMTP::Server.

Update: below you can see the full code of the module (since it is rather short). The part shown with bold is the one which needed to be changed:

package Net::SMTP::Server::Relay;

require 5.001;

use strict;
use vars qw($VERSION @ISA @EXPORT);

require Exporter;
require AutoLoader;
use Carp;
use Net::DNS;
use Net::Domain qw(hostdomain);
use Net::SMTP;

@ISA = qw(Exporter AutoLoader);
@EXPORT = qw();

$VERSION = '1.1';

sub _relay {
    my $self = shift;
    my $target;
    
    # Loop through the recipient list.
    foreach $target (@{$self->{TO}}) {
 my $rr;
 $target =~ /@(.*)/;
 my $res = new Net::DNS::Resolver;
 my @mx = mx($res, defined($1) ? $1 : hostdomain);
 
 next unless defined(@mx);
 
 # Loop through the MXs.
 foreach $rr (@mx) {
     my $client = new Net::SMTP($rr->exchange) || next;
     
     $client->mail($self->{FROM});
     $client->to($target);
     $client->data($self->{MSG});
     $client->quit;
     
     last;
 }
    }
}

What am I reading?

I've read two and a half :) really interesting posts today (warning, they are pretty long) about computer languages:

• Bambi Meets Godzilla
• a little anti-anti-hype
• The departure of the hyper-enthusiasts

And I thought I share some of my opinions about languages. I'm polyglot myself and have some experience with many types on languages.

• GWBasic / QBasic - this was the first language I've learnt. It was lot of fun even when I was typing programs which I didn't understand from books and magazines :)
• Logo - this was the second language (if we don't count BAT files) which I've used. I found it interesting but I went rather fast back to QBasic because it lacked the functions to interact with the external world.
• Pascal - This was my next step and for DOS Turbo Pascal (6.0 and later 7.0) was awesome. For windows not so much. I've looked at some source code for Windows and immediately got a headache (this was before I understood event driven programs and the theory behind Windows programs).
• Visual Basic 3.0 - my first experience with visual IDEs under Windows
• Delphi - this seemed to combine the best of both: visual IDE from VB and Pascal. My only complaint was the file sizes of the generated executables. Then I discovered KOL, but it wasn't as easy to use and didn't have all the third party components.
• C / C++ - I played around a little with them in Visual Studio and DevCPP, but I never fully mastered it. My suggestion to anyone looking to do this professionally would be: read the Thinking in C and Thinking in C++ series from Bruce Eckel and only touch code after you understood everything that is there, because these languages are very powerful but also very dangerous, like a sharp knife.
• PHP - with it I discovered the joy of scripting languages and web programming. However over the years I found that I was writing the same code over and over again. I'm planning on trying out something like CakePHP or Symfony.
• Haskell / Lisp - We cursory touched upon these languages in our university curriculum and they both seemed interesting, however my impression was that they didn't have a big enough library behind them to do actual work.
• Smalltalk - again, I was introduced to it during university and it was really interesting (we were using Squeak), but the image concept seemed to radical.
• Javascript - this is a language which I used for a long time (because I too wanted to add interesting effects to my webpage), but only recently did I learn the more advanced functionality (like prototypes and anonymous functions) and best practices (like unobtrusive javascript)
• Java - as many others I got introduced to Java through applets. It is a fine languages (garbage collection rocks!) and has a very extensive set of libraries, however I don't really use it these days because scripting languages are quicker to get work done and C / Delphi is better suited for low level OS stuff (like directly calling APIs)
• VB .NET / C# - Very nice languages and it's good to see that the size of executables is back again to normal, however this comes at the expense of the fact that everybody has to have the framework installed, and not everybody does. And it really doesn't look good when you say to people: you want to try this? Go download and install a 20+ MB framework!
• Python - It is a fine language, however it doesn't have the set of libraries Perl has (with CPAN). It is also a little more verbose than Perl.
• Perl - This is my current scripting language of choice. It has an extensive set of libraries (see CPAN), it's relatively cross-platform and has a very compact (but from time to time very cryptic) syntax. As I learn more and more of Perl I get a more stronger feeling that PHP is a stripped down Perl and really don't want to go back to PHP.
• Ruby - I didn't actually use Ruby, however I saw a couple of tutorials about Ruby on Rails and I got really interested. Two things that concern me are efficiency (if Rails always looks up the structure of the database, doesn't each request takes longer?) and security (the blending of the development and deployment environment concern me). I'm sure that there are perfectly good answers to both of my concerns, I just don't know them yet :).

Windows PIO mode fallback problem

Windows (starting from Win2k I think) has an interesting feature: after a certain number of failures on a given IDE channel, it reverts that channel from DMA to PIO mode. This is perfectly reasonable for hard disks (although a user warning would still be useful), but for CD-ROMs this is deadly because a bad CD can produce many read errors without the readers fault. And if the CD-ROM is reverted to PIO you'll see a considerable degradation in performance. Here is the solution which worked for me (attention! this is rocket science stuff! only do it if you know your way around the registry!):

1. Fire up the registry editor
2. Navigate to the key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4D36E96A-E325-11CE-BFC1-08002BE10318}
3. You should see keys numbered from 0001. Look for one which contains a DWORD value named MasterDeviceTimingModeAllowed which has a values less than 0xffffffff.
4. Change that value back to 0xffffffff.
5. Create a DWORD value with the name ResetErrorCountersOnSuccess and the value of 1.
6. Reset your computer and verify in device manager that you have DMA enabled for the CD-ROM.

For more information look to winhlp.com or Microsoft.

Update: I've put together a small program which automatizes the setting of registry values. You can download it here, and for the curios ones, you can get the source code here (it was written in Delphi with the ACL library).

The exe file has the following characteristics:

• File size: 12800 bytes
• MD5: 9cbdd9ec65c900b67f75578d082cd940
• SHA1: 50bba9d745059c806823e6b2fc2d8c4503a77e43

Are our leaders becoming dumber?

Usually I stay away from politics, but I'll make an exception from time to time. After listening to a speech of former US president Bill Clinton I have to wonder how people could choose a man who can't put talk for 10 minutes in front of a crowd without making some pronunciation or grammatical error after they had a president who could speak articulately and coherently and respond to non-scripted questions in an intelligent manner.

This isn't a US-only phenomenon. Closer to home, both in Hungary and Romania the current leaders are mentally less capable than the former ones. A sad story from the last elections in Hungary - when they asked a voter why he voted against the former leader he said something along the lines of: when he spoke he used words which I could not understand. I want a president that speaks my language! - all I can say is: get a dictionary.

Don't reinvent the wheel

From the department of don't make your own program unless you know what your are doing come the following two:

The Websense security blog RSS feed keeps marking the first post as new. While it is a very interesting post (about analyzing shellcodes form browser based exploits), I don't want to read it five times. Also they make you agree to a license agreement before they show you the link to the RSS feed, they don't put it in the head section of the document so that intelligent browsers can autodiscover it and it's not possible to comment on the posts. And one last funny things: in the feed the copyright message is as follows:

Copyright 1994 - 2005 Websense, Inc.

Someone should tell their legal department what year it is :).

The other example is the Lavasoft blog. No RSS feed, no comments, no history, just a big HTML page with the stories concatenated. Not event a box that says enter your e-mail address to receive updates. This is a page that I won't be visiting often. Just an other proof of what a hype-machine Lavasoft is. (If you didn't know, at one point they used a multiplier to say that they had more signatures than they really did)

Cookies and FUD

Thanks to Security Garden I found this very useful article about Anti-Spyware products and Cookies. It is a useful and unbiased study IMHO. It also points out some misinformation spread by quite respectable organizations on the Internet.

Webroot says (the incorrect information is highlighted in red):

Adware tracking cookies trigger the most benign form of pop-up ads. Cookies that reside on your PC serve up pop-up ads related to the site you are visiting, your personal information or online activities. The more cookies you have on your PC, the more pop-ups you'll see. A more malignant type of pop up won't stop appearing until you allow it to download spyware software. Sometimes, even the "close" or "cancel" buttons mean yes. A slightly different pop-up, which mimics a Windows Messenger Service window, is also a ploy to download spyware software onto your PC.

As you can see close to 50% of the description is inaccurate and 50% is not relevant to the subject (it talks about pop-ups). Very useful for the client.

To get a more balanced and technically correct description which a technically not so savvy user can understand visit the page of the Electronic Frontier Foundation.

An other nice looking utility

Here is an other utility that looks great: HT Editor. Editor / Hex Editor / Disassembler / Assembler. I'll take it for a spin and get back to you with the results.

A nice little utility

A nice little utility to view your logs in real time, if you are on windows and didn't install GnuWin32 or Cygwin: BareTail.

Microsoft, pantents and rant

One of the blogs I monitor Security Garden. It is a nice blog, which although it doesn't contain any original content, is a nice gathering place of interesting and useful pieces of information from around the Internet.

However I must take issue with two recent posts.

What irks me is that these people who steal software are the very ones who are responsible for upping the cost for everyone else. It takes additional research and development to add the anti-piracy mechanisms to the software. In addition, because of them, every time the rest of us holding legal licenses want to download software from the Microsoft site, we need to go through the WGA song & dance. It doesn't matter if it is software, a music CD, a video game, a coat or a loaf of bread. Stealing is stealing.

This comment shows in my opinion ignorance. Piracy does not affect software production costs in any significant way, because a very small percentage of the people who pirate a given software would buy it if they were forced too. Most of them just wouldn't use it. That's one of the many reasons the numbers published by organizations like the BSA are inaccurate (an other reason being that they don't take into account the fact that people might use software from non-BSA members, or even FLOSS software). Also the analogy of stealing bread is very, very bad. By now we should all have learned that with computers information sharing is not a zero sum game. If you copy something its not like the other person loses something. Stop using bad analogies because they lead to bad conclusions and bad legislation.

The second post is about the Linux community wanting to know which patents does Microsoft claim to have in the Linux code. This isn't just lawyer talk. This is of real interest to the community, because as soon as they have a specific claim and that claim turns out to be true, the developers will jump at it and rewrite it so that it no longer infringes any patents. If on the other hand Microsoft is not willing to reveal the exact details, it pretty much sounds like FUD and blackmail. After all, the intent of the patents is make information public, not to hide it (those are trade secrets).

On the same note: I was listening to a podcast (unfortunately I can't find it again, but if I do I'll post a link here), in which an ex-lawyer lady talked about how her consulting firm helps Fortune 2000 companies (yes, you've read right, 2000) to guard their IP. What I've found funny that during the whole podcast she kept using the terms IP, trade secret and patents interchangeably. She said several times that companies need to guard their patents against leaking out. Now this coming from a lawyer illustrates the level of FUD some people are spreading. Patents are meant to be public!

Show us your patents!

You can do anything with Perl - part 2

You can do anything with Perl - part 2

I'm a PHP person myself and only the last several months did I start to play with Perl on a more serious level. One thing I've learned pretty fast: you can (almost) anything with Perl and the first thing you should do when you try to solve a problem is to look on the CPAN.

What I want to show today is how to capture the output that your own script produces. I wanted to emulate the ob_start / ob_get_contents combo from PHP. The main idea below is to replace the STDOUT handle with a pipe and after all the output was pushed to that pipe, retrieve it. One word of warning: the close WRITE; is imperative. Without it the while loop reading the pipe would hang indefinitely.

To pack or not to pack?

After listening to an other great CyberSpeak podcast, I decided to line up the pros and cons of executable packing for programmers.

First of all, what is executable packing? In short it is similar to self-extracting archives, where as a result of the process an executable is generated which contains some unpacking code and the original code in a compressed form. A free (as in speech) one is UPX, which you can check out over at sourceforge and play around with it.

Why would you use such a thing? First of all it makes your program smaller without any noticeable effect for the end user (that is s/he doesn't have to extract the executable from an archive before running it, s/he can just double click on it and run as if nothing has changed). Second of all it protects your code from the very beginner reverse engineers (to see all the options available to a more experienced one, just listen to the podcast mentioned above). A third argument used is that the load time of your executable is smaller (it loads faster) because a smaller amount of data must be transferred from disk. This may or may not be true depending on the usage pattern, but as I describe later it definitely has an effect on the memory usage (in the negative way).

Now why shouldn't you pack your executable? First of all it will look suspicious. One of the best task-manager replacements out there even marks it with a different color. Some AV products will flag your executable as suspicious if you use certain packers to compress it (like Morphine) because these packers are known to primarily be used in malware.

Now for the biggest reason to avoid packing programs and libraries (DLLs): it consumes more memory. Under Windows (and probably under Linux too), portions of the memory can be marked as being a mapping of a portion of a given file. When such a mapping is created, just the portions of the file which are actually accessed are loaded in memory. The rest is just marked by the memory manager as being available when needed from the file. What this means that during the loading of a normal, non-packed executable, it is mapped to memory, but only the parts are actually loaded from the disk which contain code that needs executing. Now in the case of a packed file the whole file needs to be loaded in memory, because the unpacker needs to go through every bit of packed data to unpack it before it can run the original code. Even worse, when the system is running low on memory, if you have an upacked executable, it can just throw out the memory pages it occupies (if the process is idling), because it knows that it has the exact same data on the disk. In the case of a packed executable it first has to write out the memory pages to the swap file, because they are different from the contents of the file on the disk. Now for the last point (but the most important one in the case of DLLs): the Windows memory manager is able to share memory pages across processes if their content hasn't changed since their loading. Specifically in the case of DLLs, you will have the same code loaded in each process which uses them. So the memory region in which the DLL is loaded is shared between all the processes using it (there are some exception, but for sake if simplicity we'll ignore those). How does Windows define unchanged memory pages? Those pages which map to the same area of the same file and hasn't been written to. This means that as soon as you have a packed DLL, it will have a separate copy for each process using it, because what is in memory doesn't mach what is on disk and thus Windows can't share the given memory pages between processes. This means that each process that uses your DLL will have a separate copy of it. This effectively multiplies the memory needed by your DLL by the number of processes which use it! Oops...

Update: fixed typo.

Hack The Gibson - Episode #66

Read the reason for these posts. Read Steve Gibson's response.

This again will be a short one. Steve talks about Vista which I have no immediate experience with (I've seen it on some decent machines and all that I can say is that it's reeeeeeeeeeeeally slow. really, really, really slow. Event without the Aero interface).

Now for some fun (although it isn't so funny when you think about how many people listen to the podcast and get erroneous information): BitLocker and booting from an USB device has nothing in common even though Steve implies this. And TPM (the Trusted Platform Module) is not (just) a pre-boot technology. The external USB device is needed (if you don't have a TPM) to store the encryption key (similar to the way you can setup TrueCrypt). And the TPM is basically (there are of course some details like validating the BIOS before running it) a secure place to store stuff. An encrypted memory.

Now about the topic of Patchguard, Kernel patching & co.:

Even though Steve implies the contrary, there are many documented functions to create firewalls (TDI drivers, NDIS drivers, Filter Hook drivers and the Windows Filtering platform which again is just an evolutionary step, not a revolutionary one as Steve implies and you certainly don't have to wait for Vista to write a driver which uses well documented standardized APIs) and AV products (by using File System Filters and Registry Filters).

In my opinion most of the people participating in this debate are coming at it from the wrong way. No offense to anyone, but to understand the whole picture and to be able to make a fair decision you must know at least some details about the inner workings of the Windows kernel. I'll try to explain it here as simple as possible, but again, to make up your own mind it's necessary to have read the Windows Internals or an equivalent book. So here goes my version:

In programming there are certain linkage points between components which are written by different groups (think different companies here, like Microsoft and other companies who produce software that runs on Windows). These are called API and are well documented. Their advantage is that the one who is offering it is (or should be) commited to them, meaning that in future version they will work the same way (so that third party software can continue to work without modification). This also means that bugs will be fixed in it. An other advantage (this time from the point of view of the one offering it) is that s/he can change the inner workings of the software, as long as the effects of the APIs remain the same. This is the ideal world.

Now for the real life: sometimes programmers find the available APIs insufficient. This can have multiple causes: (a) the developer doesn't know about all the available APIs (ignorance) (b) the developer is trying to do something that breaks some assumption the system is built on or (c) there is truly no API for this. In this situation the developer might reverse-engineer the system s/he is developing against and try to modify (patch it) such a way that s/he is able to accomplish her/his goal. The are many, many problems with this approach: (a) it reduces system stability - there are many steps one must take to create a reliable patch and if one is missed, the stability of the system is in danger. (b) they are not guaranteed to work in every condition. because reverse-engineering includes most of the time a fair amount of black-box testing and because you never actually spoke to the original implementers of the code, you can never be sure that you covered all the possible situations. (c) they can be broken at any update of the system. because the original vendor doesn't know about this patch, there is no way it can guarantee that a future updated version is not going to break it.

The actual debate is about the fact that some vendors said that the current API is insufficient, but did not say what other APIs they would need (most probably because there is a documented API almost for anything including for notification when a process is created). They created products which rely on patchwork, even though this puts the customer at risk and now that they have to get their act together are whining. Or they created some dubious HIPS product which is pretty much useless. So no, Vista won't contain less security, it will contain more security. Even though some companies brag that they bypassed the patchguard, I'm sure that Microsoft will modify it so that they mechanism gets invalidated. Would you like to buy a product which stops working for a couple of days randomly after security updates (until the the given company catches up - it they do)?

Tracking Users Via the Browser Cache

From the department of old things I didn't know about comes the following bit:

Tracking Users Via the Browser Cache. Original story: meantime: non-consensual http user tracking using caches. Also covered here: Clearing cookies is not enough to save your privacy. And it was already posted on slashdot (so please don't post it again :)).

As you might know I personally believe that to gain 100% privacy and anonymity on the 'net is almost impossible and if you take all possible steps to achieve it, you get a web which lacks many of the functionality which make it so attractive. In the end everybody must decide for themselves, but probably most of the people have better things to do than running around in a thin-foil hat. Now that I've got this off my chest, lets see the technical details for this new method:

Whenever an object (web page, image, etc) is downloaded from the web server, additional headers are returned which signal to the browser (and any proxy which may be in between) how long the given object should be cached (caching is essential and it improves page loading speed substantially, so no, turning it off is not the solution). The idea is to give the browser an object (and again, this can be even an html page, so turning off scripts, images & stuff won't make you bullet proof) with a header that tells the browser that the object must be revalidated on every request (revalidation is the process where the browser asks the webserver: I got the version of this object which was produced on date X, do you have a newer one? And the webserver responds either with no or with the new version). Based on this the server can tell exactly when this particular client last loaded a page from it.

This is just an other possibility to track users on web pages (you can read about others on my blog). Possible defenses would be: (a) turn off caching (and live with very low performance) (b) some finer grained caching control (which to my knowledge doesn't exists in any browser) or (c) embrace the truth: as long as they have something that you want, they can demand the price.

On the upside: it seems that currently this is not suitable for cross-domain user tracking without some additional stuff (like javascript - then again if you have javascript enabled, all kind of other technologies can be used).

All natural numbers are special

From the department of too-geeky-joke-to-be-funny comes the following one:

All natural numbers are special:

• 0 is special because it is the first natural number
• 1 is special because it is the neutral element for multiplication
• 2 is special because it is the first even number
• 3 is special because it is the first odd prime number
• lets suppose that there exists a non-empty set of non-special natural numbers
• we can take the minimum of this set - but this number is special because it is the smallest non-special number
• QED :)

(Original source - in Romanian)

Spammers use Google as redirector

It seems that this isn't a new thing (see others noticing it here and here), however I've been just hit by a couple of these today, so I blog about it :)

Google offers a service which creates redirects with arbitrary targets. Just enter http://www.google.com/url?q=<an-url-of-your-wish< and it will issue a 302 permanently moved header and redirect your browser to the site of your choice. What's the big deal you ask? The big deal is that (a) people trust Google (b) there might be some white-lists in anti-spam filters which contain the link to Google.

How this should be fixed (IMHO): links should be encrypted before sending them as parameter. For example instead of http://www.google.com/url?q=http://www.example.com, one should use http://www.google.com/url?q=e86d9f95ad063cf36eb8e7 where e86d9f95ad063cf36eb8e7 is the encrypted version of http://www.example.com. For added security the encryption key should be dependent on the current time (the day of the year for example) and also the redirector should check the current and the previous key (so that if it was Nov 19 when the user clicked the link, but when the request arrived at the Google server, it was already Nov 20, s/he shouldn't receive an error).

What is not AJAX?

Not everything involving browser scripting is AJAX. The following two things are not AJAX:

1. Yellow fading effect on web pages
2. Downloading and running an executable in Internet Explorer if you have your Internet Zone security level set to low.

HIPS - just a pretty UI?

Disclaimer: the viewpoints and ideas expressed here an entirely my own and are by no means representative for any institution I am affiliated with. Also I do not want to offend anybody, since I realize that the amount of work it takes to create some of these programs and the wealth of knowledge one must posses for it is enormous (most of the features implemented are possible only using kernel mode drivers).

To start up: what are HIPSs? The term is a little nebulous (you can read the Wikipedia article to see the different shapes and forms it takes), but based on the experience I had with products it is a software application which can apply security restrictions on a per-application base (rather than on a per-user base). I will be talking about Windows based HIPSs in this post.

Because the term itself is hard to define, lets take a concrete example: you have an IM client running on your system. You can define the security boundary of this application with regards to the system resources (in windows this being: the file system, the registry and the network). This would look something like this:

• It needs to have read-only access to the following files: the executable, extensions / plugins stored in DLLs, ...
• It needs to have read-write access to the following files: the conversation log, ...
• It needs to have read-only access to the following registry keys: ...
• It needs to have read-write access to the following registry keys: ...
• It needs to communicate over the internet with the following host on the following TCP ports: ... (and BTW, this doesn't necessary mean all the hosts one chats with because most of the big IMs route the messages through a central server to avoid to have to deal with NATs).
• It has to listen on the following TCP port for incoming connections from a given set of hosts (or from any host): ...

Once you defined the security boundary, you can enter it in the configuration file of your HIPS and let it enforce it. Here are my reasons I think these systems are unusable by 99.9% of the Windows user base:

It is too complex - how many users are able to take a piece of software and define its behavior at the level required by a HIPS? Probably less than 0.01%. There are several approaches trying to get around this, however none of them very usable:

• Some HIPSs have a so called learning mode which is activated when the software is first installed or the policy for a new executable must be defined. The problems with this is:
  • it assumes that the system is in a clean state when installed
  • it won't learn non-frequent operations (because they are not executed during the learning period)
  • it may learn malicious operations and include them in the policy
  • if there is no way to transfer this policy from computer to computer, in a large scale environment every instance of the software must be trained separately
• Other products have a predefined database (possibly upgradeable from the Internet) which contains a list of known executables and the policies which need to be applied to them. This database may either be put together by the creator of the HIPS or the community around the product. Setting aside the issue of trust (if it is a community based effort), it isn't very likely that there can be a comprehensive database of policies at the rate new software comes out. An other issue would be that users might need to delay upgrading their software because no policy exists yet for the new version, even though this would expose them to vulnerabilities (if the update includes security fixes) or hinder their ability of operating (if the new version is needed to open some files). It would become a chocking point much like patching is currently.
• Yet an other approach taken is to classify the actions taken as good (for example opening a text file from the desktop), dubious (writing a new DLL to the system folder) and bad (overwriting an existing executable in the system folder). The problem with this approach is that there is no inherent goodness value associated to this actions. It all depends on the context. By exploiting the 80/20 rule the creators of the system may be right 80% overall, but their accuracy may drop to 0 in some cases. For example setup programs usually write files to the system directory, so an alert asking the user if this action should be allowed is wrong 99% of the time. This also means that the user will be trained (subconsciously) to click the allow button whenever s/he wants to install something. This will lead to zero protection against adware and spyware bundled with setup programs (or even trojans which disguise themselves as setups for useful programs).

It is useless for large scale systems (enterprises) - as mentioned earlier large scale zero-administration deployment of such a system is impossible and at best they can serve as a kind of parental control by allowing only the execution of programs which are permitted by the company policy. This can be however done by properly using the configuration properties included in Windows (which is integrated in AD - meaning that it's easily deployed to a large number of systems)

It is a hack / patch - again, I want to emphasize that I don't mean to offend anybody, but the application centric model of access control (meaning that rights and privileges are associated with a given application) offered by the HIPSs is almost orthogonal to the user based security model included in Windows (where rights and privileges are associated with user accounts and groups). Without discussing the merits of one approach over the other, just think about this: many people spent years of their lives thinking about the security system included in Windows (also known as discretionary access control)?. It is also a standard described in the orange book. Now lets ask the question: is there a possibility that a similar amount and quality of work (yes, Windows has some quality code in it if you know how to configure it right) can be duplicated in a 1-2 years by teams much smaller than the force behind MS and the orange book? If your answer is not (as is mine), you should be really worried that a technology which is still is in its infancy replaces your mature security system.

An other piece of software - installing a HIPS means adding an other piece of software to your system. An other piece of software means an other set of bugs. And especially in something as critical as a security system you should rely on something that has proven itself rather than on some new technology.

What could Microsoft do? - Create user awareness about the security features, add a GUI which makes their configuration simpler and finally add a method to alert the user when a program tries to do something it's not allowed to (this is the biggest problem currently: if a program tries to do something it is not allowed too - because it was written with the assumption that it will run under the administrative account for example - the OS won't give you any help in figuring out what went wrong. You have to rely on the program to give a meaningful error message - which most of the time isn't the case and many times it doesn't give any error message at all - or use some additional tools to figure out what went wrong). In my opinion as Microsoft integrates its security with the UI in a better way, HIPS system will become irrelevant. One such example is the LUA implemented in Vista. One area where MS was lagging behind was the firewall where one could restrict based on the application or on the protocol/port combination but not both and it wouldn't filter outgoing traffic. From what I've understood this was resolved in Vista (it was an implementation restriction not a technical one). Probably the more limited version for WinXP SP2 was chosen to be able to deliver it on time and avoid conflict with personal firewall vendors.

Hack the Gibson - episodes 63, 64 and 65

Read the reason for these posts. Read Steve Gibson's response.

Hello all. I have little time lately to blog. Also the latest episodes of Security Now seem to contain less and less errors, allowing me to do one post for three episodes.

Episode #63

Steve Gibson says: But frankly, you know, a personal firewall, a third-party personal firewall is also a kludge because, you know, it’s doing something to Windows that Windows was never designed to have done to it - I have to take issue with that. The kernel of windows is very extensible. Actually there are at least three documented and supported interfaces to write firewalls (one that is not mentioned there is an NDIS filter driver). An other good link listing the possible extension points in the Windows network stack: http://www.ndis.com/papers/winpktfilter.htm. I repeat: the Windows kernel (and the whole system) is very extensible (this is one of the difficulties when trying to configure a secure system - you have to know all the possible extension points).

kernel mode printer drivers - with this one I don't know exactly what's the situation, it may be that MojoPac support people don't know what they are talking about. Since Windows 2000 the printer drivers are implemented in user mode, as you can read on the Microsoft site (click on Is there any advantage to rewriting an existing kernel-mode driver to run in user mode?).

Episode #64

It was fairly good, however there were two topics I wanted to mention, both of which I've talked previously on my blog: Third party cookies and transparent proxys (in the context of counting podcast downloads).

Episode #65

I actually can't complain regarding anything that was said in episode #65. It was a good (and philosophic) one.

Cookie viruses? Me thinks not

The only reader of mine had a question: what is my opinion about cookie viruses? (If you also read this blog, I apologize and also I'm werry happy in this case that I have more than reader. If you have questions or topics you would like me to discuss, please post them in the comments)

Getting back to the topic here: I don't have an opinion, since there is no such thing as a cookie virus. By definition the a virus is (quote from Wikipedia):

A computer virus is a self-replicating computer program written to alter the way a computer operates, without the permission or knowledge of the user.

There is no such thing because cookies should be (and usually are) treated by browsers as opaque tokens (that is they are not interpreted in any way and are sent back exactly as received to the server). Now one could imagine the following really far-fetched scenario which would be something similar to viruses:

A given site uses cookies to return some javascript which is evaluated at the clientside by some javascript embedded in the page (that is the code embedded in the page is looking at document.cookies and doing an eval on it. Now in this case we could make the client side javascript do whatever we want, however:

• If we can modify the client side headers, probably we have very such access to either the client or the server that there are much more malicious things that we can do.
• The javascript will be executed in the very limited context of the web page, so we could only infect other cookies that go to the same site (but we already had access to it when modifying the first cookie, so there is no reason for using such a convoluted method.

Now many sensationalist sources use the word virus to refer to all kind of malicious actions to drive up hype (and we all know what my opinion is about that). There are however some real possibilities of doing harm, most of them in the area of information theft and input validation.

• The first one (which doesn't fit in any of the two mentioned categories) is the possibility that there is a buffer overflow exploit in the cookie handling code at the server. In the official standard it is stated that a cookie should be now larger than 4096 bytes and we all know that when something like this is in the spec many coders automatically write char buffer[4096];. However before you think that this is a 0-day against Apache or something, let me say the following: I threw together some quick code an ran it against an Apache server (2.2.something) and it very nicely refused to accept the headers. It also generated a return message which was properly escaped, so there is no possible XSS vulnerability there. I'm also sure that IIS has no such problem but maybe some home-brew custom http servers might have this problem.
• A scenario on which some papers are focusing is the following: the cookie contains some text which is relayed back to the server, which in turns embeds it in the HTML output without proper sanitization. This can result in the attacker embedding code of their choice in the page, including javascript, however such an attack has no real-life benefit, since if the attacker can access the clients cookies, s/he has probable write access to the file system and can do much more nefarious things with much less complication.
• A third possibility would be that the server relies on data contained in the cookie for authentication or for some other action. In the first case there are two vulnerabilities: cookie theft and creating a custom cookie to gain access (if the server relies for example on some value to be present in the cookie to indicate that the user authenticated successfully). The second case would be when there are parameters in the cookie for a server-side process (shopping cart information). If the server has no way of validating the information upon receiving it or doesn't do so, one could manipulate this information to gain advantages (to buy things at a 0 price for example). Ideally this information should be kept in a server-side session storage or if you don't want to break the REST model, encode it in the URL, but make sure that you provide a way for the server to verify the posted back information, by for example encrypting it and then appending a salted hash (where the salt is only known to the server) to it and verifying this when receiving a new request.

In conclusion: Developers - validate your input! validate your input! validate your input! (at every step)

How to know what you've installed?

I'm doing a presentation on wireless as a school project (so that it can be included in a book ;)) and came across the following problem: neither OpenOffice 2 nor Gimp can read SVG files so I had to convert them in something which they can display (preferably PNG since it can preserve the transparency and still use as many colors as necessary). Browsing through the Synaptic Package Manager I quickly found a package which seemed to do exactly that: librsvg2-bin (and it was a small download too, aroung 70KB), however after installing it I had no idea how to use it. If you find yourself in a similar situation, you can find out what executables the given package installed by going to /var/cache/apt/archives and looking for the package you just installed. Open it (it's just an archive) and in the CONTENTS directory you can see all the files that were installed.

A final tip: if you wish to remove the files from the cache (to free up disk space), do the following from the command line: sudo apt-get clean.

Do NOT simplify when you are explaining!

Or at least make it very clear when you do so! Simplifications and metaphors are very dangerous because they hide some detail deemed non-important by the person who is using them. But you can't know in which context your explanation will be read (if you are posting it on the Internet), so be sure to always include a warning if you use such techniques.

What prompted this post was an article over at ITToolbox which claimed that files can be deleted under Linux while in use, because when linux runs an executable it loads the whole file into memory and accesses it from there. This is incorrect and was pointed out in may of the comments, however the real reason which made me write this posting was the authors defense (posted as a comment):

I wrote this article to be a very simplistic model. I primarily aimed this at people who have no understanding of the technicalities of file systems. Before I wrote this I did a quick google and found no comparable article so I hammered out a few lines. While I agree that it is not to the letter correct technically I believe it is correct in a general view.

Now there are two possibilities:

• The author did not know the exact technical details of the mechanisms and offered his understanding of it - this is perfectly ok, but when the more correct explanation was posted in the comments, he should have updated the post with it. There is no shame in not knowing something, only in trying to hide it!
• The author did in fact know the technically correct explanation and as he stated, he was trying to offer a simplistic explanation for the not-so-technical people - in this case, as I've said earlier, he should have indicated this clearly and maybe include the correct explanation in the post for the people who do want to know. By not doing this he misleads users who are technically savvy but have no experience with Linux into believing that Linux has an inferior swapping mechanism or that it consumes more memory as shown in the following comment:

  I am a Windows user and suddenly started liking Linux flavours on its features. You say that 'When linux runs an executable it loads the whole file into memory and accesses it from there. This means that there is no connection to the physical file on the disk drive. When the program is closed and all connections to the file are cut the file is deleted from memoy' will this require a lot of memory when compared to a windows machine ??

BTW the correct explanation as written by Bley in the comments:

This is incorrect. Linux will *not* load a program into memory on startup. It will load pages of the program from the disk file into memory as needed, and discard them when they're no longer needed (or when the kernel needs memory for other programs); that's OK, as it can always recover them from the disk file. There is another (subtler) issue. Under Unix (and Linux), files aren't actually deleted until they're no longer in use. So if you have a program that keeps a file open and then you delete it, you won't be able to see the file in the directory listing, but the file is still there; the running program will be able to continue to see it as if it was still there. The file gets deleted when it's no longer in use by any programs. The same applies to the kernel wrt executables: if you delete an executable while it's running, the kernel will defer the actual deletion of the file until the program exits. There are still issues that make updating more complicated than it seems. For example, when updating a package, you need to update multiple files: executables, libraries, configuration files, data files, and so on. If the executable in the package is running, it will continue to see the old version of the files that it has open at the time of the update. That's OK; it's the old version of the executable that's running at that time. But if it needs to open new files, it will open the newly updated version, and this may cause unexpected results (crashes, etc) because of version inconsistencies.

So, digg - wisdom of the crowds? Not always...

Update: the author of the article included an update in it which clarifies the technical part.

The perfect solution?

I've been experimenting with different collaboration solutions, and I think I may just have found the one: Socialtext. What I really like:

• It is open source (just click the for developers link)
• It is free (if you don't want the support)
• It is written in Perl
• It has an API for manipulating the pages (I don't know yet how powerful it is)
• It has a WYSIWYG editor
• Commets can be attached to every page
• It integrates with Active Directory

What I don't like (these are my first perceptions while browsing around the site, I didn't have time yet to install it and do an extensive testing):

• The access control seems to be a little limited (like admin / non-admin), I would have preferred something more along the lines of Active Directory or at least the Unix model with groups and group members
• It doesn't seem to support code blocks. I thing one of the major features of DokuWiki is the integration with GeSHi for syntax highlighting of source code.

I've heard about it over at Perlcast

Two contest you might enjoy

Two contests you might consider checking out if you are a security person:

• The Hitchhackers Guide to the Galaxy - HTML / scripting oriented
• The malware analisys quiz 7 from SANS - a challenge oriented more at disassembly, and be aware! this is a real malware sample!

Good luck

Tracking web users

Again, this will be something new here (at least for me): I'll publish a pre-rant for Security Now! Steve Gibson expressed interest in the subject of cookies, so I'll tackle that in this post and also the more general question of user-tracking. I discuss different ways it can be accomplished, ways you could protect yourself and the question: should you?

In a way the World Wide Web is a marketing companies wet dream: just image, tracking the moves of the users, building a profile which lists their potential interests (as it can be inferred from the list of visited sites and the frequency of the visits). Using this they can show ads which they consider will be relevant to us. Of course they don't do this out of the goodness of their hard. They do it because you have a higher probability of reacting to the advertisement if it's relevant to you.

Here are the means I know of which can be used to accomplish this:

• Tracking cookies or third party cookies - this is IMHO a bad name (from a technical point of view), and I'll explain in a minute why. But first lets answer the question: what are cookies? Cookies (or HTTP State Management Mechanism as it is referred to by the official RFC) are opaque tokens (from the point of view of the client) which contain some information which helps the server side application identify the fact that different HTTP requests are part of the same session. This is necessary, since the HTTP protocol does not define any method for creating, tracking and destroying sessions. That is, whenever you request an object from the web server it will treat it as separate request, having no idea what you requested earlier. The cookie is used as token in the following way: the server says to the client take this piece of information and return it to me on subsequent requests. This way it can determine if the request is part of the same session (because it can hand out a different value to each client and when the client returns the information, it can identify the session it is part of). Before you ask: you can't use IP addresses as a reliable unique identifier because of proxies and NATs. You can observe two things here: this behavior is entirely voluntary on the clients part (it may choose not to return the token) and that it applies to every HTTP transaction, not just HTML documents (including images, flash animation, java applets, etc). Of course the standard defines a policy which specifies in which requests should the cookie be returned. The elevator speech version of this is: cookies will only be sent back to requests targeted at the server it was originally sent from and to elements the path of which is prefixed by the path contained in the cookie (for example if the cookie was set by the object located at http://example.com/set/a/cookie it will be sent in all requests which are targeted at the example.com server and contain in the url /set/a/cookie). Now how is this used to track you from site to site if the cookie is only returned to the server it was originally sent from? Enter the advertisement companies: they serve up ads from the same server to many webpages. This means that those webpages contain links to elements (usually images, flash animation or javascript) which reside on the server of the advertiser. This means that if you view a page which contains advert from a given company, it can set a cookie, which will later be sent back to it when you view an other page (possibly from an other server) which contains advert from the same company (because in both cases the object - image, flash, whathever - came from the same source the cookie was set - the server of the advertiser). This is called a third party cookie because it is set by a different entity than the server you see in your address bar. However I think that this is a bad name since it implies that some kind of spoofing is going on, like a server is setting a cookie for an other server - which by the way is explicitly prohibited by the standard and won't work in any modern browser. To sum up:
  • Applicability: (almost) every browser supports it. The standard itself if relatively old (almost 10 years)
  • Customizability: Current browsers offer ways to set a policy on what cookies should / should not be accepted both in a whitelist and blacklist format. Usually they do not include the option to view the cookies stored on the machine, but there are many free third party tools / extensions which enable you to do this.
  • Risk of disabling it: if cookies are disable altogether, many sites which have a member-only area will break and the user will be unable to log-in. Disabling of third party cookies breaks pages which host elements fetched from a third party server (which represents a small but growing percentage of the web in the age of mashups)
• Flash Local Shared Objects (AKA flash cookies) - As of version six (also called Flash MX) a feature was introduced in the Flash Player to store information which had to preserved across different page loads locally on the users computer. Before that sites used a combination of javascript, cookies and actionscript to obtain the same effect. Flash Local Shared Objects have the same restrictions as cookies for forwarding (i.e. they're only sent to flash movies which originate from the same server). Because this was a little known feature outside of the Flash developer community and the interface was hidden and because of the scaremongering many users started to remove or disable cookies, advertisers started to use it instead of cookies.
  • Applicability: on any platform which has at least version 6 of the Flash Player installed.
  • Customizability: you can go to the site of Adobe to completely disable or to manage the shared objects which are on your computer. There is also a Firefox extension, however it seems dated and not maintained any more, so probably the safest bet is to go with the official links provided above.
  • Risk of disabling it: sites which rely on it may break, however I didn't found any sites until now which relied on it for other purposes than tracking, so currently it may be disabled without any problems. This may change in the future however.
• Referrer URLs - Referrer URLs is a piece of information sent by your browser when requesting an object from a web server. For example if you click a link at http://foo.com/link.htm which takes you to http://bar.com/target.htm, the bar.com webserver will receive as part of the request (if you didn't disable it in your browser) the string http://foo.com/link.htm as the referrer. This can (and is) used by sites for statistical purposes (to see who links to them) and for security (however this is a pretty weak form of security since it relies on the client playing it straight and thus it can be spoofed. One thing which makes the privacy advocates suggest to turn this feature off is the fact that if you go to a page from a search engine (that is, you searched for bar.com on google and then clicked on one of the results), the target server can know the words you searched for (since it will be embedded in the referrer url). However, this information isn't forwarded to the advertisers unless the use third party javascript to get it (which I'll talk about later on). That is if you go: Google -> Google search results -> foo.com -> (automatically, because it is embedded in the page at foo.com) advertiser. The referrer transmitted at the last step (that is from foo.com to the advertiser) if foo.com (meaning that the only information that the advertiser gets is the fact that the ad was loaded from foo.com, not the way by which the user arrived to foo.com. I want to stress this because Steve Gibson got this wrong on episode 64 of the Security Now podcast. (I want to stress again that advertisers can get the referrer of the page which includes the advertisement by using third party javascript which I'll talk about shortly).
  • Applicability: on almost every browser
  • Customizability: you can see a tutorial about enabling it here which should point you in the right direction.
  • Risk of disabling it: you shouldn't encounter any problems because few sites use it for other purposes than statistics, but if you don't mind, give them this piece of information, it can be used to create better content for you!
• Third party javascript - usually when a site collaborates with a given advertiser, it is asked to put a piece of HTML in every page where s/he want the ads to be displayed. This code is usually an IFRAME tag or a SCRIPT tag. In the later case we talk about third party scripts - javascript code which is provided by a third party and runs in the context of the current page. This code can do almost everything, including the following things: access the referrer of the current page (so even if it isn't directly relied to the advertisement server, the script can forward it), get information about the browser capabilities (screen resolution, etc) and perform history digging (see the next point).
  • Applicability: on every browser which understands javascript.
  • Customizability: in Firefox you can use the NoScript extension. In Internet Explorer you can add the sites you want to block scripts from in the Restricted Sites Zone. An other solution would be to disable javascript entirely, but this will reduce the usability of many sites.
  • Risk of disabling it: mashups use heavily third party javascript (to embed Google Maps for example). Also some big sites host their script files on different servers than the content (to be able to optimize the servers for the specific types of files), so you can't say generally that everything third party is bad.
• History digging - This is a really cool technique, reported first as far as I can tell by Jeremiah Grossman and was later tweaked to work with IE. It is based on the fact that visited links have different styles than non-visited links (this is usually observed as different colors). If you put a bunch of links on a page and then use javascript to inspect the styles applied to them by the browser, you can tell if the given sites are in the history of the browser.
  • Applicability: there is proof of concept code for Firefox and IE. It should work in any browser which has a standard conformant implementation of javascript and DOM.
  • Customizability: you can't programatically disable just this feature. Your options are: (a) disabling javascript (b) cleaning your history before you visit sites you suspect are doing this. One important fact: if an advertiser embeds javascript on the site the ad is displayed on, it can use this technique to find out if you visited a given site. Fortunately there is a mitigating factor: in order for somebody to find out if you visited a given page s/he has to know the exact url of the page (that is this method can not be used to enumerate the entries of your history)
• Sign-in information - an often overlooked fact by people is that the big three identity providers (Google, Yahoo and MSN) also provide advertising. Because of this they can correlate tracking information obtained by any of the methods listed above with the personal information you provided at signup. Now I'm not saying that they do this, I'm just saying that they have the technical means to do it.
  • Applicability: if you are a user of any of these sites and browse sites - while you are logged on - which display advertisement from them, you are affected.
  • Customizability: log off before browsing to other sites and clear all the cookies from them. Before logging back in also clear the cookies from them placed there by the ads.
  • Risk of disabling it: the inconvenience of constantly having to clear cookies.

Now for the philosophical question: should you be worried? Should you go to great length to avoid this tracking, even at the cost of breaking useful features on the site? You should consider the following ideas (they are not absolute truths, but arguments which are used in this debate):

• Nothing is free and advertisement is an (arguably) quick and (mostly) painless way of payment for the content / service. So disabling advertisement can be thought of as a way of cheating to get what you desire without payment)
• Contextual ads can be useful. For example if I would like to buy a laptop and I see an ad for laptop, I will most probably click it. This is useful for both parties: for me because possibly I learn about an offer I didn't know about and for the company who put out the ad, because I might buy something from then.
• Some people say: but this is not right! The user should be in control! If you want to buy laptops, search for them yourself! Of course no rational person (no offense to anybody) would buy something of significant value based on one ad (because usually it's only showing one detail of the product - probably not mentioning the not-so-bright sides) but it may add value to your research. So, while you shouldn't buy based on what they say on the teleshopping channel - err I mean ad :) - it may add value to your research while you are considering your options.
• The tinfoil hat people may say: I don't want the government / Amazon / Google / whatever track my every movement! I have a right to privacy! - and they are right, they do have a right to privacy, however they must be willing to give up certain benefits or to make some additional steps. And before you object saying: why do I have to make extra efforts to get the same service everybody receives while keeping my information as private as possible? - just consider how things work in the real world - if you want to drive a car, you must get a license. It is your right to drive a car (if you are of legal age), however you still have to get a license. Because every analogy breaks down, lets consider the technical point of view: every technology can be used for good an bad (this is even more so if there is no clear distinction between good and bad). The only way of preventing 100% of the bad usages of a technology is to ban it all together. You may choose this, but be aware that you are not getting the benefits either. Now some of the technologies (like session cookies) can be emulated by other technologies (like appending the SID - the session identifier to every request as a GET parameter), however the given technology was introduced to make it easier to accomplish certain tasks without the complication and hassle the old method needed. Guess, what a rational website owner / creator would do: use the more complex, less reliable and more expensive technology for a very little percent of its visitors or go with the easier and more powerful technology?

Stars of the earth

Hello all (of my 2 readers, one of which is my girlfriend :) )! This is my 50th post, so I thought that I do something special with it. The blog has grown steadily until now and I have many more subjects that would like to write about. The picture below stands as a small reminder: even things which seem far away can be reached if you choose the right route.

If you didn't get it, the lights which can be seen and resemble the stars are actually lights of a city which is behind the observer and is reflected in the car window.

(Don't try to guess my location by applying some clever algorithm to the picture, because they are just two random pictures grabbed from flickr and combined with the help of Paint.NET and IrfanView.)

Talking out of your head (as opposed to an other body part)

Recently a hoax / misinformation / hype is making its way around the web (or at least the part of the web I see ;)). I'm talking about the article title Internet Explorer 7 - Still Spyware Writers Heaven. While I'm by no means a MS fan and criticized the IE7 team for not making some features available under Win2K3 and WinXP, for which I believe that there is no sound technical explanation (only a marketing one), I must absolutely can't stand misinformation, even more so when it seems that the author wants to generate hype!

This is exactly the case with this individual. He refers to a very old attack vector when creating a DLL with the same name as one that was loaded by the application resulted in loading the "malicious" DLL. As of WinXP SP2 (which everyone should have installed by now, otherwise your computer won't last for 10 minutes on the Internet), the search order for DLLs is the following (taken from the official MS page):

1. The directory from which the application loaded.
2. The system directory.
3. The 16-bit system directory.
4. The Windows directory.
5. The current directory.
6. The directories that are listed in the PATH environment variable.

Now lets analyze this list from the point of view of IE. The current security settings on my IE folder (in program files) is the following: Administratos - Full Control, Power Users - Modify and Users - Read and Execute. In other words you can only create files there if you have at least power user privileges, but if malicious code runs at that level you are pretty much screwed anyway as you can do much more damage than that.

Now if you run as a normal user (which you should be!), your only hope is to write to a directory in the path (or alternatively change your path so that it includes a given directory). In this case the DLL would be loaded IF it couldn't be found in any of the other places. This scenario is only possible if some application uninstalled itself and failed to remove the registry entry for the DLL.

To sum up:

• This vulnerability has been fixed a couple of years ago (in WinXP SP2)
• If you run your browser with high privileges, you don't need this method to alter the system. It can be used to hide files, but then again if you have that high privileges you can directly install rootkits.
• If you run with low privileges (as you should), the only attack would only be possible if you uninstalled a program which registered a BHO without referring with the full pathname to it and the uninstaller program deleted the dll, but failed to delete the registry entry. With other words: very, very unlikely.

My advice would be: run with low privileges (as user) and don't read articles from people who don't know what they're talking about.