Putting the eval into Java

“eval” (short for evaluate) is usually the name given to the method in dynamic languages which makes it possible for the programmer to access the compiler / runtime. Here are a few links to the documentation for the function in different languages:

• Javascript
• Perl
• PHP
• Python
• Ruby
• LUA

They are usually used to quickly evaluate a DSL (Domain Specific Language) expression. What I mean by this is the following: lets say that the user supplies an expression which can be easily (ie. with a few string replacements or regular expressions at most) converted into a valid expression in the current language. Then you don’t have to write your own lexer / parser / runtime to support this function.

To make this example even more concrete, lets say that you are implementing a simple graphing calculator where the user can supply the right part of the f(x)=... expression and you draw the function for a given interval of x. If the user supplies something like 1 + 2*x + 3*x*x, this is pretty much a valid expression in all programming languages (there are minor syntactic differences to be precise - like Perl/PHP requiring you to prefix variable names by the “$” sign), so you could simply use “eval” on it.

Warning! Running eval on unverified, user supplied code is a really, really bad idea! (yes, I know that red and bold underline is a little over the top, but this is just that bad! Never, never, ever do this! It is equivalent to letting everybody connected to the Internet (assuming that we are talking about an webapp) running arbitrary code on your server. Implement very strict filtering (based on whitelisting if at all possible) for such features!

Surely, you would say, such a dynamic feature isn’t easily accessible for a statically typed compiled language as Java... And you would be wrong! As of Java 6 each JVM install (including the JREs) includes the Java compiler, and it also includes a public API to access it. Using this feature you can implement the Java equivalent of “eval”: giving a string to the compiler and getting a class instance back, on which you can call methods. You can find the source in my SVN repo. It is (almost entirely) based on the following article from 2007 (just to give you an idea how long this option has been around): Create dynamic applications with javax.tools. An other (pleasant) surprise was the fact that this process doesn’t require any security privileges and works perfectly in restricted environments such as browser.

An additional advantage of using the JVM rather than your own runtime is speed: many man-hours have gone into optimizing both the source –> bytecode and the bytecode –> machine code transformations. Which brings me to an other possible use for this kind of solution: generating particularized instances of generic classes to give more hints to the JVM about possible implementations.

For example, the StrinkTokenizer class does the following when looking for separator characters:

char c = str.charAt(position);
if ((c > maxDelimCodePoint) || (delimiters.indexOf(c) < 0))
    break;

Now imagine how much more efficient (in the sense of: easier for the JVM to translate into an efficient machinecode) this code would be if we knew that we have exactly one possible delimiter (as it is the case most of the time). Replacing delimiters.indexOf(c) with delimiter == c can give you an order of magnitude speedup for this particular code.

The takeaway should be:

• This is a very powerful technique, but it should be used with care! Only use this method if you’ve proven (by using a profiler for example) that the given class is the dominant factor in the performance picture.
• Be particularly aware of potential security risks which could appear!
• Also, be aware that you give up many things when going this route:
  • Automated refactoring
  • Reports generated by bytecode analysis tools (like coverage or bug detection)
  • Debugger support
• In conclusion: use it with great care, but if used properly, it can result in considerable performance improvements!

Picture taken from Hexadecimal Time's photostream with permission.

Update to the Blogger Tag Cloud

A small PSE (Public Service Announcement): if you were using the Blogger Tag Cloud I’ve put together based on the WP-Cumulus plugin, you might have noticed that it stopped working some time ago (I’m not entirely sure when, since I didn’t notice it, until a reader commented and brought it to my attention – thanks again Soufiane).

The problem was that the server hosting the SWF and JS file didn’t serve them anymore, instead giving a 403 – access refused error. To mitigate this problem I’ve uploaded the SWF file to Google Code and used the JS file from the Google Ajax Library and bought the plugin back to life.

So, if you are using the plugin and you are subscribed to my feed, go to the original (now updated) post and use the new code.

Thank you and sorry for any inconvenience caused!

“Funny things I found while browsing the web” post

The Geek/Nerd/Dork/Dweeb Venn Diagram (via Joel Esler’s blog):

BTW, here is a quick way to convert JPEGs which should be PNGs or GIFs (because they aren’t photos!): simply use a photo editing software (like the GIMP or IrfanView / Paint.NET) and reduce the color depth without dithering. This should pretty much get you there. You might want to play around with the number of colors to retain.

The second one is Pixels by Patrick Jean (via Wondermark):

MC Frontalot released his newest album Zero Day (via the Veracode blog):

Updated YARPG

This has been sitting in my queue for some time: almost four years ago (it’s incredible how time flies!) – amongst the first posts I’ve published on the blog – I’ve written a random password generator in Javascript which I’ve named YARPG (for “Yet Another Random Password Generator”). The advantages to using it are the same as they were back then:

• Customizable (password length, types of characters included, etc)
• Secure (it doesn’t communicate over the network, hence no need for SSL)
• Fully reviewable (as opposed to server-based solutions, where you have to trust the server)

The only flaw it had (as pointed out by a commenter) was the fact that passwords didn’t always include all the characters you’ve selected (ie. the checkboxes represented “possible” not “mandatory” characters, which was a little counter-intuitive).

I’ve thought about how to create passwords which included at least one character from each set. My first ideas were around generating a password, then checking that it contained at least one character from each set and if not, replacing some of the characters with ones from the missing set. However this train of thought quickly ran into problems when I had to decide which character to replace. Choosing something fixed (like the first one, last one, etc) is too predictable. If I choose a random one, I run the risk of overwriting previous change. So finally I realized that there is a simple solution: just re-generate the password until it satisfies all of the constraints. Although this might seem like a brute-force solution, in practice its speed is indistinguishable from a constant-time solution.

Below you have the new and improved YARPG:

Password length:
Include letters
Include mixed case
Include numbers
Include punctuation
Include all printable characters
Quantity	12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364

I've also updated the original posting. You can get the source code for it by looking at the source of this webpage, or from my SVN repository: js_password_generator.html. Hopefully you find it useful!

Picture taken from cjc4454's photostream with permission.

Sending an X-Face email with Perl+GMail

In the latest Software Freedom Law Show Bradley mentioned the X-Face email header and challenged listeners to send them an email containing the X-Face header. So here is the small Perl script I’ve whipped together to send them an email trough GMail:

use strict;
use warnings;
use Net::SMTP::TLS;

my ($from, $password) = ('[email protected]', 'MySuperSecretPassword');
my $mailer = new Net::SMTP::TLS(
  'smtp.gmail.com',
  Hello => 'smtp.gmail.com',
  Port => 587,
  User => $from,
  Password => $password);

$mailer->mail($from);
$mailer->to('[email protected]');

my $data = <<'EOF';
X-Face: "8.]Z_3ptu\NK'CA~DM>M,G.T(h=1.y9"0gXW3V91E:dw2?|&G2R(?/no'F2g4%8Fv.
 J1p5K-^1epKXxIG)mj4}nGWTi<=iz8n)bUVhLu}MXRFl9"J%'=-;IfMXcuPK>-%^;$uW87O/B
Subject: Hello X-Faced World!

email body.
EOF

$mailer->data();
$mailer->datasend($data);
$mailer->dataend();
$mailer->quit();

The code is largely based on this snippet: Sending Mail Through Gmail with Perl. The X-Face header was generated using the Online X-Face Converter (yes, I know that there is a Image::XFace module, but it was very cryptic – it didn’t mention supported input / output formats). One word of warning: if you are using ActivePerl under Windows, Net::SMTP::TLS isn’t available in the default module list (AFAIK, because of encryption restrictions), so you might need to experiment with alternative package sources or using Linux :-). I’ve also tested the script with an email account I control (using Thunderbird with the Mnenhy plugin – which can read but not create X-Face emails) and it worked nicely.

There you have it: how to use an old (from the 1980s according to Wikipedia) method for embedding pictures which is not supported by most of the email clients :-)