But don’t give up yet, all you burgeoning totalitarians out there! For here I shall reveal a much simpler (albeit slightly less efficient) method of voting fraud. Here is what I learned in my absentee-ballot-casting experience. Oh yeah, and dear internets: please use this for good and not evil. And also don’t use it at all.

Today I cast my absentee vote for tomorrow’s Ohio presidential primary at the Franklin County Board of Elections. Like a good citizen, I arrived there prepared with my opinions and my state-issued, valid photo ID. I thought I was packing light by leaving my purse in the car and just grabbing my wallet, but it turned out I could have traveled lighter — you don’t need a photo ID, or any other kind of ID for that matter, to vote absentee. And, since there’s such a large window of time in which you can cast an absentee ballot, you can vote pretty much as many times as you want, provided you have enough people’s name, address, date of birth, and last four digits of their SSN.

Parking was hectic and when I rolled my window down to get parking instructions, I was assailed by propagandists from all sides. I was then directed to a parallel parking space into which the car barely fit. After that I waited in line for over an hour.

When I got to the voting booth without ever having been asked to so much as spell my last name, I asked an election worker, “Isn’t anyone going to ask me for identification?”

“They don’t need to,” she replied. “They have all that information in the computer.”

“So, it would be pretty easy, then, for someone to show up and pretend to be me?” I asked/commented.

“No, because there are certain things they wouldn’t know about you,” she said, matter-of-factly.

The absentee ballot request form requires a voter to fill out their name, current address, and the last four digits of their social security number. That’s right — not even the whole number. This is information that tons of people have about almost everybody. Employers, former employers, universities, high schools, the postal service, the college board, and lots of others have this info about me.

I said, “But anyone with the information on this card” — gesturing to my absentee ballot request form — “would be able to just show up and claim to be me? And no one would challenge that?”

She looked at me as though I’d expressed concern that the ballot-boxes weren’t sufficiently elf-proofed. After an exasperated hand-on-hip-eyeroll-sigh, she demanded, “Now, who would ever go to that much trouble?” and huffed away. My ID never left my pocket.

(Disclaimer: this is probably not a new development by any means. I have never voted absentee before, so I have no idea how long this opportunity for election fraud has been available, and I can only speculate that it’s been a good many years. I also can’t guarantee the efficacy of voting a bajillion times for your chosen candidate, because it seems likely that the number of people on each side of any given controversial ballot issue who carry out this kind of fraud would probably be evenly distributed enough to cancel each other out. And then there’s also the question of whether absentee votes are even counted, or whether anyone’s votes are counted, or whether the word ‘counting’ even applies at all to election polling at all anymore. )

Lately I’ve been working on integrating (or, more accurately, attempting to integrate) the Python scripting language into some plugins for an application we develop plugins for. We’ve wrapped many libraries with varying levels of success, so this one wasn’t going to be much different. Or, so we thought.

Technically, we’re interested in “Embedding” Python — We have a native Objective-C application that needs to make use of the Python interpreter at various stages of execution. The Python scripts are user-supplied, and are exceptionally free-form: the functions they write can take variable numbers of inputs, produce variable numbers of outputs, and can use any Python modules they have installed.

Unfortunately, this makes the embedding process somewhat complicated: From Objective-C, we need to be able to parse the script, find all its functions, and get all those functions’ input and output parameters. From this data, we can expose the Python module’s interface usefully. Unfortunately, Python does not appear to allow this kind of introspection from the outside (and it’s questionable whether or not it’s even possible from the inside.)

In searching for information on this, Numerous Documents are Found that Needlessly Complicate or Obfuscate the difference between Embedding and Extending. As if it’s really that difficult (embed means “fix firmly and deeply in a surrounding mass” while extend means “cause to cover a larger area; make longer or wider”. From these obvious definitions, we can infer that “Embedding” means putting something inside, while “Extending” means adding functionality or abilities. Maybe it’s not so clear for non-native english speakers. I don’t know.)

Another annoying side-trip of this research was an overwhelmingly smug idea that almost reeks of Java. This side effect is the recorded smarmy discussion of why one should “Extend” rather than “Embed” (these reasons are then used to explain away why there isn’t any good documentation on actually embedding Python). The arguments go something like this:

• Python’s so good, cross-platform, and flexible that it’s actually more cost-effective to Throw Out All Your C/C++/ObjC Code And Rewrite Everything In Python.
• Embedding is so cumbersome to code, and so difficult compared to embedding Python in Python, that you should Throw Out All Your C/C++/ObjC Code And Rewrite Everything In Python.
• High-performance code can be written in C, and then called from within Python’s runtime when you actually need to performance boost. Why not just Throw Out All Your C/C++/ObjC Code And Rewrite Everything In Python Except for the fast bits?
• If you embed Python, you’ll annoy Python developers who can’t access the modules they’re used to using. Why not Throw Out All Your C/C++/ObjC Code And Rewrite Everything In Python so you don’t annoy your Python developers?

These arguments cover some pretty diverse ground there. Unfortunately, there are 2 fatals flaws in the list above.

First, embedded Python _Can_ in fact use installed modules just like raw Python, so point 4 above is flagrantly incorrect. There are some suggestions that it’s simply annoying to develop in such an environment because namespaces are all strange (or wrong) and nothing works quite right. Guess What: Welcome To Plugin Development! It’s like these Python developers have never worked on real projects before or something.

Second, All of the above points assume that it’s possible to discard your entire code-base and rewrite everything. While this is technically possible, it’s not very likely when you don’t have access to the source of the application to be discarded (in our case). The unfortunate downside is that _All The Listed “Solutions” Hinge On This One Idea_.

What’s with these non-solutions from allegedly flexible languages? Even Objective-C, a compiled language, and the (totally undocumented) JavaVM-ObjC bridge, offer enough introspection to at least find methods and parameter counts without too much hassle. Maybe Python does this just as simply, but if so, no one’s talking about it…

But this morning he called me with a rather odd problem:

“I hooked up my Treo and started syncing it, then walked away for a few minutes. When I returned, the screen was covered with a bunch of squares, and I can’t do anything.”

He took a few photos with his cellphone camera and emailed them to me (click for full-size):

(You can see bits of the menu-bar scattered around the screen, and it looks like Missing Sync had launched iPhoto. Possibly related?)

I talked him through a few tests. He was able to see the mouse cursor and move the mouse, but clicking didn’t cause any apparent response. He could activate Dashboard — Dashboard itself looked fine, and the checkerboard showed through in the background. He then tapped the power button, and the “Are you sure you want to shut down your computer now?” dialog appeared, but it was scrambled and the buttons didn’t work.

I instructed him to power it off by holding the power button down. Since rebooting, it’s been working fine.

*Confused.*

I googled it a bit but wasn’t able to find any clear reports of this happening to other people.

In the past couple of months I thought about spending some money on either a Nintendo Wii or a multi-purpose media box (not to mention my anxiety over buying an unmodifiable blackbook). Over the past 3 years I’ve looked at smaller cases as “better”. Before that it was always the full tower case that provided more room and ease-of-use. In this vein I’ve been looking at the world of mini-itx for the past several months.

VIA created the mini-itx specification for itself, but since its creation other companies started creating motherboards with this spec and recently so has Intel. The spec allows for a low-power consumption CPU, which is great for point-of-sales machines, thin clients, and more. The other use of mini-itx, popular amongst personal users, is as a Home Theater PC (HTPC) or media center.

There are several solutions available in this market. By far the most popular media box in the past few years has been a Microsoft product, the XBox. You can slap on an open source GPL media player onto the xbox - XBox Media Center (XBMC). This was great as a video, audio, and game machine for dvds, mpeg4 (xvid), and various audio formats.

Alternatively Apple sells the Apple TV, which not only plays movies, but can connect to your shared ITunes library. However as with many things Apple (and not-Apple) it has limited uses even with the hack to allow xvid and DivX support. Author’s Note: I first wrote this before Mac World Expo January 2008.

The two products mentioned above are limited by their age. Both are not powerful enough to decode h.264 matroska encapsulated video or HDMI video input. This is the problem. Would I be able to build a mini-itx machine capable of doing what these once-awesome products do not? My requirement was pretty simple. I’ve dealt with TV Tuners and HDTV before, and I’m not interested in building a true “HTPC” or personal video recorder (PVR). That ship has sailed. Not to mention all the hullaboo about DRM in bluray/hd-dvd, hd capable monitors, video cards, etc…

After investigating hardware articles and reviews of the boards on mini-itx.com I narrowed down my search to the VIA Epia EX 15000G. This newer board offered a “fast” 1.8ghz VIA C7 processor. A PCI slot would take care of a wireless card as I wouldn’t be dragging cat5 down from the router upstairs to the TV. It seemed to offer everything that I was looking for. It even had an on-board graphics chip with a mpeg2/mpeg4 hardware decoder though underpowered in 3d acceleration!

The VIA Epia boards are also highly compatible in linux so that’s a plus. I delved deeper. I wanted to make sure that this board would suffice. After all I’ve never touched a processor other than AMD or Intel.

However I’m afraid to say that the Unichrome Pro II mpeg2/mpeg4 hardware decoder is not usable at the current time. There have been numerous complaints that hardware decoding doesn’t work for .mkv files and even in Windows! Surprisingly the only h.264 playback possible was in linux with the now seemingly defunct Via enhanced MPlayer.

This was the nail in the coffin. I’m not going to buy a board because it *may* work in linux with the supposed mpeg4 hw decoding support (and not at all in Windows). What does this leave us with? Software decoding? The C7 processor is not powerful enough to decode 720p let alone 1080p content.

The cost of this machine and project will significantly increase switching from a mini-itx C7 processor board to perhaps an intel mini-itx core 2 duo mobile board or switch to micro-atx (about $150-$200 increase in budget with a core 2 duo mobile or desktop).

The smallest really isn’t better.

Mac OS X doesn’t have the luxury of working on the same underlying CPU though, so things need to be handled a little bit differently. The solution Apple came up with was the idea of a “Fat” binary, sometimes called a “Universal Binary”. In other words, instead of a single program being contained in a program file, the program file can contain several programs for different architectures. For example:

cwright@phendrana:~>file /bin/ls
/bin/ls: Mach-O universal binary with 2 architectures
/bin/ls (for architecture i386):	Mach-O executable i386
/bin/ls (for architecture ppc7400):	Mach-O executable ppc

or an even more convoluted example:

cwright@phendrana:~>file GLTools
GLTools: Mach-O universal binary with 4 architectures
GLTools (for architecture ppc7400):	Mach-O bundle ppc
GLTools (for architecture i386):	Mach-O bundle i386
GLTools (for architecture ppc64):	Mach-O 64-bit bundle ppc64
GLTools (for architecture x86_64):	Mach-O 64-bit bundle x86_64

This increases file size considerably (4x in the last example), but it provides you with the cool side effect of being able to drop the exact same program onto a PowerPC Mac, and have it operate identically — as long as you’re properly handling architectural differences such as endianness. Overall, this is a pretty slick solution, and with the exception of a few small tweaks, I doubt I could have come up with a better idea. (The small tweaks, in case you’re wondering, would be shared data segments across the binaries inside, such that non-code stuff only needs to be included once, instead of 4 times. This doesn’t work well when the data contains code though, so you’d need to have flags to control how it operates).

One of many problems rears its ugly head though when developing such portable applications: Linking with static and dynamic libraries.

Out of the box, OS X ships with many libraries that are all appropriately compiled to support all the above architectures, so you never notice this problem when compiling against standard included libraries. However, if you stray off the beaten path, and use another library, you’re destined for trouble. Open Source libraries, especially the ones whose build system depends on the monstrosity that is AutoConf (./configure scripts and all that), are surprisingly difficult to get working. In part, because they’re not designed to be built for multiple architectures in parallel, and in part because AutoConf is infuriatingly worthless when it comes to documentation.

Of course, since I’m writing all of this, I’m obviously in the middle of such a battle :)

By default, I run configure like this:

CFLAGS=\"-Os -fomit-frame-pointer\" ./configure [options]

where options is stuff like --enable-shared and other library stuff. On occasion (only 85% of the time), I also have to override other environment variables because parts like pkg-config don’t work, because other libraries don’t install properly, and a whole host of other problems. I really can’t believe I actually liked dealing with this crap when I used linux… but I digress.

So first off, I think “Hey, I can add some magic to the CFLAGS parameter, and it’ll just compile!” … hahaha … I wish. Here’s what happens:

cwright@phendrana:~/Desktop/Recent Source Stuff/libSomeLib-X.Y.Z>CFLAGS=\"-Os -fomit-frame-pointer -arch i386 -arch ppc -arch x86_64 -arch ppc64\" ./configure --enable-shared

Configure does its thing, and then says it’s done and you’re ready to build. It’s lying, of course:

cwright@phendrana:~/Desktop/Recent Source Stuff/libSomeLib-X.Y.Z>make
make  all-recursive
Making all in libSomeLib
/bin/sh ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -I..    -Os -fomit-frame-pointer -arch i386 -arch ppc -arch x86_64 -arch ppc64 -MT io.lo -MD -MP -MF .deps/io.Tpo -c -o io.lo io.c
 gcc -DHAVE_CONFIG_H -I. -I.. -I.. -Os -fomit-frame-pointer -arch i386 -arch ppc -arch x86_64 -arch ppc64 -MT io.lo -MD -MP -MF .deps/io.Tpo -c io.c  -fno-common -DPIC -o .libs/io.o
gcc-4.0: -E, -S, -save-temps and -M options are not allowed with multiple -arch flags
make[2]: *** [io.lo] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

So, -M options aren’t allowed to have multiple arch flags, says gcc. I do some spelunking to find out exactly what these flags do, and find that they’re for creating Makefile dependencies. But wait a minute, isn’t that Configure’s job? Man these build tools are awesome. … (technically speaking, it is Make’s job to make this stuff. There just isn’t a hook to add architecture support anywhere else without completely re-engineering the build system.)

Next up, I read some Apple documentation to see how they go about doing it (for stuff like OpenSSL, etc.). They recommend making a new XCode project, and then making a zillion build targets, taped together with some shell scripts. To be honest, it doesn’t seem very Apple-like. If I’m going to be dealing with shell scripts, I’ll just do it myself in Terminal.

The second attempt consists of duplicating the source tree into 4 directories, one for each architecture. Then, my plan goes, I’ll use lipo to glue them all together into a fat binary, and I’ll be on my way.

I run configure in each directory tree, careful to include only one -arch parameter in each one. I issue make in the i386, ppc, and x86_64 trees without issue, and start to think a bit smugly to myself that I’ve defeated this silly monster. But then, another Configure Dragon charges.

cwright@phendrana:~/Desktop/Recent Source Stuff/libSomeLib-X.Y.Z-ppc64>CFLAGS=\"-Os -fomit-frame-pointer -arch ppc64\" ./configure --enable-shared
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
/bin/sh: /Users/cwright/Desktop/Recent: No such file or directory
configure: WARNING: `missing' script is too old or missing
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details.

Of course, whenever configure tells you to check “config.log” for more details, it’s about as informative as reading a Blue Screen. Also, note the 4th or 5th line, which shows a lack of quoted paths. That looks safe…

The problem, according to configure, is that it “cannot run C compiled programs,” which is actually accurate for this case: Rosette doesn’t translate ppc64 binaries to x86. However, gcc can compile such programs, so we know we can do it. Configure suggests using the “--host” command.

Running configure with --help reveals how --host is supposed to be used. It’ll look like this, I suppose: “--host=ppc64″.

It issues this warning, but continues to do its thing:

configure: WARNING: In the future, Autoconf will not detect cross-tools
whose name does not start with the host triplet.  If you think this
configuration is useful to you, please write to autoconf@gnu.org.

I hope that’s ok.. ?

I run make, and it builds, and finishes. A bit earlier than the others. In the .libs directory, there’s no .dylib, which is what –enable-shared is supposed to create. So, I look at the output from previous builds, and copy the line it inexplicably skips. It’s a pretty long, but simple gcc line that takes all the .o’s and makes them into a .dylib. No idea why it skipped over that one…

And at last, we’re able to build our fat binary using lipo. Hurray for portable cross-platform build tools!

Good morning gentle readers. I opened up my craptop this morning to witness the struggles of what seemed to be a laptop battling cancer. No, this was not another case of opteron cancer. Instead I found that Windows, unable to cope with my Cisco Aironet 350 wireless card, was creating network connections one after another in the system tray.

Recently I have been taking out my wireless network card while at work, which used to always reside in my laptop’s PCMCIA slot. At the end of the day I’d plug it back in usually either before or after I had sent my laptop into power save mode.

So this morning like all mornings I take out my laptop, set it up, open the lid and log in. Within seconds I notice my system tray bombarded by a new wireless network connection appearing once every second or so, enlarging the system tray beyond its normal size (note the arrow to the far right of the screenshot). My esteemed colleague Chris Wright and I were both disappointed that the info bubble did not in fact pop up for each of those connection icons. Luckily this cancer seemed benign, and within a few seconds Windows realized there actually was only one wireless networking device.

Though it seems this all boils down to the lesson: never mess with an IBM trying to get some sleep. Whether it’s not waiting long enough for a print process to get out of the queue causing endless beeping or windows self-replicating wireless network connections, you’ll face the seemingly impossible behavior of a laptop pushing the boundaries of sanity.

I thought to myself this morning — I want an Eggo Waffle. You see, I have never had one, but have always been interested. There were some in the freezer that Robert had bought, removed from their packaging because of his space-saving ways.

Naturally I found the need to prepare an Eggo Waffle to the correct specifications as I wanted to make sure that I was having the correct Eggo experience (microwave, toast, pan fry, what?). To my surprise the feast was ruined (credit for the title), and breakfast was not magically delicious.

Dear Kellogg’s,

I thought to myself this morning — I want an Eggo Waffle. You see, I have never had one, but have always been interested. There were some in the freezer that Robert had bought, removed from their packaging because of his space-saving ways.

However I could NOT find any directions on how to actually make an Eggo Waffle. So I thought to myself — Hey, Eggo’s web site might have this pertinent and useful information! To my surprise I found that the Eggo web site was designed like shit.

No, I cannot enjoy an Eggo waffle for the first time because I simply do not know the proper instructions. This information is surprisingly absent from a web site. Instead I have useless information thrust into my vision in a flash animated piece of shit.

My breakfast appetite is ruined. I am no longer interested in having an Eggo Waffle because your shitty web site design team put me off.

This might be humorous to you or your team, but this is no laughing matter to me. I think I will have some General Mills Cheerios instead.

Love mradcliffe

You might be amused to find my difficulty with preparing an Eggo Waffle. In my search for deliciousness I did come across one link to enjoy Eggo Waffles, which I admit did look interesting. However I wanted to pursue slightly more official instructions. And I did receive an update from Kellogg’s though it came into my inbox a bit too late in the day for breakfast.

Update: Kellogg’s responded! I bet I made someone’s day.

Thank you for contacting us regarding Eggo® website.

The feedback you provided on our website is valuable to us. We will pass along your comments to our advertising and marketing departments. As you know, things change quickly on the web, so we hope you will visit us again to see what’s new!

We hope you give our product another opportunity. For future information you can contact us toll free at 1-800-962-1413 so one of our service representatives could answer your questions.

I am quite surprised that Eggo has a toll free number. So whomever comes across this post — if you need to learn “How-To Make an Eggo Waffle” because you’ve never had one before you can call 1-800-962-1413.

Operators are standing by to assist you with your breakfast endeavor.

The final lesson to be learned is that I am grumpy in the mornings. Never get between a Radcliffe and his breakfast.

The disappointing part is that this bug was first noticed almost two years ago. While the extent isn’t nearly as bad, it’s still not clean.

This bug came up in our Speech Synthesizer patch for Quartz Composer, featured at kineme. After much testing, profiling, and refactoring, a splendidly small demonstration program emerged.

#import <Cocoa/Cocoa.h>
	
int main()
{
        while(1)
        {
                NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
                NSSpeechSynthesizer *speech = [[NSSpeechSynthesizer alloc] init];
                [speech release];
                [pool release];
        }
	
        return 0;
}

This program endlessly allocates an NSAutoreleasePool, used for memory management, and then an NSSpeechSynthesizer object. After creating them, it destroys them. Destroying the synthesizer object should free any associated memory with the object, and releasing the pool should free up any odds and ends left over, just in case.

Reality paints a much more disappointing picture:

cwright@phendrana:~/projects/SpeechTest>./rapid
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Segmentation fault

It starts off looking ok. It spins for about 15 seconds or so, but then trouble strikes. As you an see, the class starts to have difficulty mapping a voice file (this is only mentioned once in all of Google’s wisdom, with no useful insight as to what it means), and eventually the inevitable crash occurs.

smokris tried this on his MacBook Pro.. Same thing there:

smokris@etu 6 ~/Desktop -> time ./a.out
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Speech Synthesis can't map voice file (12)
Segmentation fault
	
real    0m4.479s
user    0m0.286s
sys     0m0.153s

• Target medium: 1GB XD card from a digital camera.
• Data to recover: JPEGs. Lots of them.

Whipping out some jpeg-format-and-filesystem-jutsu, here’s the solution (for less than that $20 shareware recovery utility):

Step one: Create a disk image of the card.
sudo dd if=/dev/disk2 of=~/Desktop/roberts.xd.card.image bs=1048576

Step two: Create a utility to extract JPEGs.

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
	
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/mman.h>
	
#include <unistd .h>
	
int main(int argc, char**argv)
{
        int file, filesize, i, j;
        int outfile=0, outfilenum = 0;
        int blocksWritten = 0;
        char outname[42];
        unsigned char *data;
        struct stat fileStat;
	
        if(stat(argv[1],&fileStat))
        {
                printf("Couldn't stat file!\n");
                return -1;
        }
	
        file = open(argv[1],O_RDONLY);
	
        if(!file)
        {
                printf("Couldn't open file!\n");
                return -1;
        }
	
        filesize = fileStat.st_size;
        if(fileStat.st_size % getpagesize())
        {
                // if we're unaligned, we add the difference to pad up.
                filesize = fileStat.st_size + (getpagesize() - (fileStat.st_size % getpagesize()));
        }
	
        printf("mmapping %i bytes...\n",filesize);
	
        data = mmap(0,filesize,PROT_READ,MAP_PRIVATE,file,0);
	
        if(data == -1)
        {
                printf("Couldn't mmap file...\n");
                perror("mmap");
                return -1;
        }
	
        printf("Scanning %i blocks for jpegs...\n",filesize/1024);
	
        j = 0; // found count
        outfile = -1;
        for(i=0;i < (fileStat.st_size/1024);++i)
        {
                if(data[i*1024 + 0] == 0xff && // SOI (Start of Image) Tag
                   data[i*1024 + 1] == 0xd8 && //
                   data[i*1024 + 2] == 0xff && // APP1 (Application 1; exif) Tag
                   data[i*1024 + 3] == 0xe1) //
                {
                        printf("Probable jpeg at block %i (#%i)\n",i,j++);
                        if(outfile != -1)
                        {
                                close(outfile);
                                outfile = -1;
                        }
                        snprintf(outname, 42,"recovered.%i.jpg",outfilenum++);
                        outfile = open(outname,O_RDWR|O_CREAT,0666);
                        write(outfile,data+i*1024,1024);
                        blocksWritten = 1;
                }
                else // non-header block.  write it if we're in a file
                {
                        // write another block.  stop at 4096 blocks (4MB jpegs)
                        if(outfile != -1 && outfile > 2)
                        {
                                write(outfile,data+i*1024,1024);
                                ++blocksWritten;
                                if(blocksWritten >= 4096)
                                {
                                        close(outfile);
                                        outfile = -1;
                                        blocksWritten = 0;
                                }
                        }
                }
	
                // for progress view
                if( (i%10000) == 0 && i)
                {
                        printf("block %i...\n",i);
                }
        }
	
        return 0;
}

Step three: compile it. Since we’re dealing with huge images and I was lazy and used mmap, we need to make it a 64-bit binary. To do this, we simply run gcc -m64 main.c -o main and we’re in business.

Step four: run and enjoy

cwright@phendrana:~/projects/getjpeg>./main ~/Desktop/roberts.xd.card.image
mmapping 1048481792 bytes...
Scanning 1023908 blocks for jpegs...
Probable jpeg at block 1632 (#0)
Probable jpeg at block 2864 (#1)
Probable jpeg at block 4208 (#2)
Probable jpeg at block 5424 (#3)
Probable jpeg at block 6640 (#4)
Probable jpeg at block 7840 (#5)
Probable jpeg at block 8976 (#6)
block 10000...
Probable jpeg at block 10224 (#7)
Probable jpeg at block 11488 (#8)
Probable jpeg at block 12768 (#9)
Probable jpeg at block 14192 (#10)
Probable jpeg at block 15520 (#11)
Probable jpeg at block 16880 (#12)
Probable jpeg at block 17760 (#13)
Probable jpeg at block 19024 (#14)
block 20000...
Probable jpeg at block 20208 (#15)
Probable jpeg at block 21408 (#16)
Probable jpeg at block 22608 (#17)
Probable jpeg at block 23840 (#18)
Probable jpeg at block 24976 (#19)
Probable jpeg at block 26128 (#20)
Probable jpeg at block 27296 (#21)
Probable jpeg at block 28432 (#22)
Probable jpeg at block 29584 (#23)
block 30000...
Probable jpeg at block 30720 (#24)
Probable jpeg at block 31968 (#25)
Probable jpeg at block 33056 (#26)
Probable jpeg at block 34272 (#27)
Probable jpeg at block 35456 (#28)
Probable jpeg at block 36720 (#29)
Probable jpeg at block 38000 (#30)
Probable jpeg at block 39120 (#31)
block 40000...
Probable jpeg at block 40336 (#32)
Probable jpeg at block 41488 (#33)
Probable jpeg at block 42800 (#34)
Probable jpeg at block 44064 (#35)
Probable jpeg at block 45312 (#36)
Probable jpeg at block 46528 (#37)
Probable jpeg at block 47648 (#38)
Probable jpeg at block 48864 (#39)
Probable jpeg at block 49984 (#40)
block 50000...
Probable jpeg at block 51088 (#41)
Probable jpeg at block 52192 (#42)
Probable jpeg at block 53280 (#43)
Probable jpeg at block 54416 (#44)
Probable jpeg at block 55536 (#45)
Probable jpeg at block 56480 (#46)
Probable jpeg at block 57488 (#47)
Probable jpeg at block 58448 (#48)
Probable jpeg at block 59360 (#49)
block 60000...
Probable jpeg at block 60352 (#50)
Probable jpeg at block 61040 (#51)
Probable jpeg at block 61808 (#52)
Probable jpeg at block 62736 (#53)
Probable jpeg at block 63472 (#54)
Probable jpeg at block 64256 (#55)
Probable jpeg at block 65072 (#56)
Probable jpeg at block 66016 (#57)
Probable jpeg at block 66832 (#58)
Probable jpeg at block 67680 (#59)
Probable jpeg at block 68368 (#60)
Probable jpeg at block 69168 (#61)
block 70000...
Probable jpeg at block 70016 (#62)
Probable jpeg at block 70736 (#63)
Probable jpeg at block 71504 (#64)
Probable jpeg at block 72688 (#65)
Probable jpeg at block 73776 (#66)
Probable jpeg at block 74960 (#67)
Probable jpeg at block 76016 (#68)
Probable jpeg at block 77040 (#69)
Probable jpeg at block 78176 (#70)
Probable jpeg at block 79248 (#71)
block 80000...
Probable jpeg at block 80400 (#72)
Probable jpeg at block 81504 (#73)
Probable jpeg at block 82640 (#74)
Probable jpeg at block 83840 (#75)
Probable jpeg at block 84752 (#76)
Probable jpeg at block 85664 (#77)
Probable jpeg at block 86560 (#78)
Probable jpeg at block 87552 (#79)
Probable jpeg at block 88528 (#80)
Probable jpeg at block 89456 (#81)
block 90000...
Probable jpeg at block 90416 (#82)
Probable jpeg at block 91392 (#83)
Probable jpeg at block 92384 (#84)
Probable jpeg at block 93312 (#85)
Probable jpeg at block 94400 (#86)
Probable jpeg at block 95536 (#87)
Probable jpeg at block 96688 (#88)
Probable jpeg at block 97824 (#89)
Probable jpeg at block 98832 (#90)
Probable jpeg at block 99824 (#91)
block 100000...
Probable jpeg at block 100656 (#92)
Probable jpeg at block 101520 (#93)
Probable jpeg at block 102624 (#94)
Probable jpeg at block 103776 (#95)
Probable jpeg at block 104880 (#96)
Probable jpeg at block 105952 (#97)
Probable jpeg at block 107056 (#98)
Probable jpeg at block 108208 (#99)
Probable jpeg at block 109472 (#100)
block 110000...
Probable jpeg at block 110672 (#101)
Probable jpeg at block 111824 (#102)
Probable jpeg at block 112976 (#103)
Probable jpeg at block 114096 (#104)
Probable jpeg at block 115296 (#105)
Probable jpeg at block 116384 (#106)
Probable jpeg at block 117584 (#107)
Probable jpeg at block 118704 (#108)
Probable jpeg at block 119888 (#109)
block 120000...
Probable jpeg at block 120864 (#110)
Probable jpeg at block 122016 (#111)
Probable jpeg at block 123216 (#112)
Probable jpeg at block 124352 (#113)
Probable jpeg at block 125408 (#114)
Probable jpeg at block 126480 (#115)
Probable jpeg at block 127536 (#116)
Probable jpeg at block 128752 (#117)
Probable jpeg at block 129952 (#118)
block 130000...
Probable jpeg at block 131104 (#119)
Probable jpeg at block 132192 (#120)
Probable jpeg at block 133264 (#121)
Probable jpeg at block 134416 (#122)
Probable jpeg at block 135568 (#123)
Probable jpeg at block 136784 (#124)
Probable jpeg at block 138016 (#125)
Probable jpeg at block 139264 (#126)
block 140000...
Probable jpeg at block 140496 (#127)
Probable jpeg at block 141168 (#128)
Probable jpeg at block 141952 (#129)
Probable jpeg at block 142912 (#130)
Probable jpeg at block 143856 (#131)
Probable jpeg at block 144944 (#132)
Probable jpeg at block 146000 (#133)
Probable jpeg at block 147056 (#134)
Probable jpeg at block 148272 (#135)
Probable jpeg at block 149504 (#136)
block 150000...
Probable jpeg at block 150672 (#137)
Probable jpeg at block 151888 (#138)
Probable jpeg at block 153120 (#139)
Probable jpeg at block 154352 (#140)
Probable jpeg at block 155648 (#141)
Probable jpeg at block 156944 (#142)
Probable jpeg at block 158192 (#143)
Probable jpeg at block 159472 (#144)
block 160000...
Probable jpeg at block 160768 (#145)
Probable jpeg at block 162016 (#146)
Probable jpeg at block 163264 (#147)
Probable jpeg at block 164480 (#148)
Probable jpeg at block 165760 (#149)
Probable jpeg at block 167008 (#150)
Probable jpeg at block 168272 (#151)
Probable jpeg at block 169536 (#152)
block 170000...
Probable jpeg at block 170800 (#153)
Probable jpeg at block 172224 (#154)
Probable jpeg at block 173456 (#155)
Probable jpeg at block 174704 (#156)
Probable jpeg at block 175936 (#157)
Probable jpeg at block 177184 (#158)
Probable jpeg at block 178432 (#159)
Probable jpeg at block 179712 (#160)
block 180000...
Probable jpeg at block 180992 (#161)
Probable jpeg at block 182240 (#162)
Probable jpeg at block 183568 (#163)
Probable jpeg at block 184976 (#164)
Probable jpeg at block 186288 (#165)
Probable jpeg at block 187744 (#166)
Probable jpeg at block 189104 (#167)
block 190000...
Probable jpeg at block 190512 (#168)
Probable jpeg at block 191792 (#169)
Probable jpeg at block 193184 (#170)
Probable jpeg at block 194496 (#171)
Probable jpeg at block 195776 (#172)
Probable jpeg at block 197136 (#173)
Probable jpeg at block 198512 (#174)
Probable jpeg at block 199808 (#175)
block 200000...
Probable jpeg at block 201088 (#176)
Probable jpeg at block 202352 (#177)
Probable jpeg at block 203664 (#178)
Probable jpeg at block 204992 (#179)
Probable jpeg at block 206304 (#180)
Probable jpeg at block 207504 (#181)
Probable jpeg at block 208816 (#182)
block 210000...
Probable jpeg at block 210144 (#183)
Probable jpeg at block 211472 (#184)
Probable jpeg at block 212832 (#185)
Probable jpeg at block 214240 (#186)
Probable jpeg at block 215632 (#187)
Probable jpeg at block 217056 (#188)
Probable jpeg at block 218416 (#189)
Probable jpeg at block 219808 (#190)
block 220000...
Probable jpeg at block 221072 (#191)
Probable jpeg at block 222400 (#192)
Probable jpeg at block 223808 (#193)
Probable jpeg at block 225232 (#194)
Probable jpeg at block 226576 (#195)
Probable jpeg at block 227888 (#196)
Probable jpeg at block 229296 (#197)
block 230000...
Probable jpeg at block 230560 (#198)
Probable jpeg at block 231904 (#199)
Probable jpeg at block 233216 (#200)
Probable jpeg at block 234544 (#201)
Probable jpeg at block 235920 (#202)
Probable jpeg at block 237216 (#203)
Probable jpeg at block 238528 (#204)
Probable jpeg at block 239888 (#205)
block 240000...
Probable jpeg at block 241232 (#206)
Probable jpeg at block 242512 (#207)
Probable jpeg at block 243904 (#208)
Probable jpeg at block 245280 (#209)
Probable jpeg at block 246592 (#210)
Probable jpeg at block 247952 (#211)
Probable jpeg at block 249328 (#212)
block 250000...
Probable jpeg at block 250672 (#213)
Probable jpeg at block 252000 (#214)
Probable jpeg at block 253264 (#215)
Probable jpeg at block 254608 (#216)
Probable jpeg at block 255904 (#217)
Probable jpeg at block 257200 (#218)
Probable jpeg at block 258480 (#219)
Probable jpeg at block 260000 (#220)
block 260000...
Probable jpeg at block 261504 (#221)
Probable jpeg at block 262896 (#222)
Probable jpeg at block 264320 (#223)
Probable jpeg at block 265744 (#224)
Probable jpeg at block 267120 (#225)
Probable jpeg at block 268592 (#226)
block 270000...
Probable jpeg at block 270064 (#227)
Probable jpeg at block 271488 (#228)
Probable jpeg at block 272912 (#229)
Probable jpeg at block 274400 (#230)
Probable jpeg at block 275936 (#231)
Probable jpeg at block 277360 (#232)
Probable jpeg at block 278800 (#233)
block 280000...
Probable jpeg at block 280176 (#234)
Probable jpeg at block 281712 (#235)
Probable jpeg at block 283200 (#236)
Probable jpeg at block 284688 (#237)
Probable jpeg at block 286016 (#238)
Probable jpeg at block 287552 (#239)
Probable jpeg at block 288992 (#240)
block 290000...
Probable jpeg at block 290544 (#241)
Probable jpeg at block 292000 (#242)
Probable jpeg at block 293440 (#243)
Probable jpeg at block 294800 (#244)
Probable jpeg at block 296208 (#245)
Probable jpeg at block 297472 (#246)
Probable jpeg at block 298720 (#247)
block 300000...
Probable jpeg at block 300080 (#248)
Probable jpeg at block 301504 (#249)
Probable jpeg at block 302784 (#250)
Probable jpeg at block 304064 (#251)
Probable jpeg at block 305008 (#252)
Probable jpeg at block 306016 (#253)
Probable jpeg at block 306976 (#254)
Probable jpeg at block 307984 (#255)
Probable jpeg at block 309216 (#256)
block 310000...
Probable jpeg at block 310592 (#257)
Probable jpeg at block 311888 (#258)
Probable jpeg at block 313152 (#259)
Probable jpeg at block 314352 (#260)
Probable jpeg at block 315664 (#261)
Probable jpeg at block 317024 (#262)
Probable jpeg at block 318272 (#263)
Probable jpeg at block 319568 (#264)
block 320000...
Probable jpeg at block 320928 (#265)
Probable jpeg at block 322176 (#266)
Probable jpeg at block 323568 (#267)
Probable jpeg at block 324800 (#268)
Probable jpeg at block 325984 (#269)
Probable jpeg at block 327232 (#270)
Probable jpeg at block 328480 (#271)
Probable jpeg at block 329776 (#272)
block 330000...
Probable jpeg at block 331088 (#273)
Probable jpeg at block 332320 (#274)
Probable jpeg at block 333616 (#275)
Probable jpeg at block 334912 (#276)
Probable jpeg at block 336256 (#277)
Probable jpeg at block 337648 (#278)
Probable jpeg at block 338880 (#279)
block 340000...
Probable jpeg at block 340208 (#280)
Probable jpeg at block 341264 (#281)
Probable jpeg at block 342272 (#282)
Probable jpeg at block 343552 (#283)
Probable jpeg at block 344832 (#284)
Probable jpeg at block 346080 (#285)
Probable jpeg at block 346992 (#286)
Probable jpeg at block 348368 (#287)
Probable jpeg at block 349760 (#288)
block 350000...
Probable jpeg at block 351168 (#289)
Probable jpeg at block 352448 (#290)
Probable jpeg at block 353760 (#291)
Probable jpeg at block 355136 (#292)
Probable jpeg at block 356368 (#293)
Probable jpeg at block 357712 (#294)
Probable jpeg at block 359040 (#295)
block 360000...
Probable jpeg at block 360464 (#296)
Probable jpeg at block 361808 (#297)
Probable jpeg at block 363200 (#298)
Probable jpeg at block 364144 (#299)
Probable jpeg at block 365488 (#300)
Probable jpeg at block 366752 (#301)
Probable jpeg at block 367808 (#302)
Probable jpeg at block 368848 (#303)
Probable jpeg at block 369856 (#304)
block 370000...
Probable jpeg at block 371072 (#305)
Probable jpeg at block 372272 (#306)
Probable jpeg at block 373568 (#307)
Probable jpeg at block 374864 (#308)
Probable jpeg at block 376144 (#309)
Probable jpeg at block 377504 (#310)
Probable jpeg at block 378512 (#311)
Probable jpeg at block 379824 (#312)
block 380000...
Probable jpeg at block 381120 (#313)
Probable jpeg at block 382320 (#314)
block 390000...
block 400000...
block 410000...
block 420000...
block 430000...
block 440000...
block 450000...
block 460000...
block 470000...
block 480000...
block 490000...
block 500000...
block 510000...
block 520000...
block 530000...
block 540000...
block 550000...
block 560000...
block 570000...
block 580000...
block 590000...
block 600000...
block 610000...
block 620000...
block 630000...
block 640000...
block 650000...
block 660000...
block 670000...
block 680000...
block 690000...
block 700000...
block 710000...
block 720000...
block 730000...
block 740000...
block 750000...
block 760000...
block 770000...
block 780000...
block 790000...
block 800000...
block 810000...
block 820000...
block 830000...
block 840000...
block 850000...
block 860000...
block 870000...
block 880000...
block 890000...
block 900000...
block 910000...
block 920000...
block 930000...
block 940000...
block 950000...
block 960000...
block 970000...
block 980000...
block 990000...
block 1000000...
block 1010000...
block 1020000...

Yay, images from yesterday: Recovered.

Take it down, flip it, and reverse it.

So, how do we get this elusive MAC address anyway? For many devices, it’s printed right on the device. This is by far the easiest and fastest way to get it. But some devices hide this information, and have no interface for figuring it out. For cases like this, we can whip out a handy utility called arp and some network-jutsu, and purloin these delicious octets with reckless abandon.

arp is a program that manipulates the ARP cache on a system. The ARP cache translates an IP address into a hardware, or MAC, address. So, by pulling data out of this cache, we can find out which MAC address our system uses when we speak to a specific IP address.

Using arp is very simple for what we’re looking to do. arp [ip address] will tell us what a given IP or hostname’s MAC address is. This usage looks like this on linux:

cwright@leikata:~>arp 10.1.1.7
Address HWtype HWaddress Flags Mask Iface
vaeltaja.softpixel.com ether 00:11:24:E9:92:AC C eth0

or this on OS X:

cwright@phendrana:~>arp 192.168.255.5
server.example.com (192.168.255.5) at 0:30:6e:29:b2:55 on en1 [ethernet]


So far, so good. We can pull data out of the cache, and the MAC address is right there. Is there a catch? Of course there is.

Not having an address in the cache will, of course, make a cache query useless. This is what such a condition looks like on linux:

cwright@leikata:~>arp google.com
google.com (64.233.187.99) -- no entry

or this on OS X:

cwright@phendrana:~>arp 35.42.42.42
35.42.42.42 (35.42.42.42) -- no entry

No entry!? How can this be? In this example, we’re trying to get the MAC address from an address that isn’t on our local network. Since we only use MAC address for communication on the same LAN, we don’t cache, and thus cannot acquire, MAC addresses from non-local addresses.

Sometimes, however, we’ll receive that error for a machine that is on the network. This happens because we haven’t communicated with the destination machine. To solve this, we simply need to initiate some communication so that we’ll cache the MAC address. I like to use the ping utility, like this:

cwright@phendrana:~>arp 192.168.255.11
192.168.255.11 (192.168.255.11) -- no entry

cwright@phendrana:~>ping 192.168.255.11
PING 192.168.255.11 (192.168.255.11): 56 data bytes
64 bytes from 192.168.255.11: icmp_seq=0 ttl=30 time=2.612 ms
^C
--- 192.168.255.11 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 2.612/2.612/2.612/0.000 ms

cwright@phendrana:~>arp 192.168.255.11
? (192.168.255.11) at 0:d0:d9:2:f7:6 on en1 [ethernet]

In the first arp, we have no data in the cache. So, we ping the destination, causing some communication, and thus some cached data. After that, we can query again, and we have our MAC address ready and waiting. That wasn’t so hard, now was it?