Entries : Category [ Programming and Software ]
[Miscellaneous]  [Computers and Technology]  [Travel]  [Education]  [Hacks]  [Robotics]  [Science]  [Programming and Software]  [iPhone]  [Digital TV and Video]  [Intellectual Property & Copyright]  [Personal] 

30 January

Getting c++ to use a custom python framework

This post addresses how to get the OpenCV vision package with Python bindings to compile on OS X, such that you can use a version of Python other than the one that is supplied with the original operating system installation. It's very hacky and not for casual reading, but my help some person questing for help.

Dudek faces detected
Detected faces, using opencv, Viola-Jones algorithm (boosting)

Detecting a sleeply dudek with opencv
Face and eyes detected

The problem is that OpenCV insists on being linked with the standard default installation of Python on OS X. On the other hand, many (most?) techie people using Python will have installed an alternative newer release (for instance via "fink" or "macports").

When you compile OpenCV and try to use it with your custom-installed Python (e.g. version 2.5.2 instead of the stock 2.5.1), you get an error as follows:

% python
>>> import opencv
Fatal Python error: Interpreter not initialized (version mismatch?)

Using the standard Python is a workaround, but for me it wasn't acceptable. Unfortunately, since I had no prior experience with cmake or opencv, this was problematic. Here's how to fix it. Despite many only-semi-correct discussions of this on the net, it's fairly easy to fix, once you realize the source of the problem.

On OS X (or, in fact, the core operating system kernel, Darwin), many packages are installed using Frameworks. They group together libraries and headers. The C compiler on Darwin has been extended to take a -framework PATH flag, and by default this causes the linker to look in the system directories, but not user-installed directories. You need to provide an extra -F DIRECTORY flag to specific the location of your user-installed frameworks (e.g. "-F /opt/local/Library/Frameworks/").

I never quite figureed out the "right" place to insert the flag before starting a build, but after a failed build you can just insert it into
The right place is probably interfaces/swig/python/CMakeLists.txt
using something like

set(CMAKE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -F/opt/local/Library/Frameworks")

but once I got it working I didn't want to waste more time on a really clean fix (instead, I wasted time writing it up here).

When you run make, in order to assure it's working you can set the system frameworks to be temporarily inaccessible to verify the right places are being searched. Do this with the chmod command. Afterwards, you can put things back the way they were (mode 755).
I suggest you test your patch as follows:

sudo chmod 000 /System/Library/Frameworks/Python.framework/
cmake .....
sudo make install
sudo chmod 755 /System/Library/Frameworks/Python.framework/

As an aside, I was also confronted by the error:

opencv/src/highgui/cvcap_ffmpeg.cpp:70:28: error: ffmpeg/swscale.h: No such file or directory

this can be fixed by the hack:

cp -r /opt/local/include/libswscale/* /opt/local/include/ffmpeg

By Gregory Dudek at | Read (1) or Leave a comment |    
02 February

We are moving the mobile robotics lab to Mercurial for source code control and management.

Mercurial is a source code control system (i.e. version control system) similar to CVS, SVN (subversion) or SCCS, if you know any of those. That means it helps you keep track of a set of files, recover an old version of a file, or combine the work of multiple authors. Mercurial in used via the command-line command "hg" (Hg being the symbol for the element Mercury). Mercurial is distributed inder the GPL v2 (i.e. it is true open source).

Example of Mercurial web interface

I have been looking for a CVS replacement for some time, and narrowed the choices down to a couple of options some time ago.

The full set of tradeoffs is a long story (see references below), but the key features that Mercurial has are:
1- platform independence (works on Linux, Windows, QNX, and Mac OS X).
2- web based interface (works via command line, or HTTP, or HTTPS, or SSH). GUI clients also.
3- easy to learn and use (with command-line interface similar to svn and CVS, but also differences in semantics).
4- distributed version control; no central server required.

Mercurial also allows an repository to be imported from CVS or SVN, so migration should be not-too-hard.

I believe the 4 issues above, and especially ...

1 & 3 are VERY important for us. We are a rotating pool of users, and they need to be able to learn is easily. It needs to work on many types of system including, at least, all those noted above. Git is an alternative, but it is harder to learn and does not work as well on Windows, and it needs to be repacked. Bzr is an alternative, but it is not s mature and also seems to be losing mindshare. svn is an alternative, but it's old, not distributed, and space-hungry.

One risk factor is that mercurial is not as heavily used as git and might (theoretically) have insufficient support, but that does not seem problematic as it does have a big user base and is used for several very major important projects

Mercurial is seems the best choce given the issues above (in second place: git, third place: svn, fourth place: CVS).

Basics (must read):

For CVS lovers (CVS refugees): http://www.selenic.com/mercurial/wiki/index.cgi/CvsInfo

Mecurial main site, documentation, etc.


svn, mercurial, git, bzr comparison article: http://joshcarter.com/productivity/svn_hg_git_for_home_directory

Mercurial FAQ (nice, from Mozilla, which now uses hg instead of CVS):


Example hg web interface to a project of some kind (comes free by typing "hg serve and
allows nice viewing options) [I don't know what this project it, I just picked a random hg repository ]


By Gregory Dudek at | Leave a comment |    
12 March

A fix to EXIF.py to make it find the data more consistently

The Python EXIF.py module is for extracting tags from an image. If the image has been edited with Apple's iPhoto 08 (or probably iPhoto 09 or other software) though, EXIF.py may be unable to find the EXIF tags and you might get a message like "No EXIF information found." Some people have suggested that iPhoto is stripping the EXIF data. In fact, the issue is that EXIF.py does not like it if there is other information before the EXIF segment in the file (specifically something called an APP2 segment). In short, EXIF.py only handles a subset of possible JPEG file formats. This was a particular problem for me since I use EXIF.py on my Zope web site (this one) with the Photo product to display information.

I found the problem and fixed it and also updated the Photo product as well. The modified version of EXIF.py that is able to parse these files can be found at the following link [EXIF.py]. I've also posted the fix to the maintainers, but don't know how soon they might accept it.

The basis of the problem relates to the flexibility of the JPEG/EXIF file format. A JPEG file is composed of several segments including the image, a thumbnail, and parts called APP1 and APP2. The standard EXIF.py module expected APP1 to be the first segment in the file, and didn't work if APP2 came before APP1, but this is how iPhoto creates JPEG files. The new version I have posted is smarter about interpreting the segments (although the actual code is rather ugly).

Download link:
link [EXIF.py].

Usage: python EXIF.py pic.jpg

Or, it can be used inside a python program:

import EXIF
print EXIF.process_file(f)

I have also included a substantially updated version of the wonderful Zope Photo product that was originally created by Rob Bickers (rbickers) and then updated and improved by Søren Roug [ zope.org link].

This version merges changes from the 1.3.2 and 1.2.4 branches, includes this updated EXIF.py, and has some code changes (such as a method called "exifdata" that takes an EXIF field name for display, no field at all, or a special meta-fields such as "Date"). It also includes patches that allow Zope images to be re-created as Photo objects.

You can see the product used here at dudek.org/Recruiting2009 to produce the EXIF summary at the top of the picture (click that for more details). I am calling this version 31 and it's a compressed tar file [ photo31.tgz ]. It is also used to produce the images shown here.

Picture with EXIF dataPicture with EXIF data

{'Image ExifOffset': (0x8769) Long=360 @ 138, 'EXIF CustomRendered': (0xA401) Short=Normal @ 694, 'EXIF MeteringMode': (0x9207) Short=Pattern @ 502, 'EXIF DateTimeOriginal': (0x9003) ASCII=2009:03:11 20:20:29 @ 754, 'MakerNote FirmwareVersion': (0x0007) ASCII=Firmware Version 1.0.7 @ 1490, 'EXIF ExifImageWidth': (0xA002) Short=5616 @ 622, 'EXIF SubSecTimeOriginal': (0x9291) ASCII=74 @ 574, 'Image XResolution': (0x011A) Ratio=72 @ 196, 'Image YResolution': (0x011B) Ratio=72 @ 204, 'EXIF ShutterSpeedValue': (0x9201) Signed Ratio=21/8 @ 794, 'EXIF ISOSpeedRatings': (0x8827) Short=3200 @ 406, 'EXIF ExposureTime': (0x829A) Ratio=1/6 @ 738, 'MakerNote ImageType': (0x0006) ASCII=Canon EOS 5D Mark II @ 1458, 'MakerNote Tag 0x00AA': (0x00AA) Short=[12L, 1124L, 1024L, 1024L, 715L, 1L] @ 5300, 'Image YCbCrPositioning': (0x0213) Short=Co-sited @ 114, 'Thumbnail XResolution': (0x011A) Ratio=72 @ 10532, 'EXIF ExposureProgram': (0x8822) Short=Program Normal @ 394, 'Image GPSInfo': (0x8825) Long=8660 @ 150, 'Image Copyright': (0x8298) ASCII=Copyright: Gregory Dudek @ 296,

Picture with EXIF data

Please let me know if you have any difficulties with or (or not).

By Gregory Dudek at | Read (8) or Leave a comment |    
Rate item 154: Rating: 7.7/10 (3 votes cast)
05 May

I recently installed and used a miniature Sanav MK-7 GPS logger from SparkFun. This device is really small: a little smaller than one AA battery. I wanted to use it with Apple's OS X Leopard operating system: here's how.

GPS logger
GPS logger

The MK-7 is built around the MTK datalogger chipset and the Transystem EB-230 GPS engine, and includes Flash memory for storing GPS logs. It is connected to a computer with a USB-like cable to either charge the embedded battery, or to upload a data log. Like several other embedded devices there days, the Sanav actually uses a TTL-level serial (RS232) protocol, and needs a GPS-to-serial converter. This is embedded in the cable so that although the whole the cable looks like USB-to-mini-USB, it's actually got a chip inside it that converts USB signals to RS232, put uses a mini-USB connector on the logger to achieve the RS232 and power connection.

The means that to communicate with the device, you need to have the correct USB drivers. The cable I got with my device uses the Prolific Technology Inc. PL-2303 USB-to-serial bridge chip (0x067b), and the drivers for just about any operating system including OSX can be found here:

Once the driver is installed, connecting to the serial port lets you see the live data logging (use a baud rate of 115200):

screen /dev/tty.usbserial 115200

My MK-7 seems to run a firmwre suite called M-core_2.02.

To extract logs, one option is to use the Java-based software package BT747 available from: http://bt747.free.fr. Be sure to load the version that includes "RxTx" libraries. You can also find it at the Sourceforge link http://bt747.wiki.sourceforge.net/mac_installation

By Gregory Dudek at | Read (2) or Leave a comment |    
16 November

I gave a talk on recommender systems at the Drupal Camp Montreal meeting. The nice people at DBN.ca did a great job of
recording the talk and have put it on line. It's not super-technical but it gives an overview of recommender systems in general and a bit about how our particular system, Recommendz.com, operates.

By Gregory Dudek at | Leave a comment |    
06 April

Here are the top 4 reasons why I am frustrated and disappointed about the Google Nexus-1 made by HTC. Two weeks ago I borrowed a Google Nexus-1 phone for a few days and last week I bought one. I was initially unhappy with my purchase (although I have gotten used to it).

All-in-all I would say the Nexus-1 is usable, but it has numerous rough edges and for such an expensive phone it was a substantial disappointment and frustration.

The key downsides are:

1 - Poor power management and power consumption. The first two days I had the phone, it went from fully charged in the morning to out-of-power before I went to bed. If you turn off the cell-phone radio, the GPS and the wireless and don't run anything intensive, then the battery can last 24 hours. It's not a very useful device in that case, however. This isn't just bad, it's utterly ridiculous and unforgivable.

Low battery -- this should be familiar to Nexus-1 users
Low battery -- this should be familiar to Nexus-1 users

2 - Limited or missing support for WPA-Enterprise ...

Wifi networks. There is lots of discussion on the web about this, but for commonplace network configurations for in professional environments and universities, Android is sometimes unable to connect. For me, this was a show-stopper! The very first iPhones had this limitation also, but that was a long time ago and Android has been around for 2 years.

3 - Excessive dependence on the cloud. This phone transmits and synchronizes data via Google. That means sharing almost everything with Google, which is a serious privacy consideration. Moreover, it means lots of data exchange and a need for a good data plan.

4 - A generally unpolished user experience. The worst thing in this regard is frequently laggy or erratic performance, probably caused by multiple applications using processor time at once. There are lots of other ergonomic rough spots thought: excessively small attachment icons in the gmail program (too small to click reliably), a quit button placed to close to an important key in some cases, inconsistent operating modes, etc.

The dependence on the cloud is Google's key business advantage. I can understand and accept it, but I am very squeamish about them having a bead on everything I do, including who I call and where I go. The absence of robust and complete WPA-Enterprise support is inexcusable -- the Android operating system kernel has the support in it, but the user interface is just too rough. The power management and laggy interface are both indications of what appear to be poor management choices and an absence of sound leadership. This are fixable, but entail hard choices that would limit some the the geekish features at the expense of an overall better (i.e. generally usable) experience.

All in all, this phone has served as an impressive illustration of the great ergonomic design and decision-making that went into the iPhone. I am not ready to buy an iPhone now, but the comparison of the iPhone and the Nexus one provides many object lessons in making though design decisions, and the Nexus-1 seems to come out the loser in several ways.

By Gregory Dudek at | Read (1) or Leave a comment |    
Prev  1   [2]   3   Next