Entries : Category [ Hacks ]
Computer and electronics tricks and fooling around
[Miscellaneous]  [Computers and Technology]  [Travel]  [Education]  [Hacks]  [Robotics]  [Science]  [Programming and Software]  [iPhone]  [Digital TV and Video]  [Intellectual Property & Copyright]  [Personal] 

27 February

I needed to record some screen casts for a presentation we are doing. The best tool I found on the Mac was Snapz Pro from Ambrosia Software, although I did find at last one other good tool if you wan to record a live video feed on top of the screencast.

For some of our work I wanted to record a screen dump of an automated process, and didn't want to sit there to baby sit the process. Snapz Pro records HUGE intermediate files, so it is important to stop the recording at the right time. In addition, since it does pretty slow post-processing, I wanted this to start automatically. Snapz Pro does not provide a mechanism to do that.

As a result, I wrote a couple of shell scripts to help:

- snapto redirects the recording cache for the program to an arbitrary destination. For my hour-long recordings, I needed to cache the intermediate files on a disk that could hold a 100GB file, unlike my system volume.

- sleepafter waits a specified amount of time and then either (a) puts the computer to sleep (so you can manually end the recording and start the save process later on), or (b) automatically ends the recording and starts the post processing using default settings.

These scripts are a bit hacky, but serve a useful purpose for me. If people want them the links are here. If you use these, or like them, please let me know.

They can be involved via the terminal in OS X. Using the syntax "sh snapto.sh"
or "sh sleepafter.sh".

snapto.sh is available as a shell script Version 1.0 was put in place Feb. 2011, but it might have been updated since.

sleepafter.sh is available is another shell script. Version 1.0 was put in place Feb. 2011, but it might have been updated since.

By Gregory Dudek at | Read (4) or Leave a comment |    
01 May

Grub's "out of disk" means the disk can't be figured out: here's how to fix it.

After installing a new Linux system on an old machine, I got grub's:

error: out of disk
grub rescue>

Here's what it means.

(This blog entry is not meant for casual reading, but for others who need to diagnose and fix this problem.)

Sigh. All I wanted to do today was do some work and some reading. Instead I spent the better part of the day getting an old linux server back on it's feet. Of course, I ended up in the typical rat hole whereby one problem leads to another until you are fixing a problem that is so deeply nested inside others that you can barely recall how it all started.

Well, when grub says "out of disk" there is a good chance it is related to a BIOS problem. Unfortunately, the error is not well documented in the official grub2 places. The simple story is that grub (or grub2) cannot properly understand the layout of the data on the disk.

Here are my suggestions:

1) Check your BIOS settings making note of any that alter a disk interface (i.e some BIOS version have an option for making a CD look like a hard disk, or altering the disk detection system).

2) make sure your disk partition is not too big, if the BIOS is an old one. In my case, I needed to keep my partition size below the critical magical 137G limit. In fact I created a 60G partition and my problem was fixed!

3) Check for a damaged disk structure. Either re-initialize the disk, or do a check (e.g. fscheck) on it.

4) If you are really desperate, consider updating your BIOS (this was one of the steps down the rat hole for me).

By Gregory Dudek at | Read (7) or Leave a comment |    
30 December

At last, an update to the long-standing Zope plugin, for use with Zope 2.11 and up

For a long time I have been using the Zope Photo product to create on-line image sets. I have been managing a branch on my own for Zope2, and realized I should release it since the original branch has become obsolete.

The Photo product (i.e. package) has been updated to version 4.0 and supports Zope 4.0, EXIF tags and generally works better with modern versions of Zope.
It is highly-configurable, template-based, and has stored backed by the Zope2 database engine. The original was developed by Ron Bickers and was later adopted and updated by Soren Roug (now Head of software development group at the European Environment Agency).

The plugin (product) has always provided multiple configurable sizes of the photo. Photo Folders provide a way to manage a group of Photo objects by providing a way to specify display sizes and properties for all contained photos.

Key changes are: more EXIF format support, better support for very large images, more EXIF GPS tags are supported, support for newer Zope methods, various bug fixes.

photo 4.0 product tarball for Zope2 can be downloaded here.

A simple example can be seen on-line, but it has not been configured to be pretty.

By Gregory Dudek at | Leave a comment |    
22 August

Today I (again) installed OpenCV 2.4.2 on OS X (Apple MacBook Pro under Darwin). As we all know, OpenCV is that great computer vision and image processing library that is also integrated into ROS. Getting it to compile on OS X, and to work with Apple's Python, can be a hassle (even with macports or homebrew, both of which have had problems when in comes to OpenCV).

The following special tricks were needed to get OpenCV to compile:

  • 1) Download OpenCV.

  • You need to have XCode and CMake installed.
    Go to the OpenCV folder.
    Edit CMakeLists.txt to add these lines( as per blog.yageek.net)

    list(APPEND HIGHGUI_LIBRARIES ${BZIP2_LIBRARIES} -lmp3lame -lbz2 -lfaac)

  • 2) Install libfaad

  • curl -L -o faad2-2.7.tar.gz http://sourceforge.net/projects/faac/files/faad2-src/faad2-2.7/faad2-2.7.tar.gz/download
    tar xvzf faad2-2.7.tar.gz
    cd faad2-2.7
    ./configure --disable-dependency-tracking CFLAGS="-arch i386 -arch x86_64" LDFLAGS="-arch i386 -arch x86_64"
    sudo make install
    cd ..

  • 3) Install lame from http://lame.sourceforge.net/:

  • tar xzvf lame-3.99.5.tar.gz
    cd lame-3.99.5
    sudo make install

  • 4) Install/update ffmpeg

  • curl -O http://ffmpeg.org/releases/ffmpeg-0.11.1.tar.gz
    tar xzvf ffmpeg-0.11.1.tar.gz
    cd ffmpeg-0.11.1
    ./configure --enable-libmp3lame --enable-libfaac --enable-nonfree --enable-shared --enable-pic --disable-mmx --arch=x86_64
    sudo make install
    cd ..

  • 5) Now, for Python compatibility...

  • Use ccmake -G "Unix Makefiles"

    Hit "t" for advanced mode.

    change PYTHON_LIBRARY value to /opt/local/lib/libpython2.6.dylib
    PYTHON_INCLUDE_DIR to /opt/local/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6.
    within ccmake.

    For me to have it work with Python, I also had to assure that the architecture was x86_64 not i386. When done, hit "c" to create the configuration, then "g"to go and create the Makefile.

  • 6) make; make install

  • Many of the special configuration issues for OS X and MacPorts seem to be automatically handled, but
    for MacPorts users, Taylor Squared Designs has a nice post on the issues involved in installing OpenCV.

    By Gregory Dudek at | Read (1) or Leave a comment |    
    09 October

    Still working as of 2016

    Streaming radio has been a common use of the Internet for many years. At one point, almost all audio streaming had only limited commercial implications. As the use of advertising has become more important as a source of revenue for radio stations, audio streams on the Internet have become entwined with players, typically flash-based, that serve advertising, collect user data, and provide other services. One side effect of this is that it can may it very difficult to debug the set of network connections being used. If, like me, you run a tight firewall, you need to have a sense of where data is coming from, and going, to allow the ports to be opened and closed.

    Thus, I needed to figure out how audio streams were being sent from an audio station I wanted to listen to. The audio stream I wanted was being provided from streamtheworld.com and the actual transactions are handled by a rather complex mess of javascript and flash objects. Here's how to find the streams using Wireshark, a fancy network analysis tool.

    There are two steps which I elaborate below:
    1) Find the call sign for the station. With luck, you can guess this. This will often (but not always) be related to the station's familiar name. For example, for the station Q107 in Toronto the nationally registered call sign is CILQ and the callsign for streamtheworld.com is CILQFM; sensible. On the other hand, the streamtheworld identifier for CBMT CBC Radio One Montreal is CBC_R1_N_MTL_H.

    2) Find the playlist (.pls file) or stream ID.

    Call sign identification

    Firstly, note that Wireshark is a very powerful open-source network analyzer (formerly known as ethereal). It can be downloaded free for most platforms and runs using the X-windows windowing system (that works on Linux [default], Mac [standard but optional] and [with some effort] Windows).

    Turn on packet capture in Wireshark and then using your browser on the same computer load the page with the stream you want.

    Enter (i.e. specify) a filter as follows (see the picture):
    tcp contains "callsign"
    that will be used to "listen in" on the data exchange.
    This should return an HTTP GET statement of the probable form:
    GET /xmlconfig.php?view=xml&callsign=CILQFM&streamid=86
    and an associated full URI of the form:
    It's the callsign argument you want (in this case, CILQFM, for example).

    Getting the callsign using wireshark
    Getting the callsign using Wireshark (click to expand)

    Play list via wireshark

    At this point, you can just go to
    to get the playlist, which can be used to see what's really happening.
    Note the example use of the CILQFM call sign in the URL.

    Playing the stream without needing wireshark

    Once you have the play list, you can directly inspect it to determine the ports in use. You can also drag it into Apple's iTunes.

    If you know the callsign and just want to play the live stream, or see how this can be done programmatically, then there is a nice python script available from Dan McGee at http://www.toofishes.net/blog/streamtheworld-streams-command-line/.

    [ Originally posted 2012/10/15, with minor updates, Oct 2017, 2020. Apparently this still works. ]

    By Gregory Dudek at | Read (12) or Leave a comment |    
    23 December

    Try gregs-gsm-test.py for testing cell phone modules

    You can readily purchase stand-alone GSM/GPRS modules that provide cellular network access for computer applications without the involvement of any traditional cell phone. This post briefly discussed these modules and puts forward by little tool for debugging or evaluating them.

    I was looking at using such a module to let one of our robot vehicles literally phone home if it becomes lost. Several such modules are available, either as stand-alone cellular modems, or with additional computing integrated in the unit. One of the most full features is the Telit GM682-GPS that combines a cellular modem with GPRS (Internet access including a TCP stack), a GPS receiver, a microprocessor, and an embedded Python interpreter (this device also has many variants like the GM682-QUAD). Unfortunately, the GM682-GPS seems to have become hard to find.

    Command protocol

    These modules seem to almost universally use a protocol derived from the very simple one developed by the Hayes Microcomputer Products company for analog telephone modems, back in the 1980's. This "Hayes protocol" uses a set of short commands that each start with AT to control the modem. In fact, even in iPhone uses a variant this protocol to talk to its internal modem.

    GSM modules for sending SMS or making cell phone calls. Telit 682, ADH9066 and SM5100B modules shown

    GSM modules for sending SMS or making cell phone calls. Telit 682, ADH9066 and SM5100B modules shown (click to enlarge)

    Unfortunately, although the generic command set for various modem devices follows a common template, each manufacturer is free to add extra commands or implement only a subset of the common core. As a result, each GSP modem I looked at really needed a distinct set of commands to work best.

    Another annoying issue is that many of these models are GSM/GPRS only. Phone services and modems about that claim to be GPRS/3G/UMTS/HSDPA, as well as 3G, 4G, LTE and more alphabet soup, but these terms are often used imprecisely and inaccurately and, in fact, it's very hard to be sure if a company that supports HSDPA actually supports the simpler older 2G GSM protocol as well (e.g. Canada's Telus supports HSDPA but not GSM).

    Alternatives and issues

    The three I think are the best contenders for use in our systems as the Telit GM682-QUAD, the AHD ADH8066 and the LinkSprite SM5100B. These all do GSM 850/EGSM 900/DCS 1800/PCS 1900 and can send SMS messages, do GSM/GPRS, TCP/IP and voice.

    For use in embedded systems, it should be noted that the power use by these modules can shoot up to a heft 2 Watts when data is being transmitted. The SM5100B, for example, can run just off powered only off a USB cable until to attach a good quality multi-band antenna, and the it will probably just shut down as it attempt to actually register on the network.

    Importantly, the baud rate might need to be specified correctly; common values are 9600 and 115200. Also, the radio band to be used is critical and on at least one unit can only be set using a command that is not properly documented! Testing and adjusting these features was a bit tedious.

    My program for evaluating a module: gregs-gsm-test.py

    In attempting to evaluate these units, I also found many users on the net have trouble configuring them initially. The first time I touched one, it took a while before I figured out which sequences of commands was useful, and how to evaluate the unit. As a result, I created a small Python program (gregs-gsm-test.py) that uses the somewhat standard command vocabulary to determine the baud rate for a module, check out that it is working, and examine a few features.

    In it's current state this program is overkill for just testing my modules, but it also
    should be useful to others. New versions can be found here named
    gregs-gsm-test.py and (maybe) updated occasionally.

    The version number of the most recently released version can be found via this link.

    Typical program output

    python gregs-gsm-test.py -
    2.2 Copyright Gregory Dudek (c) 2012.
    gregs-gsm-test.py using /dev/tty.usbserial-A10137IB

    Seek device ID (being thorough, so it will take a few seconds)
    Spreadtrum Communication CO.,Ltd SM5100B-D

    SIM card status READY

    Signal strength (RSSI): 14
    Setting message format for text style.

    Network registration status
    ['+CREG:', '0,1']
    Registration status: registered, home network

    Currently selected operator:
    REPLY: ['+COPS: 0,2,302720', '', 'OK']

    Phone number stored on SIM:
    REPLY: ['OK']

    IMEI number:
    REPLY: ['35...deleted...5', '', 'OK']

    International Mobile Subscriber Identity (IMSI) of SIM card:
    REPLY: ['+CIMI: "30...deleted...9"', '', 'OK']
    302 720
    Canada Rogers
    Mobile Subscription Identification Number: 50...deleted

    SIM number:
    REPLY: ['+CCID: "89...deleted ...49"', '', 'OK']

    By Gregory Dudek at | Read (2) or Leave a comment |    
    Prev  1   2   3   4   5   [6]   7   Next